Data analytics

The JRC CbM data analytics is the environment where the information generated by and stored in the backend and made available through the frontend is used to perform typical CbM tasks with the goal to confirm/reject the farmer declaration on specific parcels in the context of a CAP scheme. In this page with describe the main JRC CbM data analytics tools developed so far, particularly for the identification of agricultural activity based on marker detection and for advanced data reporting. Here you can find the links to the relevant documentation of each tool with examples of the use of the data generated by the JRC CbM system. An introduction to the JRC CbM system is provided in the OVERVIEW OF THE JRC CbM SYSTEM ARCHITECTURE PAGES. Data analytics is based on the data stored in the JRC CbM backend, described in the BACKEND PAGES, and accessed through the JRC CbM frontend, documented in the FRONTEND PAGES.

Goals

The main functions of the JRC CbM data analytics are:

• to provide a set of tools and a simple interface for decision makers with limited technical background in coding and remote sensing for the analysis of CbM data

• to enable analysts to develop and apply algorithms for the detection of agricultural practices

• to ensure full reproducibility of decisions

• to build out an augmented application logic

Links to the technical documentation

1. MARKER DETECTION

2. CALENDAR VIEW

3. FOI ASSESSMENT

4. MACHINE LEARNING

Marker detection

One of the key tasks in CbM is the identification of agricultural activities at parcel level. We created a flexible and extensible computational framework based on Python that analyse Sentinel-1 and Sentinel-2 time series generated by the backend for each parcel. This tool detects markers (observations of a relevant bio-physical change of state, made on the Sentinel signal, on a given date) that can be associated to agricultural practices. The processing architecture is built on four main modules: 1) import of band statistics, 2) data preprocessing, 3) identification of relevant changes in the signal time series (marker), 4) aggregation of markers from different sensors and association with land management practice, through the relevant bio-physical stages. The code, stored in the JRC CbM GitHub repository is constantly under development and new features are frequently added.

In the Overview page, we illustrate the goals of the tool, the target users, the technical skills needed to use it and how it fits into the CbM logic. We also introduce the main modules and the structure of the code.

In the Modules page, we provide a detailed description of the 4 main modules and how they interact to produce the final output. Links to the Python code helps to explore the logic of the framework.

In the Option file page, we describe in details what option files are, their structure and how to create them, with links to practical examples of settings, illustrated by diagrams.

In the GUI page, we present the graphic user interface that we created to facilitate the interaction with the tool for user with limited knowledge in Python coding.

In the Examples page, we list the Jupiter notebooks that we created with practical examples of marker detection.

Applications

In this section we show some complex workflows that implement use cases. Here the frontend tools for analysts are used to generate exmples of typical outputs or carry out necessary tasks for the CbM process.

In the Calendar view page, you learn how to use the run_calendar_view.py script, based on the calendar_view Python package, to download, process and display Sentinel-1 and Sentinel-2 data products for specific parcels. The graphical outputs has been designed and optimized to provide an immediate and intuitive access to both temporal and spatial dimensions of Sentinel-derived data, including time series of image statistics (per parcel) and structured sequences of Sentinel chips.

The FOI assessment page describes the concept of Feature of Interest in the context of the JRC CbM and its characteristics. It describes the tests that can be run to verify that a FOI (area of declared parcel) corresponds to an homogeneous area and that it does not contain non-agriculture features.

In the Machine learning (ML) page we discuss the use of ML TO check the agricultural parcels information given in the farmers’ declarations by generating relevant markers that confirm compliance with the scheme under which the declaration is made. ML can exploit the information provided by the continuous, territory-wide Copernicus Sentinel. We show examples that are complementary to classification, applying machine learning to the parcel averages. The documentation explains hot to prepare the data set, running and testing ML and checking results.

Expertise required

Analysts in Member States/Paying Agencies that work on CbM data analytics:

• working knowledge of Python coding

• knowledge of CAP schemes

• knowledge of basic principles of agriculture monitoring with remote sensing