DIAS for CAP Checks by Monitoring

The Joint Research Centre (JRC) Checks by Monitoring (CbM) is as a proof of concept that demonstrates the feasibility of agricultural monitoring systems for the common agricultural policy (CAP) of the European Union (EU) based on Copernicus Sentinel data on cloud-based infrastructures, particularly the Data and Information Access Systems (DIAS). Its goal is to help Paying Agencies (PA) of Member States (MS) and their technological partners to get started with CbM and to define a set of best practices in context of the Integrated Administration and Control Systems (IACS). The same approach can potentially be applied to the area monitoring system (AMS) of the future CAP (2023-2027).
 In this page we introduce DIAS and Sentinel in the context of the CAP and we give an overview of the goals, the architecture and the main elements of the solutions developed by GTCAP (JRC CbM). The detailed description of methods and tools is provided in the JRC CbM technical documentation. The code is made available as open source in the JRC CbM GitHub repository.

Main elements of the JRC CbM

The CbM solution proposed by JRC is based on two architecture layers, the backend and the frontend, that provide users with data and tools for their analysis.

  • The backend server provides the end-points to retrieve data: it includes the physical infrastructure based on DIAS and the routines that process Sentinel data and generates the information needed for the data analytics. It is developed, managed and maintained by a system administrator with expertise in cloud computing and big data analytics. More information on this layer is reported in the backend documentation.

  • The frontend is the intermediate layer that provides access to the data generated by the backend through standard Application Programming Interface (API). Users can retrieve and process data, both statistics and satellite images, ready to be analysed. They can get data using simple commands without backend expertise. More information is available in the frontend documentation.

  • The data analytics use data generated by and stored in the backend and made available through the frontend to perform CbM tasks, particularly to confirm/reject the farmer declaration on specific parcels in the context of a CAP scheme. A typical example is the use of time series of Sentinel data statistics extracted per declared parcel to detect markers that can be associated to agricultural activities. More information is provided in the data analytics documentation.

In the JRC CbM system the boundary between the backend, the frontend and the data analytics is fluid. Once new procedures are defined and tested by analysts, they can be integrated into the backend and automatically available to users as starting point for additional processing (e.g. analysis, reporting, classification).
 JRC CbM software modules are all open source and make use of open standards. The code is available in the JRC CbM GitHub repository.
 A complete overview of the system is illustrated in the JRC CbM architecture page.

DIAS and Copernicus Sentinel

The DIAS are cloud-based platforms providing centralised access to Copernicus data and information, as well as to processing tools (processing as a service) closely coupled with the data store. The European Commission has funded the deployment of five DIAS instances (CREODIAS, MUNDI, ONDA, SOBLOO, WEKEO) managed under contract with the European Space Agency (ESA) that facilitate and standardise access to the full archive of Sentinel-1 and Sentinel-2 data. The Copernicus Sentinels are a family of satellite missions for the operational needs of the EU Copernicus earth observation programme. Sentinel-1 provides all-weather, day and night radar imagery, nominally acquired every 6 days with 5 meters spatial resolution. Sentinel-2 provides multispectral high-resolution optical imagery with approximately 5 days revisit time and 10 meters spatial resolution. Their image archive provides free images of agricultural areas since June 2015 for all Europe.
 Each of the five DIAS competitive platforms also provides access to additional commercial satellite or non-space data sets as well as premium offers in terms of support or priority. Thanks to a single access point for the entire Copernicus data and information, DIAS allows the users to develop and host their own applications in the cloud, while removing the need to download bulky files from several access points and process them locally.
 A key service offered by the DIAS and essential for all Member States/Paying Agencies is the generation of Copernicus Application (or Analysis) Ready Data (CARD), which are data over the area of interest prepared for a user to analyse without having to pre-process the imagery themselves. It includes georeferenced, calibrated sensor data (Level 2). Sentinel 1 and 2 data are, by default, delivered by ESA through the Copernicus Hub as Level 1 data. Since March 2018, Level 2 (atmospherically corrected) Sentinel-2 data is also made available. For Sentinel-1, this is not the case (yet).

CbM in the context of the CAP

In 2018 Checks by Monitoring (CbM) were introduced for area-based support schemes as an alternative to classical on-the-spot-checks (Article 40a of implementation regulation (EU) 746/2018 of 18 May 2018) for the IACS managed by the European Union Member States. The CbM approach is based on the automated analysis on the whole population of aid recipients verifying the consistency of farmers’ aid applications with satellite-based evidence. The DIAS provide the relevant functionalities and an optimized use of Sentinel-1 and Sentinel-2 imagery for the continuous detection of agricultural conditions and activities on all the individual land parcels retrieved from the Land Parcel Identification System (LPIS) and the Geospatial Aid Application (GSAA). The information extracted from the Sentinel data (signatures) per individual parcel and date (time series) can be used to observe specific events (e.g. mowing, ploughing) or sequence of events (scenarios) when certain conditions are met (markers). This process results in a simplification of the huge Sentinel data set with a more efficient and largely automated check of all aid applications by Paying Agencies and, potentially, in the assessment and monitoring of the impact of the CAP measures.

JRC solutions for CbM

The ongoing implementation of the CAP and the development of a CAP post-2020 that comply with new EU commitments on climate change and sustainable development are priorities for the Joint Research Centre (JRC). In this framework, the JRC Directorate D5 and the Directorate General Agriculture and Rural Development - DG Agri signed an Administrative Arrangement for CAP Implementation Support Activities CAPISA 2 that covers the period from October 2020 to September 2022. One of the work packages (WP) is on supporting (onboarding) Member States/Paying Agencies on the deployment and use of DIAS cloud solutions for CAP area aid schemes. The GTCAP group in liaison with other JRC teams and in close consultation with DG AGRI:

  • validates the specific functional requirements of the CAP on cloud instances and addresses deployment issues including portability;

  • develops and operates a repository of shared technical solutions, best practices and algorithms, particularly on the implementation of methods based on earth observation (EO) signals;

  • provides training sessions to Member States/Paying Agencies on the use of DIAS for CbM;

  • implements and monitors contract for purchase of DIAS services;

  • advice on long term cloud-based solutions for CAP monitoring to ensure their future/continued availability.

The JRC solutions for the CbM data analytics are developed in the framework of the the GTCAP institutional project Outreach, that builds on the infrastructure development under CAPISA/CAPISA2 to benchmark CbM and, in extension, AMS application contexts. Outreach aims at producing good practices for Member States for the creation of a CbM system offering tools, support and a platform to explore Sentinel data and experiment algorithms to detect agricultural practices. It bundles the processing needs of a large selection of Member States on a JRC managed DIAS platform, in order to lower the technological deployment hurdles.
 JRC CbM has substantial differences compared to the ESA project SEN4CAP .

Contributing to JRC CbM

JRC CbM is the result of a collaborative effort between the JRC and many Member States. The code stored at https://github.com/ec-jrc/cbm is open to contribution. In the HOW TO CONTRIBUTE documentation page you can find all the information to raise an issue, report a bug and create a pull request to integrate new improvements. Contact us for if you need more details.