NRT Forest Disturbance Monitoring Intercomparison Challenge
A standardized benchmarking exercise evaluating operational remote sensing workflows across European forests.
1. Context & Objectives ▼
While a wide array of near-real-time (NRT) forest change-detection systems exists, evaluating their operational efficacy remains difficult due to divergent validation protocols and fragmented study areas. To address these challenges, the JRC convened members of the remote sensing land mapping community for an expert meeting on March 3, 2026. The core concept of this intercomparison challenge was pitched during the event and subsequently discussed and refined collectively with the participants. As a result of this co-design process, the initiative now establishes a common, transparent benchmarking framework to systematically assess the strengths and limitations of current methodologies across the European continent.
International research teams and algorithm developers are invited to run their processing workflows across a standardized assessment period. By centralizing validation against a common benchmark reference dataset, this challenge moves past self-reported accuracy boundaries to assess NRT disturbance monitoring capabilities for European forests in a rigorous, transparent and interoperable way.
This challenge is part of the European Commission's forest science partnership that aims to support the development of new forest indicators based on remote sensing.
2. Rules of Engagement ▼
The benchmarking window focuses on the historical year 2022. While multi-year historical time series are provided to build stable baseline states, the target disturbance flags must be generated dynamically as individual observations become available.
To keep the computational footprint accessible, results only need to be produced for the 100 designated
3. Sampling & Validation Design ▼
Two-Stage Clustered Sampling Framework
The performance assessment framework relies on a rigorous two-stage cluster sampling design. The first stage consists of 100 Primary Sampling Units (PSUs), structured as
Spatio-Temporal Response Design & Custom Interpretation Interface
To ensure defensible baseline validation data, the JRC is solely in charge of executing the final visual interpretation of the sample metrics (SSUs). This process uses a dedicated spatio-temporal response design interface, allowing human experts to cross-reference multi-sensor time series, high-resolution imagery, and spectral indices to confirm the exact date and characteristics of canopy modifications. Examples of these visual verification samples can be reviewed via the Visual Response Design Directory; please note that this public interface is hosted strictly for demonstration and methodological review, and participants are not expected to perform any manual interpretations.
Standardized Verification Metrics
Submissions will be evaluated by comparing predictions directly against reference samples using the nrt-validate utility package. Statistical calculations are strictly governed by established remote sensing accuracy guidelines:
- Timeliness Assessment: Governed by the detection-lag parameters established by Bullock et al. 2022
- Spatial Accuracy Assessment: Estimated via unbiased error-adjusted matrix estimation protocols defined by Stehman 2014.
4. Data & Technical Specifications ▼
The challenge infrastructure separates data inputs into required constraints and optional support sets. All reference layers use the standard European projection coordinate system: EPSG:3035.
| Resource Layer | Status | Technical Description & Core Mandate | Data Location / Source |
|---|---|---|---|
| Tree Cover (TC) Masks | REQUIRED | Provided at 10m, 20m, and 30m resolution. Participants must choose the layer matching their internal processing scale. This layer explicitly defines the official forest domain for evaluation and dictates grid alignment and CRS constraints. | .../data/forest_masks/ |
| Sentinel-2 L2A Cube | OPTIONAL | Structured in GeoZarr format for all Primary Sampling Units (PSUs), preserving native 10m and 20m bands. Provided solely for convenience; there is no obligation to use this data configuration. | .../data/sentinel2/ |
| Sentinel-2 SCL Mask | OPTIONAL | The native Scene Classification Layer is included. Use is entirely optional; participants are strongly encouraged to implement their own optimized cloud-masking workflows. | Included within the S2 GeoZarr structure |
| Alternative EO Data | OPTIONAL | Teams are free to utilize Landsat, Sentinel-1, Harmonized Landsat Sentinel (HLS), or standalone automated atmospheric preprocessing variations. | User-defined external sourcing |
5. Output Submission Format ▼
To pass automated evaluation, submission blocks must adhere strictly to the following parameters:
- Spatial Structure: Raster format only, perfectly snapped to the coordinate grid and extent of the respective mandatory Tree Cover mask.
- Data Type Encoding: Standard
Int16array. - Pixel-Level Values:
- Pixels with a detected canopy disturbance must report the detection date encoded as the number of days since 1970-01-01 (Unix epoch days).
- Undisturbed forest pixels must be encoded as
0.
- Attribution Layer (Optional): If your workflow outputs a disturbance driver classification map, use an accompanying
Int8raster, with legend. - Naming Protocol: File names must strictly follow the syntax layout below:
disturbance_date_{PSU_id}.tifdisturbance_type_{PSU_id}.tif (Optional)
6. Reference Execution Pipeline & Demonstration ▼
To assist participants in setting up their workflows, this section provides a conceptual blueprint for a fully reproducible verification sequence. This reference pipeline serves strictly as an illustrative demonstration; participants will replace the baseline algorithm shown here with their own custom monitoring workflows.
1 Result Generation (Example Baseline)
The demonstration pipeline ingests the 5 public demonstration PSUs. It applies the historical Interquartile Range (IQR) anomaly tracking algorithm bundled directly inside the core nrt package. This step outputs the raw disturbance date rasters according to the required Int16 Unix epoch day encoding rule.
2 Local Conformance Verification
Before evaluating performance, the generated rasters are passed through the JRC conformance validation script. This automated pre-screening ensures that grid snapping, coordinate reference system definitions (disturbance_date_{PSU_id}.tif) align perfectly with evaluation constraints.
3 Pseudo-Performance Assessment
Finally, a mock accuracy assessment is executed locally. The pipeline utilizes the open-source nrt-validate library to calculate preliminary precision and timeliness metrics. This assessment matches the prediction outputs against the 100 visually interpreted sample points (SSUs) specifically provided for these 5 demonstration zones.
7. Submission Protocol ▼
Submissions are managed via an institutional mailbox. Before sending your data, please use the provided verification script to check conformance of your results locally to confirm metadata and coordinate array alignment.
Package your raster directories as a standard compressed archive (.tar.gz or .zip) and submit them directly to: JRC-FOREST@ec.europa.eu
8. Frequently Asked Questions (FAQ) ▼
Are we forced to use the provided GeoZarr data cubes?
No. The GeoZarr Sentinel-2 cubes are provided strictly for convenience. You are completely free to use your own data pipelines, including alternative platforms like Landsat or Sentinel-1.
What happens if our output projection grid doesn't match EPSG:3035?
Submissions that deviate from EPSG:3035 or fail to align perfectly with the spatial resolution grid of the provided Tree Cover masks will fail the automated pre-screening conformance check and will not be evaluated.
Can we update our model parameters dynamically during the monitoring window?
Yes. Dynamic state or parameter adaptation (e.g., updating background variance using recursive updates, Kalman filter transitions, or running averages) is fully permitted as the system steps through 2022 chronological timelines. The only hard restriction is that the calculation at observation
How will I find out how my algorithm performed?
The JRC guarantees a transparent feedback loop. Once the independent evaluation phase is complete, every participating team will receive a comprehensive performance report.
Will the challenge results be published, and can I be included as a co-author?
Yes, absolutely. Our overarching goal is to advance the state of the science collectively. The JRC intends to synthesize the findings of this intercomparison exercise into a peer-reviewed scientific manuscript mapping the European NRT monitoring landscape. All contributing teams who submit a conforming dataset and brief methodology summary will be invited to join the publication as co-authors.