Moderate Resolution Sensor Interoperability (MRI) Initiative Committee on Earth Observation Satellites Moderate Resolution Sensor Interoperability (MRI) Initiative Gene Fosnight, USGS 2017 CEOS Chair Initiative USGS EROS – Sioux Falls, USA 16th May 2017

Moderate Resolution Sensor Interoperability (MRI) Initiative This initiative will include effort towards making optimal use of the increasing number of data streams available in the moderate resolution class, with a focus for 2017 on Landsat/Sentinel-2. As higher level, multi-sensor, time series products are developed, the integration of these products requires verification and validation of their interoperability, such as when are these products interoperable, when are they not interoperable, and under what conditions?

2017 Deliverables Develop a MRI Framework paper for moderate (10-100m) resolution interoperability, identifying multi-sensor interoperability concepts that need to be addressed for successful implementation of multi-sensor interoperable time series Address factors such as radiometry, geometry, data formats, browse, metadata, data access, metrics, and reporting A Landsat/Sentinel-2 interoperability case study document utilizing the MRI Framework, including lessons learned and best practices identified through the implementation and use of the Harmonized Landsat Sentinel- 2 (HLS) Surface Reflectance product

Deliverable #1: MRI Framework (Concept) Interoperability solutions follow two paths: Changes to products or post processing methodologies to create interoperable products - Harmonization radiometric cross calibration to standard references acceptance of common geographic reference grid and DEMs compatible atmospheric models Accommodation to inherent differences between products – Homogenization pixel size spectral response curves available bands

Deliverable #1: MRI Framework (Concept) Component Measure Threshold (Minimum) Target (Goal) Verification Results Next Steps General Metadata Geometric Specifications & Accuracy Radiometric Specifications & Accuracy Temporal Specification Per Pixel Metadata Cloud Cover and Shadow Land, Water, Snow/Ice and Vegetation Pre- maps DEM and Terrain Shadow Solar and View Angle Data Quality and No Data Atmospheric Model Inputs Radiometry Measurements Solar and View Angle Corrections Atmospheric Corrections Spectral Band Difference Corrections Geolocation Geometric Corrections Resampling

Interoperability Check List Do the products meet the CARD4L product family specification? Determine common map projection, origin and pixel size to meet user requirements. What are the geometric parameters of the merged product? Extract bands from each product appropriate for a merged product. Bands may be available for some, but not for other source data sets. What bands are used in the merged product for each sensor? Does the joint relative geometric accuracy among the products meet the interoperability criteria for the user application? What is the individual and joint geometric accuracies? If the relative geometric accuracy is not met, is it possible to perform an image-to-image registration? What methods are used to meet the user’s geometry criteria? Are the products all from the same product family, for example Surface Reflectance? If the products are not from the same product family, is the necessary atmospheric and geometric per-pixel metadata and methodologies available to create a consistent merged product. What atmospheric and BRDF models are applied? Does the radiometric accuracy of the input data sets meet the interoperability criteria for the user application? What are the radiometric accuracies of the products? Are spectral band adjustment factors available to correct significantly different spectral response curves? What are the factors and how are they applied?

Deliverable #2: Landsat/Sentinel-2 Interoperability Case Study The NASA Harmonized Landsat 8 Sentinel-2 (HLS) product generation addresses many of the interoperability issues laid out in the MRI Framework. The EC vegetation dynamics monitoring project using the HLS products in conjunction with the phenology algorithm will be the primary interoperability use case. Case study evaluation will utilize the MRI Framework and address multi-sensor interoperability concepts. Beyond HLS, agencies and users will be surveyed to obtain a more comprehensive list of ongoing efforts to use Landsat and Sentinel-2 data together. These projects can provide a basis for understanding “lessons learned” and best practices.

Harmonized Landsat Sentinel-2 (HLS) Project Merging Sentinel-2 and Landsat data streams can provide 2-3 day global coverage Goal is “seamless” near-daily 30m surface reflectance record including atmospheric corrections, spectral and BRDF adjustments, regridding Project initiated as collaboration among GSFC, UMD, NASA Ames

MRI: Relationships

Deliverable #2: Landsat/Sentinel-2 Interoperability Case Study HLS is a merged Landsat 8 Sentinel-2 Surface Reflectance product Machine readable metadata Resampled to 30-meter Sentinel-2 reference Cloud masks View and solar angle BRDF corrections Atmospheric correction Band pass adjustment Products available for over 747 MGRS tiles and covering 63 regions Document Best Practices and Lessons Learned through the HLS product generation and HLS Phenology Use Case

Cross-calibration issue or spectral band adjustment not optimal? HLS Example: Radiometry South Africa Cross-calibration issue or spectral band adjustment not optimal? NDVI

HLS Example: Crop Phenology South Africa Southern France NDVI NDVI time series from Landsat (red) and Sentinel-2a (blue) reflectance values show detailed crop phenology

HLS Example: Within Season Monitoring

MRI Team Co-leads Team Members Gene Fosnight USGS USGS chair team co-lead, Landsat, LSI-VC, SDCG Cindy Ong CSIRO WGCV co-lead, imaging spectroscopy Richard Moreno CNES WGISS co-lead, Copernicus Jeff Masek NASA Landsat Sentinel-2 Case study, LSI-VC Zoltan Szantoi JRC Adam Lewis GA LSI-VC, FDA, CARD4L Paul Briand CSA LSI-VC, GEOGLAM, SAR Yves Crevier SDCG, GEOGLAM, SAR Brian Killough SEO, LSI-VC, SDCG, FDA, CARD4L Debajyoti Dhar ISRO Optical/SAR data fusion Takeo Tadono JAXA LSI-VC, CARD4L, SAR Amanda Regan EC User needs for satellite constellations Nigel Fox UKSA WGCV IVOS Kurt Thome WGCV Andy Mitchell WGISS Kerry Sawyer NOAA Represent SIT vice chair Kevin Gallo LSI-VC, LPCS data integration Eric Wood USGS Represent USGS Chair Team

Roadmap Conduct Kickoff telecon (March 2017) Discuss MRI at LSI-VC-3 (March 2017) and at WGISS (April 2017) Identify Landsat/Sentinel-2 case study (early April 2017) Present MRI at SIT-32 (April 2017 ) Distribute working copy of MRI Framework document (May 2017) Discuss MRI at WGCV (May 2017) Distribute final draft of MRI Framework document for review (late July 2017 ) Provide HLS product interoperability evaluation and phenology use case for review (late August 2017) Present initial case study results at LSI-VC-4 and SIT TW (September 2017) Present final MRI Framework and case study results at CEOS Plenary (October 2017) Present proposed way forward at CEOS Plenary (October 2017)

Organization & CEOS Interactions The Moderate Resolution Sensor Interoperability (MRI) initiative is an activity under the Land Surface Imaging Virtual Constellation (LSI-VC) In particular, MRI team is also working closely with the LSI-VC CEOS Analysis Ready Data for Land (CARD4L) team CARD4L are satellite data that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets. CARD4L provides product family specifications, where examples are top-of- atmosphere reflectance, Surface Reflectance, Surface Temperature, SAR sigma or SAR gamma naught. MRI provides the basis for combining and comparing multiple products. LSI-VC is working in partnership with the Working Group on Calibration & Validation (WGCV) and the Working Group on Information Systems and Services (WGISS)

MRI General Metadata Items MRI concepts MRI Alternatives Coordinate Reference System Different pixel sizes, origins and projections require the data be spatially resampled. The greater the differences in pixel sizes the greater the effect on radiometry. Reproject to either the smaller or larger pixel size depending upon application requirements and common origin and projection. Geometric correction Different and less accurate reference grids will cause greater uncertainty in spatial alignment of pixels. Minimize absolute error of individual reference grids. When possible share reference grids across sensors. Geometric accuracy Spatial RMSE distributions for each sensor are needed to establish the joint distribution for multi-sensor time series. Reduce RMSE to the theoretical minimum per sensor in relationship to reference grid. Spectral bands Different bands between sensors require methodologies that can adapt to band availability. Different bands are inherent differences. When possible provide the user community the richest set of alternatives, even when it creates a discontinuity in the time series record for some bands. Spectral response curves Different spectral response curves exist for similar bands between sensors causing land surface dependent variability. Spectral Band Difference Factors (SBAF) can be used to adjust the radiometry to accommodate differences under many conditions. Radiometric Accuracy Biases and uncertainty need to be minimized between sensors using spectrally uniform references. Cross calibration using uniform reference surfaces can minimize overall biases between sensors. Revisit time & lifetime Multi-sensor time series extents time series and increases the temporal density at the cost of increase variability in the signature.   Longer time series permit long-term change to be monitored. More frequent revisit times increases the probability that land cover change can be detected by decreasing the sampling interval and by increasing the probability of cloud free data.

MRI Per-Pixel Metadata Items MRI concepts MRI Alternatives Clouds Verified and validated cloud masks are needed to estimate radiometric contamination for specific and known cloud characteristics. Known confusion such as with snow/ice needs to be quantified. Verify and validate cloud algorithms. Share algorithms to minimize variability when appropriate. Different band availability will cause differential results. These differences need to be documented Cloud Shadow Verified and validated cloud shadow masks are needed to estimate radiometric contamination for specific and known cloud characteristics. Known confusion such as with water needs to be quantified. Cloud shadows contain spectral information that can be used within many applications. Adaptive algorithms can use the masks appropriately. Land/water mask Simple premapping of land and water provides useful information for other radiometric corrections. Consistently handle differential corrections over land and water. Snow & Ice masks Premapping of snow and ice assists in cloud cover assessment. Time series are used both in the detection of clouds and in the monitoring of snow and ice. Verified and validated uncertainty estimates improve both applications. DEM The required accuracy of the DEM is dependent upon the corrections implemented, swath width and pixel size. Share DEMs when possible both among operational agencies and with users. Requirement are highly variable. Terrain Shadow mask Terrain shadow masks are needed to estimate radiometric contamination associated with shadows. Known confusion such as with water needs to be quantified. Terrain shadows are particularly important for mountainous areas, wide swaths and for SAR sensors. Terrain shadows contain spectral information that can be used within many applications. Adaptive algorithms can use the masks appropriately. Illumination and Viewing geometry Solar illumination angles are needed for reflectance calculations. Solar and View angles are needed for BRDF related corrections including terrain illumination corrections. Define minimum correction versus full BRDF corrections. Issues opportunities, uncertainties. Data Quality No data, saturated, contaminated, terrain occlusion pixels need to be identified. Distinguish between no data and contaminated data. Document how these pixels are handled during resampling.

MRI Measurement Data Items MRI concepts MRI Alternatives Measurements Absolute calibrated measurement units with or without corrections below Minimum requirement is reflectance calibrated and validated to a known absolute source. Atmospheric, solar angle and viewing angle, SBAF corrections are usually needed to minimize variability and uncertainty to create an interoperable product. Measurement nomalisation Radiometry viewed through time is significantly impacted by variation in solar and viewing angles Solar and viewing angle corrections compensate for both within and between sensor variability. Aerosol, water vapor and Ozone corrections Different atmospheric models can introduce significant between sensor variability. Atmospheric model must either be shared or validated and verified to common reference. Available bands and ancillary data will impact quality of corrections SBAF corrections Different spectral response curves will introduce biases between instruments. Spectral Band Adjustment Factors (SBAF) compensate for different spectral response curves and will be application and surface type variable.

MRI Geolocation Items MRI concepts MRI Alternatives Geometric Corrections Mis-registration between images and between reference databases introduce variability in the radiometry measurements Minimize misregistration through orthorectification and precision registration to a controlled reference grid Resampling   The number and type of spatial resampling will impact the radiometric signal Minimize the number of resampling operations and consider impact objects such as contaminated pixels including clouds.

Interoperability Case Study Where does CARD4L end and interoperability begin? These steps standardise the data to be ‘analysis ready’. They are consistent with CARD4L but exceed the threshold specification (BRDF) Subsequent steps are specifically blend these two data streams. The method of blending is not unique; other approaches are also possible.