1. CROP-CIS User utility assessment of Geoland2 BioPar products Comparison of G2 BioPar vs. JRC-MARSOP SPOT- VGT NDVI & fAPAR products M. Meroni, C. Atzberger, O. Leo. JRC-MARS

2. G2 Interim Review Meeting, JRC Ispra 2 15/12/2011 Index  Objective of the analysis  Methods (spatial and temporal analysis)  Data and study areas  Main results of the comparison  Ongoing activities on BP full archive (JRC + IGIK)

3. Objective G2 Interim Review Meeting, JRC Ispra 3 15/12/2011 To provide a first assessment of new BioPar products by comparison with the “well known” JRC-MARSOP using a comprehensive statistical protocol  The analysis can:  describe existing differences between the two datasets  identify and point out inconsistencies in a specific product  provide a basis for more in-depth analysis at specific locations / times  The analysis can’t:  say which product is best! (validation is required for this purpose)

4. Methods G2 Interim Review Meeting, JRC Ispra 4 15/12/2011  Analysis of spatial and temporal agreement separately  Spatial comparison  Compare layer by layer  Summarize this comparison by a metric (e.g. R 2 )  Plot the metric across time (possibly stratified by land use classes) The result of the spatial comparison is a time series Data cube 1 Data cube 2 x y z data 1 data 2 Land cover class aclass bclass c

5. Methods G2 Interim Review Meeting, JRC Ispra 5 15/12/2011  Compare pixel by pixel  Summarize this comparison by a metric (e.g. R 2 )  Plot the metric across space (possibly deriving some summary statistics of such maps) The result of the temporal comparison is a map  Temporal comparison Data cube 1 Z1Z1 Z2Z2 Z1Z1 Z 2 Data cube 2 Land cover

6. Data: SPOT-VGT 10-day G2 Interim Review Meeting, JRC Ispra 6 15/12/2011  Geoland2 BioPar data vs.  JRC-MARS data  MARSOP-FS for the global window (FOODSEC action)  MARSOP-A4C for the extended European window (AGRI4CAST action), original and filtered (mod-SWETS)

7. G2 Interim Review Meeting, JRC Ispra 7 15/12/2011 Biopar compositing window  dekadal products: composites updated every 10 days;  30 days compositing* window is asymmetric around the “most representative” day (16 day before and 13 after it, equally weighted); Considering the required processing time, the overall delay for data delivery is 16 days (MARSOP delay = 8 days) An issue for MARS NRT applications * Note that for BP the term “compositing” is not fully appropriate because the value assigned to the dekad is derived from the inversion of the linear reflectance model of Roujean et al. (1992) applied to normalize the bidirectional effects during the synthesis period of 30 days.

8. G2 Interim Review Meeting, JRC Ispra 8 15/12/2011 Data: SPOT-VGT 10-day  Time domain:  2 years of BP GEOV1 demo products available (2003 and 2004)  Spatial domain: Three 10° x 10° BioPar tiles (1120 x 1120 pixels) selected in different agro-climatic regions monitored by JRC-MARS:  France (temperate - Mediterranean);  Brazil (tropical);  Niger (arid).

9. RESULTS – Cloud screening G2 Interim Review Meeting, JRC Ispra 9 15/12/2011  Fraction of valid observations (examples using fAPAR) MARSOP BIOPAR NIGER (semi-arid) BRAZIL (tropical-humid) Similar in arid areas, MARSOP>BIOPAR in humid areas

10. RESULTS – Cloud screening G2 Interim Review Meeting, JRC Ispra 10 15/12/2011  Temporal profile of fraction of valid observations MARSOP BIOPAR NIGER (semi-arid) BRAZIL (tropical-humid) NIGER (semi-arid) BRAZIL (tropical-humid) MARSOP>>BIOPAR in cloudy/rainy periods Cropland, Niger Cropland, Brazil

11. RESULTS – Cloud screening G2 Interim Review Meeting, JRC Ispra 11 15/12/2011  BP has consistently lower fraction of valid observations;  Difference is large for Brazil (severe cloud cover) and small for Niger (low cloudiness);  Unrealistic drops in MARSOP temporal profiles. Cloud screening algorithm applied by BIOPAR is more conservative and realistic (.. larger compositing window for BIOPAR..) When MARSOP shows unrealistic drops, BIOPAR is missing or not/less affected  Both NDVI and fAPAR, MARSOP-FS and -A4C:  Temporal profile of fAPAR (pixel of forest, Brazil)

12. RESULTS – Overall agreement (space and time pooled together) G2 Interim Review Meeting, JRC Ispra 12 15/12/2011 ~70% FAPAR < 0.5 ~30% FAPAR < 0.5  Example for fAPAR  FS and A4C data: fAPAR MARS < fAPAR BIOPAR  Largest differences observed for France (A4C)  statistically significant differences between distributions were found for Brazil (MARSOP-FS) and France (MARSOP- A4C) ECDF, example for all land cover classes (pooled together) NIGER BRAZIL FRANCE % of pixels showing statistically different data distribution BP Vs. MARSOP-FS BP Vs. MARSOP-A4C

13. RESULTS – Overall agreement (space and time pooled together) G2 Interim Review Meeting, JRC Ispra 13 15/12/2011  Correlation (example for fAPAR) Regional differences in OLS coefficients:  Niger: very small offset and slope greater than 1; similar profile minima, larger BIOPAR maxima;  Brazil: positive offset and slope close to 1. BIOPAR is consistently higher than MARS;  France: large offset and slope smaller than 1. Highest differences between the two datasets are found for low fAPAR (wintertime values). NIGER BRAZIL FRANCE Density scatter plot and linear regression (BIOPAR = intercept + slope * MARS) BP Vs. MARSOP-FS BP Vs. MARSOP-A4C

14. RESULTS – Spatial comparison G2 Interim Review Meeting, JRC Ispra 14 15/12/2011  Temporal evolution of spatial agreement (fAPAR of forest land cover)  Mean profiles  Large systematic component of the difference  Seasonality in spatial Agreement Coefficient (Ji & Gallo, 2006) rainy season winter time rainy season BP Vs. MARSOP-FS BP Vs. MARSOP-A4C Spatial AC varying over time. What’s the source of this variability/scatter?

15. RESULTS – Spatial comparison G2 Interim Review Meeting, JRC Ispra 15 15/12/2011  Factors contributing to the scatter:  Different cloud screening effectiveness (example on fAPAR) BIOPARFS

16. RESULTS – Spatial comparison G2 Interim Review Meeting, JRC Ispra 16 15/12/2011  Different cloud screening effectiveness (example on fAPAR)  Presence of brightness contrast in MARS-FS (example for NDVI, Woodland, Niger) due to BRDF Dekad 22 MARSOP-FS BIOPAR  Factors contributing to the scatter:

17. RESULTS – Spatial comparison G2 Interim Review Meeting, JRC Ispra 17 15/12/2011  Factors contributing to the scatter:  Different cloud screening effectiveness (example on fAPAR)  Presence of brightness contrast in MARS-FS (example for NDVI, Woodland, Niger) due to BRDF  «Unexpected» wintertime BIOPAR signal (France)

18. RESULTS – Temporal comparison G2 Interim Review Meeting, JRC Ispra 18 15/12/2011 BP Vs. MARSOP-FS BP Vs. MARSOP-A4C  Temporal agreement  Regions of very low agreement in Niger and Brazil can be explained Spatial distribution of Agreement Coefficient (example for fAPAR) NIGER BRAZIL FRANCE AC Arid area with very low fAPAR variability Areas with high cloud cover Low agreement over large areas

19. RESULTS – Temporal comparison G2 Interim Review Meeting, JRC Ispra 19 15/12/2011 Starting from the assumption that fAPAR varies smoothly over vegetated land we investigated the temporal smoothness of the two datasets. mean absolute value of the first derivative of fAPAR over time  40% of FS absolute dekadal variation > 0.05 FAPAR units  such frequency is implausible in the given geographical setting. BP appears more realistic. Mean |fAPAR′| Forest Cropland MARSOP-FS BioPar  Temporal smoothness, example of Brazil (MARSOP-FS)

20. Conclusions on 2 years of demo data G2 Interim Review Meeting, JRC Ispra 20 15/12/2011  BP “Compositing” window may be problematic for MARS NRT application  Significant differences between BIOPAR and MARSOP (both spatial and temporal variability)  The differences in cloud screening effectiveness and compositing method make BioPar products more realistic than MARSOP-FS  Same holds true for MARSPO-A4C. However, positive BP anomalies in wintertime deserve further investigation Scientific paper submitted to INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION

21. Ongoing activities [JRC] G2 Interim Review Meeting, JRC Ispra 21 15/12/2011  Intercomparison of the products extended to the full archive (1999-2011):  Statistical approach (similar to that described so far)  Operational approach (simulating actual MARS operations):  Analysis of vegetation anomalies  Bulletin production: differences in data quality (with BP taking more observations into account) against delivery time (with MARSOP data being in principle “more recent”)  Within-season crop yield predictions in Tunisia: evaluate possible performance improvements using BioPar data instead of MARSOP

22. Anomaly analysis, preliminary results G2 Interim Review Meeting, JRC Ispra 22 15/12/2011  Overall correlation (example for France, MARSOP-A4C)  fAPAR from HIST archive (1998-2010)  Focus on anomalies as z-scores, i.e. normalization of each dekad as distance from mean expressed in SD units  Low correlation of z-scores

23. Anomaly analysis, preliminary results G2 Interim Review Meeting, JRC Ispra 23 15/12/2011  Example of time profiles  MARSOP and BP: roughly parallel development, but important scatter

24. Anomaly analysis, preliminary results G2 Interim Review Meeting, JRC Ispra 24 15/12/2011  Trend analysis (France) For each dekad, the z-score values of all pixels are averaged (one line for each dekad).  BP shows a clear positive trend with time, not visible in MARSOP. Is this greening really happening? MARSOP BP 0 2 z -2 z 0 2 z -2 z

25. Anomaly analysis, preliminary results G2 Interim Review Meeting, JRC Ispra 25 15/12/2011  Example of application: detection of known droughts Monthly averages of fAPAR over France 2003 (heat waves between May and August)  Good agreement between datasets, both see the anomaly, spatial pattern more plausible for BP MARSOP BP AprilMay JuneJuly AugustSept AprilMay JuneJuly AugustSept 0 2 -2 Z-score

26. Wheat yield forecasting in Europe. Comparison of performances using G2 BioPar and MARSOP time series, preliminary results. Katarzyna Dabrowska-Zielinska IGIK, Institute of Geodesy and Cartography, Warsaw (Poland) User utility assessment of Geoland2 BioPar products

27. Objective Test the performance of MARSOP and BioPar for wheat yield monitoring/forecasting in Europe Data RS: dekadal SPOT-VGT NDVI and fAPAR from HIST archive (1999-2009) Yield: Regional Agricultural Statistics Database of EUROSTAT

28. Calibration of a Partial Least Square model tuned at NUTS2 level The explanatory variables are all the RS dekadal observations extracted from the growing season period as defined on the basis of an agro-climatic classification. Agro-climatic zones in Europe (Iglesias, A. et al., 2009) Two modes of operation of the model: Monitoring mode: (yield estimation after EOS) all dekadal RS indices of growing season are available Forecasting mode: (yield estimation within season) unknown dekadal indices are set to their long term average values Methods SOS & EOS

29. Results in monitoring mode Best performances are marked in yellow Yield estimates over 1999-2009, comparison BioPar, MARSOP and “null model” (mean yield): Cross-validation (Jackknifing) prediction errors (RMSE, MPE, MAPE) for agro-climatic zones The model doesn’t outperform the “null model” in all regions Small performance differences using either MARSOP or BioPar

30. Results in monitoring mode The largest errors in absolute terms are observed in Southwest of Europe and in the most northern region of Finland; Again small performance differences using either MARSOP or BioPar. Spatial distribution of the error (MPEs)

31. Results in forecasting mode No substantial differences between MARSOP and BioPar Forecasted yield performs better than simple average in few regions only (red bars shorter than the blue ones) Example of forecast for year 2009 using data 1999-2008. Model performances (DecMAPE, mean absolute forecast error) and compared to the “null model” (the mean yield).

32. SUMMARY No statistical differences in predicting wheat yield using either MARSOP or BioPar data. The differences in crop yield prediction are minimal and in favour of BioPar [MARSOP] in monitoring [forecasting] mode; Overall, poor performances of the model, especially when used in “forecasting mode”. This behaviour could be explained by the short RS time series available (11 years) and the huge gaps in EUROSTAT yield data; Current activities: investigation of different methods for region grouping (period of forecast); collection of more ground truth EUROSTAT data