GEOGG142 GMES Calibration & validation of EO products Dr. Mat Disney Pearson Building room

2 Outline  Calibration & validation  Example: MODIS LAI and NPP products  Meaning of parameters??  Time, space, measurements?  Scaling? Good place to start: CEOS Working Group on Cal/Val, Land Product Validation sub-group: and see eg Biophysical & references therein: and good practice guidelines: 1.pdfhttp://lpvs.gsfc.nasa.gov/ 1.pdf

3 Calibration & validation? Calibration: –Process of adjusting empirical relationship between empirical estimates of biophysical parameter estimated from 2 (or more) sources –e.g. ground-based and EO-derived LAI, or NDVI and LAI, or NDVI and fAPAR etc. etc. –Local calibration to estimate fitting parameters (slope, intercept for a linear relationship) and uncertainty –Limitations???

4 Calibration & validation? Calibration: –process of converting an instrument reading to a physically meaningful measurement –Particularly radiometric calibration i.e. from DN to radiance measurement –OR –Process of adjusting empirical relationship between estimates of biophysical parameter estimated from 2 (or more) sources e.g. ground and EO-derived

5 Calibration & validation? Validation: –experiments designed to verify instrument measurements using independent measurements –Caveat: EO ‘validation’ often means testing one model-derived estimate against another EO LAI, NPP etc. all require models Field estimates of LAI also require models –i.e. NOT validation in true sense at all –See later: when is LAI not LAI BUT: cal/val both essential to scientific remote sensing

6 Aside: focus on validation here but ….  Eg LAI CEOS WGCV: recommends CALIBRATION comparison between EO and ground-based  NEEDS: reference estimates traceable to in situ measurements  3 sources available: 1.LAI measurements over individual Elementary Sampling Units (ESUs) 2.Spatially extensive LAI reference maps based on data driven relationships calibrated using ESU LAI 3.Spatially extensive LAI reference maps based on functional relationships calibrated using ESU LAI.  And what is an ESU? See later

7 Validation example: MODIS NPP  Productivity recap: Net Primary Productivity (NPP)  annual net carbon exchange  quantifies actual plant growth  Conversion to biomass (woody, foliar, root) –i.e. not just C0 2 fixation (GPP) –NPP = GPP – Ra (plant respiration) MODIS product example used here –MOD17 GPP/NPP ATBD ntsg.umt.edu/MOD17 –Turner et al (2005)

8 Productivity recap GPP/NPP from MODIS Requirements? MOD17 ATBD Running et al. (2004) Turner et al. (2005) Zhao et al. (2005) Heinsch et a. (2006)

9 MODIS GPP/NPP + QC??

10 MOD17 validation approach  Need to address time (days to years) and space (local to global)  Permanent network of ground validation sites  Quantify seasonal and interannual dynamics of ecosystem activity (cover time domain)  EO to quantify heterogeneity of biosphere  Quantify land cover, land cover change dynamics  Models to:  Quantify, understand unmeasured ecosystem  Provide predictive capability (in time AND space)

11 How on earth…..???? …can we “validate” an EO-derived estimate of something that depends on soil, climate, land cover etc.? Given that it requires various models to go from a satellite observation (radiance), to reflectance, to LAI/FAPAR, to PSN, to GPP to NPP At 500m-1km pixels. Globally. And how do you even “measure” NPP on the ground??

12 So, how might we validate? Need to consider scale Relate measurements at the small scale to 1km pixels?? Flux tower approach Eg BIGFOOT approach, FLUXNET etc. Measurements and validation at many scales Models to bridge time/space scales –(but how good are models…?) Fig from MOD17 ATBD

13 Ecosystem measurements: FLUXNET Fig from MOD17 ATBD

14 Ecosystem measurements: FLUXNET

15 Ecosystem measurements: FLUXNET

16 Ecosystem measurements: FLUXNET

17 Ecosystem measurements: FLUXNET by biome Some distribution of biome types, but clearly biased in location Even considering only limited biomes

18 BigFoot approach to validating MODIS NPP  E.g. Turner et al. (2005), 6 sites spanning range of vegetation and climate  Crops, forest, tundra, grassland  5 x 5 km site at each plot (25 MODIS pixels)  Flux tower & 100 (25x25m) sample plots within each area, seasonally measured for LAI and above-ground (A)NPP (from harvested leaf and wood material)  Land cover from high res EO  Use measured data at sample plots to calculate NPP, GPP  Spatially distribute across site using (vegetation-calibrated) BiomeBGC model  Requires daily met data, land cover, LAI  Gives measured estimate from ground AND flux tower

19 BigFoot v flux tower GPP Turner et al. (2005)

20 BigFoot v MODIS GPP Turner et al. (2005) Not such good agreement as for flux tower (not surprisingly)

21 Comparison of MODIS NPP with flux data Turner et al. (2005) Differences due to Ra (autotrophic i.e. plant respiration)? PAR, VPD differences between those from DAO and actual? (VPD = deficit between the amount of moisture in the air and how much moisture the air can hold when it is saturated)

22 DAO PAR, VPD? Turner et al. (2005) Clearly some sites better agreement than others PAR generally good (relatively easy to measure) VPD less so e.g. SEVI (desert grassland site) VPD Other issues?

23 MODIS-estimated v BigFoot FPAR Turner et al. (2005) How do you measure FPAR even on the ground?? Requires models to interpret measurements of radiation

24 MODIS-estimated v BigFoot LUE (light use efficiency) Turner et al. (2005) LUE inferred from flux data Again, hard to even measure this on the ground…..

25 Zhao et al. (2005) Heinsch et al. (2006)

26 Process/SVAT (soil-veg-atm-transport) models Fig from MOD17 ATBD

27 From Running et al. (2004) MOD17 ATBD Biome-BGC model predicts the states and fluxes of water, carbon, and nitrogen in the system including vegetation, litter, soil, and the near- surface atmosphere i.e. daily PSN Process models: how do we test/validate?

28 Process models: how do we test/validate? Fig from MOD17 ATBDhttp://

29 Canadell et al Data-Model Fusion [Using multiple streams of datasets with parameter optimization] C stock and flux measurements Inventory analyses Process-based information Climate data Remote sensing information CO 2 column from space Inverse modeling Process-based modeling Retrospective and forward analyses

30 Multi-level model/data validation MOD17 ATBD: Synergy of various carbon measurement programs Fig from MOD17 ATBD

31 How do we decide on ground-based sampling strategy, scale?  CEOS WGCV LPV : Elementary Sampling Units (ESUs)  “ …. a contiguous spatial region over which the expected value of LAI can be estimated through in situ measurement … corresponds to finest spatial scale of LAI estimates used for reference LAI maps.”  ESU size: “….at least as large as one measurement footprint of the in situ instrument and typically includes a number of instrument measurements.”  ESU size: “ …varies with surface condition, instrument field of view, illumination conditions (when transmission based measurements are used) and spatial sampling design.”  ESU size: “….should be sufficient to allow repeat visit with minimum uncertainty due to changes in illumination or geolocation.”

32 ESUs: CEOS WGCV LAI validation protocol

33 Summary  Calibration  Needed to allow comparison of data & products from multiple sensors & algorithms over time  AND/OR to  Can be done on-board, or via sensor intercomparison etc.  Validation example: NPP  Far removed from EO measurement & spatially, temporally variable  Requires: observation networks over time and space and measurement of met. & biophysical data  Models to interpolate spatially from ground-based, site-scale measurements  Testing and intercomparison of models  Ideally: optimal combinations of models + data across scales (e.g. via data assimilation)

34 References: NPP Running et al. (2004) A Continuous Satellite-Derived Measure of Global Terrestrial Primary Production, Bioscience 54(6), Ganguly et al. (2008a, b) Generating vegetation leaf area index earth system data record from multiple Sensors, RSE, 112, (Part II) and (Part I) Turner et al. (2005) Site-level evaluation of satellite-based global terrestrial gross primary production and net primary production monitoring, Glob Change Biol, 11, Zhao et al. (2005) Improvements of the MODIS terrestrial net and gross primary production data sets, RSE, 95, Heinsch et al. (2006) Evaluation of Remote Sensing Based Terrestrial Productivity From MODIS Using Regional Tower Eddy Flux Network Observations, IEEE TGRS, 44(7), General validation Morisette et al. (2002) A framework for the validation of MODIS Land products, RSE, 83, Disney et al. (2004) IJRS, 25(23),

35 Other cal/val links  NPP:  Cal/val programs  CEOS-WFGCV (Committee on EO Working Group on Cal/Val)     SAFARI2000:  VALERI:  NCAVEO:  JAXA:  Etc etc etc

36 Example: MODIS core val sites Justice et al. (1998) Privette et al. (2002) and RSE 83, 1-2, 1-359