Outlier Analyses What is an outlier? data point unrepresentative of its general location or otherwise “difficult” to represent by a generalized NPP model. Outlier analysis tests used to assign individual flags:  outside of reasonable fixed limits for variables (e.g., NPP > 2000 gC/m2)  unrepresentative site for model comparisons (e.g., crop, pasture, plantation or wetland sites)  extreme values within a biome, (e.g., outside of.95 percentiles)  inconsistencies at a site, such as precipitation reported for the site being different than the precipitation derived from global climate data  comparison of measured versus modeled ensemble NPP (average NPP from all models): Bias, Normalized Error, and Mean Absolute Error  outliers based on regressions with driver variables: NPP vs evapotranspiration (AET), NPP vs precipitation, and NPP vs temperature  unrepresentative based on visual inspection and knowledge of sites Overall site flag:  overall flag calculated as weighted sum of individual flags  minor outlier comparisons: weight = 1  major outlier comparisons: weight = 10  managed sites: weight = 100  sites with flag > 5 were excluded Outlier Analyses What is an outlier? data point unrepresentative of its general location or otherwise “difficult” to represent by a generalized NPP model. Outlier analysis tests used to assign individual flags:  outside of reasonable fixed limits for variables (e.g., NPP > 2000 gC/m2)  unrepresentative site for model comparisons (e.g., crop, pasture, plantation or wetland sites)  extreme values within a biome, (e.g., outside of.95 percentiles)  inconsistencies at a site, such as precipitation reported for the site being different than the precipitation derived from global climate data  comparison of measured versus modeled ensemble NPP (average NPP from all models): Bias, Normalized Error, and Mean Absolute Error  outliers based on regressions with driver variables: NPP vs evapotranspiration (AET), NPP vs precipitation, and NPP vs temperature  unrepresentative based on visual inspection and knowledge of sites Overall site flag:  overall flag calculated as weighted sum of individual flags  minor outlier comparisons: weight = 1  major outlier comparisons: weight = 10  managed sites: weight = 100  sites with flag > 5 were excluded EMDI Initial Results: 11 Models and Field NPP Data at 87 sites Measure d NPP Comparison between Temperature reported for sites and estimated from gridded climate 3. Conduct outlier assessment of NPP and model driver data Outlier Analysis Results Number of records Steps in Outlier Analysis Class A B C Initial NPP measurements, sites with multiple measurements NPP measurements passing outlier analysis Percent measurements that were excluded 7% 29% 5% Mean NPP measurements for unique sites excluding outliers Class ASites Class C Cells Class B Sites EMDI Modeling Groups ModelModelersInstitution AVIMJinjun JiInstitute of Atmospheric Physics, China BGCPeter ThorntonUniversity of Montana, USA CARAIBBernard NemryUniversity of Liege, Belgium CENTURYBill PartonColorado State University, USA GLO-PEMSteve Prince, University of Maryland, USA Daolan Zheng GTECMac PostOak Ridge National Lab., USA IBISChris KucharikUniversity of Wisconsin, USA LPJVictor Brovkin.Potsdam Institute for Climate Impact Galina Churkina Research, Germany PNetJohn AberUniversity of New Hampshire, USA SIB2Jörg KadukCarnegie Institution, Stanford, USA STOMATENicolas ViovyCNRS, France VECODEVictor BrovkinPIK, Germany 4. Provide model driver data for sites to modelers 5. Run global/regional models for sites without fine tuning models 4. Provide model driver data for sites to modelers 5. Run global/regional models for sites without fine tuning models Summary What have we learned?  problems in NPP and driver data were identified and resolved  there was general agreement between model predictions of NPP and NPP field measurements  there were obvious differences that indicated areas for potential data and model improvement  model data comparison is a complex task that we are only beginning  unique database of terrestrial productivity information will be available for modelers and ecologists Summary What have we learned?  problems in NPP and driver data were identified and resolved  there was general agreement between model predictions of NPP and NPP field measurements  there were obvious differences that indicated areas for potential data and model improvement  model data comparison is a complex task that we are only beginning  unique database of terrestrial productivity information will be available for modelers and ecologists What’s next?  identify and explain outliers in both data and model predictions  re-run model simulations with new land cover and climate datasets, with outliers excluded  assemble multiyear NPP measurements from CO2 flux towers, long-term research plots (e.g., LTER projects, the ORNL Throughfall Displacement Experiment), forest inventory remeasurement data, and agricultural production data  EMDI-2 workshop to evaluate interannual variation, spring 2001 What’s next?  identify and explain outliers in both data and model predictions  re-run model simulations with new land cover and climate datasets, with outliers excluded  assemble multiyear NPP measurements from CO2 flux towers, long-term research plots (e.g., LTER projects, the ORNL Throughfall Displacement Experiment), forest inventory remeasurement data, and agricultural production data  EMDI-2 workshop to evaluate interannual variation, spring 2001 Initial problems identified and resolved by EMDI participants: inconsistent assignment of biome/vegetation type to Class A and B sites inconsistent NPP units of measure in Class C sites error in extrapolation process for site-specific monthly climate lack of consistent set of sub-monthly climate data Preliminary results: general agreement between model predictions and field measurements obvious differences that indicated areas for potential data and model improvement 6. Compare model results with field measurements