1. The North American Carbon Program Site-level Interim Synthesis Model Data Comparison (NACP Site Synthesis) Daniel Ricciuto, Peter Thornton, Kevin Schaefer, Kenneth Davis Flux Tower PIs Modeling Teams NACP Site Synthesis Team

2. Site Synthesis Objectives Activity initiated in 2008 by NACP to answer: - Are the various measurement and modeling estimates of carbon fluxes consistent with each other - and if not, why? Quantify model and observation uncertaintyQuantify model and observation uncertainty 58 flux tower sites; 29 models 58 flux tower sites; 29 models Gap-filled observed weather Gap-filled observed weather Observed fluxes, uncertainty, ancillary data Observed fluxes, uncertainty, ancillary data Link model performance to model structureLink model performance to model structure Which model characteristics associated with “best” models? Which model characteristics associated with “best” models? How does this performance vary among sites? How does this performance vary among sites?

3. Flux Tower Sites AmeriFlux sites over 35 sites Data provided by CDIAC Standardized “Level 2” format Canadian sites Over 15 sites Data provided by - La Thuile synthesis activity - FLUXNET Canada Site selection based on: Representativeness of biomes Length of record Quality of data - gap fraction Ancillary data availability Meteorological drivers and flux observations gap-filled by NACP synthesis team

4. Models Results submitted from 22 models to dateResults submitted from 22 models to date On average 10 simulations per siteOn average 10 simulations per site Total of over 1000 simulated site yearsTotal of over 1000 simulated site years

5. Analysis Projects

6. Selected results Observed flux uncertainty (Barr et al.) NEE: random, U* filtering, gap-fillingNEE: random, U* filtering, gap-filling GPP & Re: random, U* filtering, gap-filling, partitioningGPP & Re: random, U* filtering, gap-filling, partitioning Random UncertaintyU* Threshold Uncertainty

7. Selected results Overall model performance (Schwalm et al.) Taylor Skill Normalized Mean Absolute Error Chi-squared Based on monthly model-data differences Large spread among models, sites Perfect Model

8. Taylor Skill by Model Characteristics (Schwalm et al.[2010])

9. Spectral NEE Error (Dietze et al.) Diurnal Annual Noise level based on NEE observation uncertainty Largest errors associated with diurnal and annual cycles Large variation in performance at synoptic scales

10. Phenology (Richardson et al.) Harvard ForestHarvard Forest Leafout too earlyLeafout too early Senescence too lateSenescence too late Errors of 25-50 days based on NEEErrors of 25-50 days based on NEE Errors in GPP/NEE correlated with LAI in spring, but not autumnErrors in GPP/NEE correlated with LAI in spring, but not autumn

11. Future work Objectives for new simulationsObjectives for new simulations Non steady-state Non steady-state Previous simulations assumed steady state, not consistent with observed fluxes Previous simulations assumed steady state, not consistent with observed fluxes Incorporate known information about disturbance history Incorporate known information about disturbance history Under-analyzed biomes Under-analyzed biomes e.g. wetland, tundra e.g. wetland, tundra Model sensitivity analyses Model sensitivity analyses Good idea of inter-model uncertainty, but intra-model uncertainty? Good idea of inter-model uncertainty, but intra-model uncertainty? What are the key parameters? What are the key parameters? Recruit more modeling teamsRecruit more modeling teams Invite wetland modeling teams Invite wetland modeling teams Expand number of IPCC GCMs Expand number of IPCC GCMs Coordinate with other synthesesCoordinate with other syntheses LBA DMIP LBA DMIP NACP regional interim synthesis, MsTIMIP NACP regional interim synthesis, MsTIMIP Make our database more visible, user-friendlyMake our database more visible, user-friendly 29 potential analysis teams making use of interim synthesis dataset 29 potential analysis teams making use of interim synthesis dataset Long-term, dynamic dataset Long-term, dynamic dataset Coordinate with CDIAC, La Thuile, ESG, other activities Coordinate with CDIAC, La Thuile, ESG, other activities

12. Summary Highly collaborative effort, made possible byHighly collaborative effort, made possible by Efforts (largely unfunded) of model and tower investigators Efforts (largely unfunded) of model and tower investigators Bringing together data, model and observation communities Bringing together data, model and observation communities A productive series of workshops discussing protocol, analysis A productive series of workshops discussing protocol, analysis Standardized inputs and flux observations Standardized inputs and flux observations Coordination by NACP team, CDIAC, FLUXNET to determine and collect necessary ancillary data for models not already available Coordination by NACP team, CDIAC, FLUXNET to determine and collect necessary ancillary data for models not already available Valuable dataset for model developersValuable dataset for model developers First formal estimates of observation uncertainty in a standard dataset First formal estimates of observation uncertainty in a standard dataset Testbed for regional/global models to validate against a large observation network Testbed for regional/global models to validate against a large observation network Opportunity for model, observation PIs to learn from each other Opportunity for model, observation PIs to learn from each other

13. Additional Slides

14. Missing Affiliations Missing Model Affiliations Missing Site Affiliations

15. Lessons Learned Baseline parameter vs. structureBaseline parameter vs. structure Std vs. CADM parameter runs Std vs. CADM parameter runs Better way to process submission filesBetter way to process submission files Better IC criteria and dataBetter IC criteria and data Need so many sites?Need so many sites? Focus on what we do not have Focus on what we do not have Not random missing sites: which are missing? Not random missing sites: which are missing? NSS vs SS runsNSS vs SS runs Coordinate model needs with Site data collectionsCoordinate model needs with Site data collections Better detail site info/ancillary data (tree bands, resp chambers) Better detail site info/ancillary data (tree bands, resp chambers) Mike Dietze leaf level photosynthesis Mike Dietze leaf level photosynthesis Need support for background/CADM data Need support for background/CADM data Weeks per CADM file Weeks per CADM file Central lab model e.g., for leaf N Central lab model e.g., for leaf N Encourage repository for data, esp ancillary data Encourage repository for data, esp ancillary data

16. Lessons learned Chance to improve model (not tuning, use CADM)Chance to improve model (not tuning, use CADM) Clarify protocol not “out of box” Clarify protocol not “out of box” Need better phenology obsNeed better phenology obs

17. New site BondvilleBondville Not much anc data Not much anc data PermafrostPermafrost Daring Lake, Toolik Lake, other Canadian sites Daring Lake, Toolik Lake, other Canadian sites 8-mile lake (schuur) 8-mile lake (schuur) Chronosequence sites (priority 3 UCI)Chronosequence sites (priority 3 UCI) Augment under rep biomesAugment under rep biomes Grassland, savanna, shrubland, wetlands Grassland, savanna, shrubland, wetlands

18. Next round ObjectivesObjectives Non-SS Non-SS Under-analyzed biomes Under-analyzed biomes Sensitivity analyses Sensitivity analyses Survey existing analyses Survey existing analyses OAT few sites survey param OAT few sites survey param Recruit Model teamsRecruit Model teams Invite wetland model teams Invite wetland model teams IPCC GCMs IPCC GCMs Coordinate with LBA DMIPCoordinate with LBA DMIP LULC input to models (Peter T.)LULC input to models (Peter T.) Weather (Dan R.)Weather (Dan R.) SupportSupport Money to model teams, proposal to CCIWG Money to model teams, proposal to CCIWG Postdoc to coordinate Postdoc to coordinate

19. Improving Infrastructure Model submission tool (alma_var)Model submission tool (alma_var) Standard model processing (Dan Ricciuto)Standard model processing (Dan Ricciuto) Tool to Process Barr et al. uncertainty filesTool to Process Barr et al. uncertainty files Manpower (Barbara Jackson)Manpower (Barbara Jackson) Consistency across productsConsistency across products Update Wiki and FTPUpdate Wiki and FTP

20. Inter-annual (Raczka et al.) Annual total NEE at US-Ha1

21. NEE Seasonal Cycle (Schwalm et al.) Taylor Plot: All Sites Forest sites better than non-forestForest sites better than non-forest Ag models do best at Ag sitesAg models do best at Ag sites Mean (P) and optimized model (N) do wellMean (P) and optimized model (N) do well

22. NEE Error by Time Scale (Dietze et al.)

23. GPP All Sites (Schaefer et al.) Mean is best Optimized Unit Problems? Top 3 models for NEE

24. GPP Bias and Phenology Bias (  mol m -2 s -1 ) CA-Ca1 US-Ne3

25. What does all this mean? Model performance varies with structureModel performance varies with structure Peak NEE error at 1 day and 1 year periodPeak NEE error at 1 day and 1 year period Bias & phenology dominate GPP errorBias & phenology dominate GPP error GPP error large source of NEE errorGPP error large source of NEE error Must link model structure with performanceMust link model structure with performance

26. Flux Tower Sites

27. Disturbance Uncertainty ORCHIDEE at 1850 burn site, Manitoba

28. NEE Seasonal Cycle US-UMBCA-MerCA-Ca1 BestTypicalWorst

29. GPP Seasonal Cycle CA-Ca1 US-Ne3 CA-Mer BestTypicalWorst

30. NEE Diurnal Cycle CA-Ca1CA-ObsUS-Ha1 BestTypicalWorst

31. GPP Diurnal Cycle CA-Ca1CA-ObsCA-Oas BestTypicalWorst

32. Uncertainty at Diurnal Time Scale Corn Year Soybean Year Mead rain-fed corn-soy rotation site (Nebraska)

33. Observed Flux Uncertainty (Based on Richardson et al., 2006, Ag. For. Met. 136:1-18)