1. NRC April 23, 2007 1 NPOESS EDRs vs. Climate Data Records (CDRs) John J. Bates, Chief Remote Sensing Applications Division NOAA’s National Climatic Data Center

2. NRC April 23, 2007 2 Outline  What is a CRD and why should NOAA produce CDRs?  Steps Toward Operational Production of Climate Data Records – Scientific Data Stewardship –Prioritization –Production –Productivity –Organization  Conclusion

3. NRC April 23, 2007 3  NOAA’s Scientific Data Stewardship rooted in NRC dialogue and reports  NOAA/NRC SDS leads –Bates –Goldberg Scientific Data Stewardship

4. NRC April 23, 2007 4 NRC – NOAA Response  With the transition of the U.S. Global Change Research Program into the Climate Change Science Program (CCSP), NOAA was identified as the lead U.S. Agency on climate.  “NOAA’s new climate mandate is fundamentally different from its traditional weather forecasting mandate and raises a new set of challenges owing to the varied uses of climate data, the complexities of data generation, and the difficulties in sustaining the program indefinitely.” (NRC)  In response to this change and the planned transition of NASA research climate observing missions to the NPOESS mission, the Scientific Data Stewardship Project was proposed to begin the operational production of climate data records.

5. NRC April 23, 2007 5 Key Elements of a Successful CDR Program CDR Organizational Elements High-level leadership council Advisory council to represent climate research community and other stakeholders Fundamental Climate Data Record (FCDR) Teams Thematic Climate Data Record (TCDR) Teams CDR Organizational Elements High-level leadership council Advisory council to represent climate research community and other stakeholders Fundamental Climate Data Record (FCDR) Teams Thematic Climate Data Record (TCDR) Teams CDR Generation Elements High accuracy and stability of FCDRs Pre-launch characterization of sensors and lifetime monitoring Thorough calibration of sensors Well-defined criteria for TCDR selection Stakeholder involvement and feedback for TCDRs Well-defined criteria for TCDR validation Use of in-situ data for validation CDR Generation Elements High accuracy and stability of FCDRs Pre-launch characterization of sensors and lifetime monitoring Thorough calibration of sensors Well-defined criteria for TCDR selection Stakeholder involvement and feedback for TCDRs Well-defined criteria for TCDR validation Use of in-situ data for validation Sustaining CDR Elements Available resources for reprocessing CDRs as new information becomes available Provisions for feedback from scientific community Long-term commitment of resources for generation and archiving of CDRs and associated data Sustaining CDR Elements Available resources for reprocessing CDRs as new information becomes available Provisions for feedback from scientific community Long-term commitment of resources for generation and archiving of CDRs and associated data Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors together with the ancillary data used to calibrate them. Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by blending satellite observations, in-situ data, and model output. A Climate Data Record (CDR) is a time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change

6. NRC April 23, 2007 6 Sensor Data Records (SDRs) Data (Direct & Remotely Sensed) Fundamental Climate Data Records (FCDRs) Thematic Climate Data Records (TCDRs) Climate Data Records or Homogenized Time Series Homogenization and Inter-Calibration Time-tagged Geo-Referenced & Calibrated Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Converted to Bio-Geophysical Variables What is a CDR vs. EDR? Climate Data Records

7. NRC April 23, 2007 7 Only CDRs Provide the Accurate Rate of Global Warming We require Intercalibrated CDRs in order to merge multiple, legacy observing systems (Red is EDR, Blue is CDR)

8. NRC April 23, 2007 8 Is There an El Niño? Expect the Unexpected Volcanic eruptions contaminate infrared-only SSTs. Lesson is that we must correct and reprocess EDRs to CDRs for critical climate variables

9. NRC April 23, 2007 9 CDRs Seamlessly Transition Over Major Technology Upgrades: AVHRR to MODIS

10. NRC April 23, 2007 10 Operational Climate Data Records – Prioritization, Production, & Productivity

11. NRC April 23, 2007 11 Brief History of Prioritization  Mid-June 2006 CCSP Prioritization Workshop –Included WG Representatives –Had difficulty in balancing satellite measurement priorities with in-situ priorities – and priorities of different communities  Initial Development of a Hypertext Alternative –Workshop constrained by 2-D spreadsheets –Hypertext (Web-based) navigation approach provides better balance between complexity and understandability –Led to a representation showing how Essential Climate Variables interact with  Science Issues  Societal Benefits

12. NRC April 23, 2007 12 Prioritization Approach  CCSP and Other Groups have identified ~40 “Essential Climate Variables” (ECVs), e.g. –Total Solar Irradiance –Snow Cover –Sea Level  Agencies involved in CCSP work have identified –Thematic Areas where data may provide societal benefit –Scientific Questions where data is needed for answering research questions  Prioritization aided by weighting contribution of ECV data in both Thematic Areas and Scientific Questions  Started with Spreadsheet – then have been extending weighting approach to database with web interface  Creates a priority according to connection and breadth of impact –Trying to avoid disciplinary myopia –Trying to ensure considering both scientific interest and societal benefits  Adding additional information based on availability of current and past data sets

13. NRC April 23, 2007 13  Societal benefits of the Global Earth Observations System of Systems (GEOSS) emphasize the tradition of applied climatology but using 21 st century systems  Key Science Intergovernmental Panel on Climate Change (IPCC) Issues address the policy-relevant scientific research topics of highest priority for Climate and Global Change studies Prioritizing Observations and Climate Data Records – Societal Benefits and Key Science Issues

14. NRC April 23, 2007 14 Prioritization - Future Work  Extend Input Opportunities –Creating web site with extensive information on satellites, instruments, in-situ data sources, and data products  Need Deeper and More Rigorous Approach to Prioritization –Can we quantify the economic value of climate data? –Can we quantify the required “Signal to Noise” for climate data on a rigorous basis?  Rigorous error budgets – and identification of physical basis for error estimates  Objective approaches to weighting influence of different variables on climate change (e.g. OSSEs)

15. NRC April 23, 2007 15 NOAA CDR Production - Climate Central Requirements –For reprocessing, SDS requires an IDPS-like system, (Climate Central) to process: –SDS interdependent with CLASS, e.g., large data set I/O –CLASS and SDS mutually dependent –Requires full production engineering NOAA/CLASSUsers ACQUIRE RDRs IMPROVED SDRs MULTISATELLITE FCDRs PROCESS TCDRs PROVIDE ASSESSMENTS IMPROVEMENTS NOAA SDS CLIMATE CENTRAL Products and Services Provide FCDRs and TCDRs RDR SDR Multi-satellite FCDRs & TCDRs

16. NRC April 23, 2007 16 Production Example - Sea Surface Temperature Linkages & processes needed to create daily SST product

17. NRC April 23, 2007 17 SatelliteClimate In-SituModel NOAA POES NASA AMSR GTS Buoy & Ships NCEP Sea Ice Production - Sea Surface Temperature Observations

18. NRC April 23, 2007 18 Production - Sea Surface Temperature Data Management / Processing All data management and processing performed at NCDC

19. NRC April 23, 2007 19 Production - Sea Surface Temperature Analysis / Examples of Expected Uses

20. NRC April 23, 2007 20  Hurricane intensity historically estimated from “best track” data, in spite of its inherent temporal heterogeneities  An objective reanalysis of homogeneous 23-year satellite data set produced by NCDC  U Wisconsin developed an objective analysis algorithm to work with NCDC data.  “UW/NCDC” intensities have little temporal bias Productivity - Hurricane Intensity Reanalysis ~169,000 images ~2,000 tropical cyclones

21. NRC April 23, 2007 21 Climate Working Group NOAA Science Advisory Board Scientific Data Stewardship CLASS Working Group Scientific Data Stewardship Program Management Research Climate Data Science Teams FCDR Teams Observations Scoping Requirement Systems C 2 D 2 NCDC ORA FY06 EDSM $2.5M C 2 D 2 $1 to $1.5M Governance and Management Structure TCDR Teams R&D Products and Services Theme Areas NOAA (5-10%) External (1-2%) NOAA, Other Agencies External Scientific (~90%) NOAA Climate Program & Climate Board NOAA Observing System Council Operational CDR Generation & Data Mgmt. CDR Generation NOAA Other Agencies Universities Private Sector CLASS NOAA IT Infrastructure operators Research to Operations Currently existsFY05FY06 NOAA’s Scientific Data Stewardship Program 21

22. NRC April 23, 2007 22 Conclusions  NPOESS EDRs  CDRs, but are a helpful first step  The foundation of operational production of satellite CDRs has begun within NOAA  Prioritization, production, productivity, and organization have been scoped and analogs begun using current data sets  Resource requirements for SDS are now being captured in the NPOESS climate remanifesting exercises