OSTP Phase II Assessment: Climate Record Generation Briefing to Panel on Options to Ensure the Climate Record from the NPOESS and GOES-R Spacecraft Jeffrey L. Privette NOAA’s National Climatic Data Center 19 June 2007

2 Outline Background NOAA’s Climate Data Record Framework OSTP Climate Record Generation Assessment Next Steps

3 National Academy has emphasized the need for Climate Data Records –Current and past NASA and NOAA efforts have been ad hoc, need-driven and experimental NOAA initiated the Scientific Data Stewardship project to develop and implement a long-term, systematic, and comprehensive approach –Rooted in NRC dialog –Planning and project is joint with NASA Climate Data Record Generation Background © 2004 Other key NRC reports: Global Environmental Change: Research Pathways for the Next Decade (1999) Ensuring the Climate Record from the NPP and NPOESS Meteorological Satellites (2000) Climate Change Science: An Analysis of Some Key Questions (2001) Planning Climate and Global Change Research (2003)

4 NRC Identified Key Elements of a Successful CDR Program CDR Organizational Elements High-level leadership council Advisory council to represent climate research community and other stakeholders Fundamental Climate Data Record (FCDR) Teams Thematic Climate Data Record (TCDR) Teams CDR Organizational Elements High-level leadership council Advisory council to represent climate research community and other stakeholders Fundamental Climate Data Record (FCDR) Teams Thematic Climate Data Record (TCDR) Teams CDR Generation Elements High accuracy and stability of FCDRs Pre-launch characterization of sensors and lifetime monitoring Thorough calibration of sensors Well-defined criteria for TCDR selection Stakeholder involvement and feedback for TCDRs Well-defined criteria for TCDR validation Use of in-situ data for validation CDR Generation Elements High accuracy and stability of FCDRs Pre-launch characterization of sensors and lifetime monitoring Thorough calibration of sensors Well-defined criteria for TCDR selection Stakeholder involvement and feedback for TCDRs Well-defined criteria for TCDR validation Use of in-situ data for validation Sustaining CDR Elements Available resources for reprocessing CDRs as new information becomes available Provisions for feedback from scientific community Long-term commitment of resources for generation and archiving of CDRs and associated data Sustaining CDR Elements Available resources for reprocessing CDRs as new information becomes available Provisions for feedback from scientific community Long-term commitment of resources for generation and archiving of CDRs and associated data Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors together with the ancillary data used to calibrate them. Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by blending satellite observations, in-situ data, and model output. CDR - A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change

5 Observational Requirements Weather vs. Climate Weather –Best instant snapshot of an individual product –Weather variations are large compared to time-dependent observation biases –Always marches forward using the present to predict the near future Climate –Best blend of all snapshots from a family of records –Climate variations are small compared to time-dependent observation biases –Revisits the past to provide a richer context for the present and future

6 Sensor Data Records (SDRs) Data (Direct & Remotely Sensed) Fundamental Climate Data Records (FCDRs) Thematic Climate Data Records (TCDRs) Climate Data Records or Homogenized Time Series Homogenization and Calibration Time-tagged Geo-Referenced Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Converted to Bio-Geophysical Variables What is Needed for EDRs vs. CDRs?

7 Sensor Data Records (SDRs) Data (Direct & Remotely Sensed) Fundamental Climate Data Records (FCDRs) Thematic Climate Data Records (TCDRs) Climate Data Records or Homogenized Time Series Homogenization and Calibration Time-tagged Geo-Referenced Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Converted to Bio-Geophysical Variables Mature Developing What is Needed for EDRs vs. CDRs?

8 Prelaunch Characterization Peak EDR knowledge (applied to final products only) CDRs Exploit Accumulated Sensor and Algorithm Knowledge Sensor Knowledge (~ Product Quality) Time (years) Launch Checkout & Characterization End of Operations Operational Lifetime Peak CDR knowledge (applied to complete record) Long-term Climate Mission Duration

9 AVHRR Surface Reflectance EDR Shows Spurious Jumps and False Trends Result: Incorrect Climate Conclusions

Problem: Orbital drift caused change in observation time  EDR depicts erratic trends and variability due to observatory rather than environment. Solution: Reprocess with calibration and orbital corrections  CDR provides proper record of environmental variability. Error Source

11 End-to-End CDR Planning Landscape: Prioritization, Production, & Knowledge Knowledge

Global Essential Climate Variables with Heritage Records * With Certified NPOESS Program (Post Nunn-McCurdy) *

13 Example Key Uncertainties, Gaps in Knowledge, and Research Needs Identified in the IPCC 4 th Assessment Report Changes in Human and Natural Drivers of Climate Solar changes - TSIS Modification of clouds by aerosols – VIIRS, APS, ERBE Radiative forcing due to stratospheric water vapor – CrIS Atmospheric methane – CrIS Tropospheric ozone – CrIS Land surface properties – VIIRS

Production Example - Sea Surface Temperature Linkages & processes needed to create daily SST product: Climate-quality Algorithm Complexity Far Exceeds That for Weather  Higher Quality, Known Uncertainties, Connection to heritage record

15 Climate Product Hierarchy Designed to Address IPCC4 Key Questions IPCC4: Limitations on hurricane frequency, intensity, and data interpretation Major Climate Uncertainties Product Taxonomy Example

16 Phase II Assessment and Background Phase I Assessment (January 2007): “Impacts of NPOESS Nunn-McCurdy Certification on Joint NASA-NOAA Climate Goals” Phase II Study: “Identifying Alternatives in Response to Joint NASA-NOAA Climate Goals” Terms of Reference (21 February 2007) –Joint NASA-NOAA Study –NASA Primary Role Analyze and cost options for space measurement restoration –NOAA Primary Role Analyze and cost processes to (re-)produce, archive, and steward climate quality data records

17 Phase II Climate Product Considerations Key Issues 1.Climate Product Generation is an essential element for development/continuity/stewardship of global climate record 2.Priorities are to minimize measurement gaps and provide climate products 3.Starting point: assume raw instrument data are available in archive Data may come from non-NPOESS programs, incl. foreign partners Address all Certified / Demanifested / Alternative instruments CDRs merge current and relevant heritage satellite and in-situ data 4.Lifecycle Cost Estimates: Through FY26 (19 years) 5.Roles and Responsibilities currently under discussion

18 ECVs Can Require Data Fusion ALBEDO (SURFACE) LAND SURFACE TEMP LAND COVER OCEAN COLOR VEGETATION LEAF AREA VEGETATION FAPAR FIRES DISTURBANCE CLOUD PROPERTIES SEA ICE SNOW COVER SEA SURFACE TEMP UPPER AIR WIND (POLES) SOLAR IRRADIANCE AEROSOL PROPERTIES PRECIPITATION SURFACE WIND SPD & DIRECTION ATMOS. WATER VAPOR UPPER AIR TEMP OZONE METHANE, CO2, GHG RADIATION BUDGET (SFC & TOA) SEA STATE SEA LEVEL VIIRS (14) TSIS (1) APS (1) MIS (and OVW CAPABILITY) (10) CrIS/ATMS (4) OMPS (2) ERBS/CERES) (1) Altimetry (4) LAKES GLACIERS/ICE CAPS Plus derived NOAA-Unique Information Products: e.g., Hurricane trends, trends in extreme temperatures and rainfall

Sensor Essential Climate Variables (#) Climate Data Records (#)Subproducts* (#) VIIRS CrIMMS MIS5876 OMPS (Nadir Sensor Only)145 CERES (C1 Platform Only)1212 TSIS126 Altimeter2216 APS1330 Totals: *Subproducts include data in different assemblages, resolutions, formats, etc. Demanifested Partially Certified Certified ECV, CDR and Product Counts by NPOESS Sensor

20 Key CDR Activities  Climate Algorithm Development and Refinement (Leveraged)  Characterization, Time-varying Bias Adjustment, Seamless Records (Leveraged)  Data (Re-)Processing (New for climate)  Archiving and Distribution (New for climate)  Data Stewardship (New for climate)  CDR Science and Information Product Development (New for Climate)

21 Climate Record Generation Key Activity Relationships Data Stewardship (Ensures interoperability of subsystems, integrity of complete system, and long-term data preservation and extendability) Instrument and Data Characterization Algorithm Development, Refinement & Validation Data (Re-)Processing Archiving & Distribution CDR Science & Information Product Development Joint agency development to provide long-term, systematic and comprehensive CDR generation

22 What Is Data Stewardship? Ensures interoperability of all CDR subsystems, integrity of complete system, and long-term data preservation and extendability Begins pre-launch (e.g., guidance and co-development with system segments) Required throughout lifecycle (e.g., CM, documentation, tools, product support) Examples of Pre-launch and Early Mission Stewardship Requirements –Develop Policies and Processes to Ensure Traceability, Provenance and Reproducibility Metadata –Developing system-level standards –Sufficient for cataloging and user access –Developing product-by-product information and formats (pixel, granule, and collection levels) Auxilliary, Ancillary and Intermediate Product Archiving and Distribution –Develop CDR Configuration Control System Versioning, Filename structures, Data Retention/Deletion Process –Document and Configuration Manage: Algorithms, Subsystems, and Products Algorithm Theoretical Basis Documents, Algorithm Design Docs, Detailed Design Docs, Hardware Description –Develop CDR Uncertainty Definition and Reporting Process e.g., MODIS Stage 1, 2, 3 validation definitions –Develop Characterization/Cal/Val data Ingest/Archive/Distribution Framework Metadata, Documentation and Configuration Control Intermediate Product archiving/distribution requirements –Develop Guidelines for CLASS Product Assembling: Aggregation, Bundling, Compositing

23 Hypothetical Transitioning of Agency Roles & Responsibilities

24 Next Steps Listen closely to discussions/recommendations from this Workshop Refine CDR Science Support budget and risk estimation Continue to work with OSTP Continue dialogue regarding potential international and/or domestic partnerships

Backups

26 Total ECVs & CDRs MIS

27 Societal benefits of the Global Earth Observations System of Systems (GEOSS) emphasize the tradition of applied climatology but using 21 st century systems Key Science Intergovernmental Panel on Climate Change (IPCC) Issues address the policy-relevant scientific research topics of highest priority for Climate and Global Change studies Prioritizing – Key Uncertainties, Gaps in Knowledge, and Research Needs Identified in the IPCC 4 th Assessment Report

28 Production “Climate Central” Concept –For reprocessing, NOAA’s Scientific Data Stewardship (SDS) requires an IDPS-like system (Climate Central) to process: –SDS interdependent with CLASS, e.g., large data I/O –Reprocessing –Requires full production engineering NOAA/CLASS Users ACQUIRE RDRs IMPROVED SDRs MULTISATELLITE FCDRs PROCESS TCDRs PROVIDE ASSESSMENTS IMPROVEMENTS NOAA SDS CLIMATE CENTRAL Products and Services Provide FCDRs and TCDRs RDR SDR Multi-satellite FCDRs & TCDRs

29 Assumptions for “Cost to Nation” Scenario Climate Record Generation –All CCSP/GCOS satellite-based ECVs plus NOAA-unique ECVs (e.g., hurricane metrics) –NASA budgeting used for demanifested sensors Traditional sensor and science teams Includes funding of field validation networks/campaigns –NOAA budgeting used for Certified sensors Team size linked to CDRs assigned to sensor Validation based on operational field networks (e.g., buoys, sondes, Climate Ref. Net) –NOAA Stewardship Augmentation Climate Central activities beyond traditional Science Team responsibilities –Agency roles and responsibilities not yet defined Research-to-Operations transition expected

30 Science Support Budget Development Cost ECV Coverage Cost/Schedule Reliability ■ Cost_to_Nation (complete) ■ Detailed_Model (Summer 2007) Approach: 1.CDR-based budgeting spreadsheet provided Rough Order of Magnitude (ROM) Budget for May deliverable 2.Comprehensive “Bottom-Up” cost model in development

31 Characterization: Inventory of Tasks Example Pre Launch Design Review/Analy./Accept. Trade Study Support –EDR Impact Analyses On-site T/V Testing Support EDU Subsystem, SystemTest Data Analysis FU Subsystem, System Test Data Analysis System/Algorithm Coefficient Table Generation –ISO Reference Traceability Operational Data Monitoring System Design& Build In-situ Cal/Val Network Gap Analysis/Augmentation –Protocols for C/V Data Collection, Metadata, Archiving and Distribution Inter-satellite Cal/Val System Design/Build –Acquisition Coordination Launch & Checkout Diagnostic Telemetry Analy. Onboard Cal System Checkout System/Algorithm Coefficient Table Revision (on-orbit data) Sensor Diagnostic Operation and Data Analysis Calibration Field Campaigns Inter-satellite Calibration Evaluation Lunar Calibration Analysis (if applicable) Operational Life Diagnostic Telemetry Analy. Cal Data Monitoring/Trending –Intra-orbit Seasonal Analy. –Long Term Stability Analy. Cal/Val Field Campaigns Cal/Val Network Operations and Maintenance SDR/EDR Product Validation System/Algorithm Coefficient Table Updates and Tuning Anomaly Detection, Analysis & Resolution Graceful Degradation Support Algorithm Modifications to Accommodate Degradation s/c Maneuver Support and Analysis (if applicable) Long-term Climate Heritage Data Calibration/Rectification –Inter-satellite –Vicarious –Lunar –Develop Dynamic System Coefficient Tables Multisatellite Long Term Stability Analysis and Correction CDR Product Validation and Error Characterization Heritage Data Anomaly Detection, Analysis & Resolution Heritage System Graceful Degradation Development Algorithm Modifications to Accommodate Heritage System Anomalies/Artifacts (e.g., orbital drift)

32 Next Steps: Roles and Responsibilities Space Is Complex Activity Time Contributors Considerations Activity set (coarse scale) is readily developed Contributor set must be developed, evaluated Agency, Interagency, Industry, Academia Timing must address multiple challenging factors “Research-to-Operations” hand-offs Existing/Funded Projects and Capabilities Possible extended EOS / POES lifetimes Proposed Projects and Capabilities Mission roadmap decisions NRA responses (proposals) difficult to predict

33 Proposed Path Forward Defining Tasks, Roles & Responsibilities 1.Develop task inventory for each major Climate Support activity  Tasks, subtasks may vary with activity, sensor and/or product 2.Determine agencies/organizations with heritage capabilities 3.Determine projects that can cover needs with existing funds  e.g., NPOESS operational program, NASA SDS, NOAA STAR/GSICS 4.Identify gaps in current capabilities and funding 5.Develop plan for allocating uncovered tasks among agencies  Time varying  Justify overlaps where needed 6.Develop plan for allocating uncovered tasks to organizations within agency  Justify overlaps where needed 7.Roll up into master budget profile

34 Team Organization Steering: Mary Kicza and Mike Freilich Leads: –NOAA: Jeff Privette and Jim O’Neal –NASA: Bryant Cramer and Steve Neeck –IPO’s PEO/SUAG Representative: Mike Bonadonna Ex Officio (NOAA): Dave Young, Mike Tanner, Chet Koblinsky, Tom Karl Working Group (NOAA): –NOAA Chief Financial Office: Darrell Robertson and Mahendra Shrestha –Cost Modeling Consultants: John Bates (NOAA) and Bruce Barkstrom (NOAA) –Sensor Expert Teams TSIS: Rossow (City University of New York [CUNY]), Barkstrom (NOAA) ERBS: Rossow (CUNY), Ackerman (Wisconsin) ALT: Wilson (NOAA), Miller (NOAA) OMPS: Flynn (NOAA), Ravishankara (NOAA) APS: Murphy (NOAA), Laszlo (NOAA) CMIS: Janowiak (NOAA), Chang (NOAA), Chelton (Oregon), Arkin (Maryland) CrIMMS: Barnett (NOAA), Goldberg (NOAA) VIIRS: Menzel (Wisconsin), Reynolds (NOAA), Privette (NOAA) Support: Goldberg (NOAA), Menzel (Wisconsin), Justice (UMD), Weng (JCSDA)