1. 1 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 The Emerging Interface among the Private, Federal & Academic Sectors in the Archive of Climate Data & Information A NOAA Climate Data Perspective August 12, 2008 (AMS Commission on Weather/Climate Enterprise) Tom Karl, Director NOAA’s National Climatic Data Center

2. 2 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 Outline Overview: NOAA Archive Community Interfaces –Examples of input to NOAA archive activities National Research Council Panel on NOAA Archive & Access NOAA Science Advisory Board’s Data Archive and Access Requirements Working Group (DAARWG) NOAA data user workshops A Possible Model for Partnerships Among the Community of Users and Providers –The National Integrated Drought Information System (NIDIS) Drought Portal A NOAA Archive and Access Business Model –A potential way to help define sector roles

3. 3 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 Community Interface for input into NOAA Archive and Access Activities – (highlighted in red) Under Secretary for Oceans and Atmosphere Reps. from NOAA Goals Reps. from NOAA Line Offices NOSC NOAA Observing System Council NOAA’s Data Management Committee The NOSC agent for coordinating the development and implementation of data management policy across NOAA NOAA SAB Examples of input into NOAA Archive & Access Activities NOAA SAB Data Archive & Access Requirements Working Group (DAARWG) (includes Academia, Private Sector, and other non-NOAA members) National Academy of Sciences NRC: Committee on Archiving & Accessing Environmental & Geospatial Data at NOAA NOAA Data Users’ Workshops and User Surveys

4. 4 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 National Research Council Panel on Archiving & Accessing Environmental & Geospatial Data at NOAA NRC Panel requested by NOAA’s Data Management Committee (Nov 2005) –Panel Goals: Develop principles & guidelines What types of data should be permanently archived What types of data could be stored for shorter durations under budgetary constraints How best to provide access to different variables, data sets and derived products –November 2007: Final NRC report completed 9 main principles and guidelines June 2008 NOAA workshop to address report NRC Principles & Guidelines Reports to be used by NOAA Science Advisory Board’s Data Archiving & Access Requirements Working Group (DAARWG) to evaluate NOAA’s process for prioritization of specific NOAA data sets NRC report (116 pages) Released Nov 2007

5. 5 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 NOAA Science Advisory Board’s (SAB) Data Archive and Access Requirements Working Group NOAA Response to Dec. 2007 recommendation made by the SAB Comprehensive procedure developed to determine what scientific records (new and existing) should be preserved in a NOAA archive Vetting Process NOAA reviews: –April 3-21 & May 1- 23 All NOAA Line Offices (NESDIS, NWS, OAR, NMFS, NOS) External NOAA Reviews: – Apr 23-May 16 & May 19-28 USGS -EROS NCAR/UCAR ECMWF Academic sector (limited) To be advertised in Fall 2008 for public comment on the NOAA Policy on Partnerships in the Provision of Environmental Information website: http://www.nesdis.noaa.gov/PSPS/psps_partnership.html

6. 6 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 User interaction & feedback are critical to NOAA’s success in terms of feedback on NOAA archive and access activities 2003 Workshop Boulder, CO 350 in attendance Keynote: VADM Lautenbacher 188 recommendations 2005 Workshop Asheville, NC 250 in attendance Keynote: Jack Kelly 102 recommendations 2007 Workshop Asheville, NC 215 in attendance Focus on 3 business sectors –energy, transportation, insurance Data Users’ Workshops 2003 Satisfaction Survey (Scale 1-5; 5 extremely satisfied) NOAA Offices: NCDC, NODC, NGDC, SAA Population Size: 25K Response Rate: 6.3K (25%) 2008 Survey Update in progress Survey link: https://ols.nndc.noaa.gov/survey_2008.pdf NOAA Data User Workshops & Satisfaction Surveys Examples of sector input to NOAA activities

7. 7 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A Potential Model for Partnerships Among the Community of Users and Providers NIDIS

8. 8 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A National Integrated Drought Information System (NIDIS) U.S. Western Governor’s Association –1996: Recommendation: U.S. to prepare for, respond to drought –2000: Creation of National Drought Policy Commission –2003: NOAA Partnership on drought monitoring & forecasting –2004: Formal document published recommending NIDIS Subcommittee on Disaster Reduction, President’s Nat. Sci. & Technology Council –2005: Highlighted drought as one of the grand challenges in disaster reduction Proposed action: development of an implementation plan for NIDIS U.S. Integrated Earth Obs. System –2005: NIDIS identified as 1 of 6 IEOS near term opportunities

9. 9 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 NIDIS – A potential model for partnerships among Gov’t, Academic and Private Sectors NIDIS Executive Council –Co-chairs NOAA Climate Program Univ. of Nebraska’s National Drought Mitigation Center –Members are senior reps. from lead organizations USDA, DOI, others the Regional Governor’s Assn., Water Councils, River Basin Commissions State government Private sector Academia Major NCDC Effort

10. 10 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A NOAA Archive and Access Business Model A potential way to help define sector roles

11. 11 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A NOAA Climate Archive Business Model Data Quality versus Latency Component National & Intl. Scientific Assessments State of the Climate Climate Data Products Model Reanalysis Climate Information Records Confidence Intervals CDRs with Multiple Independent Observing System CDRs with Multiple Analysis Team Climate Data Records (CDRs) Missing Data Random Errors 0 days1 day1 week1 monthSeasonalAnnual2-10 years HeavyModerate Light Little or noneNot relevant Color reflects NOAA Climate Data and Monitoring activity level: Data Improvements TIME Adheres to all of the GCOS Monitoring Principles

12. 12 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A NOAA Climate Archive Business Model Data Quality versus Latency Component National & Intl. Scientific Assessments State of the Climate Climate Data Products Model Reanalysis Climate Information Records Confidence Intervals CDRs with Multiple Independent Observing System CDRs with Multiple Analysis Team Climate Data Records (CDRs) Missing Data Random Errors 0 days1 day1 week1 monthSeasonalAnnual2-10 years Color Reflects NOAA Climate Data and Monitoring Activity Level: HeavyModerate Light Little or noneNot relevant Data Improvements TIME Adheres to many, but no all, of the GCOS Monitoring Principles

13. 13 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 A NOAA Climate Archive Business Model Data Quality versus Latency Component National & Intl. Scientific Assessments State of the Climate Climate Data Products Model Reanalysis Climate Information Records Confidence Intervals CDRs with Multiple Independent Observing System CDRs with Multiple Analysis Team Climate Data Records (CDRs) Missing Data Random Errors 0 days1 day1 week1 monthSeasonalAnnual2-10 years Color Reflects the NOAA Climate Data and Monitoring Activity Level: HeavyModerate Light Little or noneNot relevant Data Improvements TIME Adheres to few, if any, of the GCOS Monitoring Principles

14. 14 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 Potential Method to Define Roles Federal, Academic & Private Sector Partners NOAA Business Model for Archive & Access Data Quality versus Latency Component –NOAA heavy focus: On data improvements for delayed data (~1 month) On data improvements from observing systems adhering to GCOS monitoring principles –NOAA light focus: On real-time, short-term(~1 week) data improvements On observation systems not adhering to GCOS monitoring principles

15. 15 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 Possible Method to Define Roles Federal, Academic & Private Sector Partners NOAA Business Model for Archive & Access Examples of other components that could be expanded to define Federal, Academic, and Private sector roles Data Storage Compression Techniques Metadata New Products New Access Methods Others Questions