2. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center National Climate Assessment (NCA) –Version 3.0 –Interagency climate portal NCA Data Management Planning –Considerations –Requirements Approach –End-to-end data management and connecting in the middle –Earth System Grid Federation Challenge and Next Steps 2

3. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 3 Reproducible ------ Use cases ------ operational Increasingly stringent criteria for use More or different requirements have to be met NCA severe weather forecast External data policy

4. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 4 Reproducible ------ Use cases ------ operational Increasingly stringent criteria for use More requirements have to be met NCA severe weather forecast long-term retention requirement

5. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 5 Section 106 of GCRA: Scientific Assessment On a periodic basis (not less frequently than every 4 years), the Council, through the Committee, shall prepare and submit to the President and the Congress an assessment which – integrates, evaluates, and interprets the findings of the Program and discusses the scientific uncertainties associated with such findings; analyzes the effects of global change on the natural environment, agriculture, energy production and use, land and water resources, transportation, human health and welfare, human social systems, and biological diversity; and analyzes current trends in global change, both human- induced and natural, and projects major trends for the subsequent 25 to 100 years.

6. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 6 NOAA has played consistent leadership role in previous National Assessments NOAA ‘hosting’ and supporting Federal Advisory Committee for next assessment Web-framework, data policy - quality and transparency, publication – editorial and graphics, modeling/”downscaling”, scientific leadership, climatologies, regional partners and engagement

7. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center A product AND a sustained process More stakeholder engagement and contributions –“Request for Information” will be sent out shortly More transparency, traceability, accessibility and usability More data, more authors, more users A suite of Indicators (discrete and composite, physical and societal) Next report due in 2013, draft due in 2012, Federal Advisory Committee just formed Snapshots plus frequent updates –More frequent interim reports, updated indicators and fresh web-content are expected… BUT –An assessment is necessarily a snapshot in time and should be reproducible Regional and Sectoral chapters, cross-sectoral and inter-regional, adaptation and mitigation 7

8. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center National Climate Adaptation Summit, John Holdren, Director of the White House Office of Science and Technology Policy, : “Creating a federal climate information portal. This would provide single-point access to data from all relevant federal agencies and programs and would evolve over time into a more ‘national’ portal with information about relevant non-federal efforts.” Also, Executive Office of the President, Committee on Environment, Natural Resources, and Sustainability (CENRS) Roundtable on Climate Information Services called for: an interagency climate information portal plan, focusing on National Climate Assessment Reflects broad calls from stakeholders – real motivation 8

9. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center A National Climate Portal as envisioned is an immense task Use the National Climate Assessment (NCA) to focus the initial scope The NCA can be… –A starting point, extensible to interagency data –A driver of requirements for Data Management Planning: transparency, access, usability, data quality, and business rules –A test case for connecting robust and federated data structures with appropriate user experiences 2009 Assessment is being redeployed online as we speak and explores better transparency and data access 9

10. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center External Data ( At least 2 tiers): –External to NOAA, but still federal  (even here, egs of citizen science (eg climate effects network – USGS) –Clear need to move beyond government data:  peer reviewed scientific literature as information source,  Non-peer reviewed, non-traditional –but who guarantees access and a continuous/permanent archive? –Tiered “acceptance”? Core vs. supplemental? –Using external data in NCA implies that quality and access are guaranteed Quality –Information Quality Act requirements (highest level) –Peer review requirements (highest level) 10

11. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 11 NCA is classified as ‘A Highly Influential Scientific Assessment’ (HISA) - must meet the highest level of quality in the Information Quality Act (NOAA IQA) –is a scientific assessment that: (i) has a potential impact of more than $500 million in any one year on either the public or private sector (the economic test); or (ii) is novel, controversial, or precedent-setting, or of significant interagency interest (the narrative test).

12. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center Volume –Enormous volume growth  E.g. physical data growth driven by satellite/radar and model data Kinds of data –Physical climate data  Model data, reanalysis, satellite…  e.g. downscaled and right-scaled data needed and new model runs (CMIP5) –Also socioeconomic, biophysical, ecological… –Original data (e.g., satellite and in-situ observations) –Synthesized data (e.g., model data, indicators, GIS application) –Interpreted products (e.g., regional assessments) –Also will need to consider ‘new’ types of data such as: State and local government, public utilities, traditional knowledge, operational data, insurance risk data 12 J T Overpeck et al. Science 2011;331:700-702 Example of projected growth in “Physical” climate data

13. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center NCA will likely not ‘host’ basic data, but will provide central access to federated system (critical questions remain for external data) –For example, when external data is of high quality but permanent access not guaranteed – do we agree to host? Which agency? –Highly-processed, derived data used in the report may be centrally hosted –Use of data.gov? Must accommodate huge data sets –Too big to move, too big to host all data in one location –Implies distributed data architecture, federated storage Metadata must adhere to common standards Uncertainty must be documented Quality and original source of data must be clear and peer reviewed Data system must be interoperable –Use of data.gov must have clear guidelines Narrative assessment and graphics must link to source data and information 13

14. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center 14 User requirements should drive business rules Federated data system Missing Layer Making the connection between user interface and complex data system – different communities of effort User Interface Data Stewardship Business Rules What data sets? What formats? What time periods? How to cite data consistently? How to document data reviews? How to ensure traceability from key findings to data? How will we curate the data used on the assessment? How will we deal with non- traditional data?

15. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center Too much data to move, must leave in place Promote sharing of knowledge, software and tools among partners Define APIs and protocols for interoperability among data centers, PROTOCOLS FOR ACCESS OF EXTERNAL DATA? Collaborative development of some software components Deployment of common software infrastructure

16. 22 June 2011Data Management PlanningNOAA’s National Climatic Data Center Develop an implementable strategic plan for the immediate data management demands of the NCA, and the longer-term interagency/ National Climate Portal Guidance to external contributors: how to meet the requirements – near term need! Focus on the business rules SUGGESTION: Case study/use cases to illustrate approach Work with NCA to co-develop NOAA guidance If we can meet rigorous IQA standards of NCA, then we likely meet quality for other uses, but too restrictive for some uses? Points to tiered approach. If one policy is preferred, must incorporate different uses 16