Linking Earth Observations to Societal Benefits: The Data Management Connection Vice Admiral Conrad C. Lautenbacher, Jr., U.S. Navy (Ret.) Under Secretary of Commerce for Oceans & Atmosphere The Federation of Earth Science Information Partners January

Linking Earth Observations to Societal Benefits: The Data Management Connection 2 Menu NOAA Data Management: Current and Future Challenges Global Earth Observations: Where Data Management Makes the Difference Integrated Observations at Work: Drought Monitor Effective Partnerships: How ESIP Can Help

NOAA Data Management

Linking Earth Observations to Societal Benefits: The Data Management Connection 4 NOAA’s Mission & Vision NOAA’s Vision An informed society that uses a comprehensive understanding of the role of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions NOAA’s Mission To understand and predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet the Nation’s economic, social and environmental needs

Linking Earth Observations to Societal Benefits: The Data Management Connection 5 NOAA’s climate data holdings benefit all sectors of the Nation’s economy Weather and climate sensitive industries account for about 1/3 of the Nation’s Gross Domestic Product Climate data access by user type vs. platform (FY05): NOAA’s Data Dissemination Critical to Economic Benefits

Linking Earth Observations to Societal Benefits: The Data Management Connection 6 NOAA’s Integrated Global Observation and Data Management Plan—2005 Update Refined and updated observing system baseline architecture Performing gap analysis between observing requirements and observing capabilities Begun investment analysis to determine efficient mix of observing systems to achieve requirements Established NOAA-wide data standards group

Linking Earth Observations to Societal Benefits: The Data Management Connection 7 Comprehensive Large Array-data Stewardship System (CLASS) CLASS is a web-based data archive and distribution system for NOAA’s environmental data It is an evolving system which will support additional “campaigns,” broader user base, new functionality as implementation continues for the next 10 years CLASS will be the principal IT system supporting NOAA’s responsibility as an environmental data steward CLASS concurrently supports both ongoing operations, and the implementation of new requirements/capabilities

Linking Earth Observations to Societal Benefits: The Data Management Connection 8 Access to CLASS Information Provide for archive & access to all NOAA data Incorporate Open Architecture & Scientific Data Stewardship Provide an interoperable framework compatible with GEO Integrated Data Environment model

Linking Earth Observations to Societal Benefits: The Data Management Connection 9 NOAA Challenges in Data Management Managing exponentially growing data volume Appropriately describing metadata to ensure long-term utility Providing data in formats that enable integration Providing clear and easy discovery of, and access to, data and information products Collecting data at risk to extend the environmental data record

Linking Earth Observations to Societal Benefits: The Data Management Connection 10 Challenges NOAA Faces in Data Dissemination Incompatible syntax and semantics among NOAA science disciplines Naming standards Meteorology (WMO) named “Temperature/ dry bulb temperature” Meteorology (air pollution) named “Boundary layer temperature” Oceanography named “Air Temperature” Location standards: Lat/Lon can be degrees/minutes/seconds or degrees to tenths and hundredths Latitude E/W, positive and negative, or running east or west Z used to designate elevation in both atmosphere and ocean but positive is up in the atmosphere and down in the ocean Time standards (Greenwich, local standard, local daylight) Data Formats (>50 formats used within NOAA) GRIB, NetCDF, HDF and others used for gridded data BUFR, NetCDF, and many others used for observations Observing standards – Direction Wind: the direction the wind is coming from Ocean Current: the direction the current is going to Thousands exist. Data & Metadata standards are needed. Several examples: Potential for no answer or the “wrong” answer to important societal issues Integrated NOAA data management needed

Linking Earth Observations to Societal Benefits: The Data Management Connection 11 Expected Data Growth

Linking Earth Observations to Societal Benefits: The Data Management Connection 12 Global Earth Observation Integrated Data Environment (GEO IDE) GOAL: A single system of systems (from the user perspective) that is used to access the data sets needed to address significant societal questions NOAA will: Identify and address integration gaps in data management systems Create interoperability across existing data management systems With partners, develop and adopt data standards for formats and terminology Integrate measurements, data, and products Examine the need for future data management requirements and will achieve: Cost avoidances in NOAA business through improved efficiency and reduced duplication Reduced risks for US IEOS and GEOSS

Linking Earth Observations to Societal Benefits: The Data Management Connection 13 Integration of data across disciplines Improved data stewardship Increased efficiency Leverage industry and community initiatives WeatherClimateHydrologyOceanographyBiology Geophysics Standard procedures, protocols, metadata, formats, terminology. Translators and middleware GEO IDE Bridging the gaps between stove-pipe systems

Global Earth Observations: Where Data Management Makes the Difference

Linking Earth Observations to Societal Benefits: The Data Management Connection 15 Global Earth Observation System of Systems Integrated Observations & Data Management

Linking Earth Observations to Societal Benefits: The Data Management Connection 16 Linking Earth Observations to Societal Benefits

Linking Earth Observations to Societal Benefits: The Data Management Connection 17 GEO Moves Forward: Working Towards Implementation Received approval for formal GEO organization and 10-year implementation plan Held GEO-I in May 2005, and GEO-II in December 2005 New GEO Secretariat Director, Jose Achache Agreed to 2006 Work Plan and adopted a budget Formally created GEO Committees GEONETCast as tangible near term project to implement GEOSS United States announced intention to move GOES satellite to a position to help offset the lack of sounder data over South America

Integrated Observations at Work in the United States: The Drought Monitor

Linking Earth Observations to Societal Benefits: The Data Management Connection 19 USGS Streamflow CPC Daily Soil Model Satellite Veg Health 30-day Precip. USDA Soil Ratings Palmer Drought Index Drought Monitor Inputs

Linking Earth Observations to Societal Benefits: The Data Management Connection 20 Map Analysis Differences Due to Late Station Reporting LESS SEVERE MORE SEVERE National Integrated Drought Information System (NIDIS): Importance to Nation Drought severity can be significantly under or over-estimated due to inadequate drought observations; affects Disaster Declaration, Mitigation Strategies & Prediction Annual direct losses to the US due to drought are $6-8 billion (FEMA) Early emphasis: real-time updates (Cooperative Observer Network data, wells, reservoirs) & integrated information via NIDIS drought portal

Linking Earth Observations to Societal Benefits: The Data Management Connection 21 National Integrated Drought Information System (NIDIS) NIDIS: Integrated National Drought Monitoring & Forecasting System Multi-agency collaboration Facilitates information exchange between local, state and federal agencies Proactive, Not Passive, Drought Response 21

Effective Partnerships— How ESIP Can Help

Linking Earth Observations to Societal Benefits: The Data Management Connection 23 Cross-walk of ESIP “Clusters” to GEO/USGEO Activities ESIP “Cluster”USGEO Societal Benefit Area USGEO Near-Term- Opportunity Integrated Data Management Air QualityHuman Health Air Quality Assessment and Forecasting System Coastal Management OceansSea Level Observing System Disaster Management Disasters Improved Observations for Disaster Warnings Ecological Forecasting Ecological Forecasts Global Land Observation System Public HealthHuman Health Air Quality Assessment and Forecasting System Water Management Water Resources National Integrated Drought Information System

Linking Earth Observations to Societal Benefits: The Data Management Connection 24 How can ESIP help? Work with USGEO to advance efforts on the Near-Term-Opportunities Continue to promote USGEO and GEO efforts and bring in stakeholders (academia, industry, scientific community, etc.) Provide us with user feedback and input

Questions?