Status of Open Environmental Information Services Edward Johnson NWS Strategic Planning and Policy Office Ben Kyger NCEP Central Operations David Michaud Office of the Chief Information Officer NWS Partners Meeting August 4, 2014

Outline Introduction / Overview – Ed Johnson NOAA Big Data Partnership – Dave Michaud NCEP Model Data Access – Ben Kyger Discussion Presentation Title 2

NOAA Big Data Partnership Goals and approach Goals Unleash full potential of NOAA data through innovative approaches Enable private sector to build businesses and create jobs Improve compliance with Open Data policy Approach Position NOAA’s data with analytic, compute and storage capabilities Create sustainable private/public partnership where industry: Moves NOAA data to cloud at no net cost to government Provides free public access to NOAA data Creates new profitable value-added services 3

NOAA Big Data Partnership - Architectural Concept 4 Earth Observations Earth Observations Model Outputs Agency Service Tier Access ServicesCatalog MetadataFormatting agency security boundary Consumer 1Consumer 2Consumer 3 integration functions analysis functions master copy of data agency-provided services network service provider cloud provider working copy of data application & product providers consumer Custom Product/ App #1 Custom Product/ App #2 Custom Product/ App #3

Challenges 5 Build partnerships for all NOAA data - not just weather data Encourage competitive market forces to drive efficiency of partners while protecting the integrity of mission systems Account for all the costs to implement partnership Define expectations upfront to protect equity of access and non-exclusivity of data Establish appropriate SLAs to protect data integrity and bound liability Use data access methods based upon open standards

Pre-Decisional Customer Data Access Project Status as of: 23-July 2014 Issues/Risks Issues/Risks: 1)Issue – Change in strategy from original CRADA intent 2)Issue – Increasing spacial and temporal resolution to models is a common theme but will require significant coding changes 3)Issue - Increasing resolution will consume additional unplanned high performance computing (HPC) and dissemination resources Mitigation: 1)Work directly with customers to meet current unmet requirements 2)Allow for these requirements to be added over the next 2-3 years 3)Resource planning 1)For HPC, allocate additional resources at the expense of other plans 2)For Dissemination, leverage FY16 budget increase to address this limitation SchedulingProject Information and Highlights Funding Sources and Obligation Status YTD NWS-Level AOP MilestonesDateStatus Gather unmet needs from customersQ4FY14G Respond to all customers with plans to satisfy needs with existing infrastructure if feasible Q1 FY15G Complete Big Data statement of objectivesQ4FY15G Dissemination infrastructure refresh if necessaryQ4FY16G Implement GFS hourly output through day 5Q4FY16Y Management Attention Required Potential Management Attention Needed On Target GY R Y Y G G Y Leads: David Michaud, Ben Kyger Scope: 1)Increase accessibility of Department of Commerce Data (DOC) 2)Develop standards for data access and measurement 3)Improve data interoperability across DOC 4)Meet with customers to surface and deliver on current unmet needs 5)Upgrade NOMADS capacity if necessary TBD

Pre-Decisional 7 Participants Thus Far AccuWeather Corp Weather Analytics Delta Airlines Synoptic Data Corp (including MesoWest and WeatherFlow) Commodity Weather Group, LLC University of British Columbia

Pre-Decisional 8 Themes Models of interest –GFS, GEFS, NAM, HIRESW, RTMA, RAP, HRRR, NDAS, Wave Watch 3 What additional output is sought –Greater temporal resolution - hourly –Native spatial resolution for all parameters –Maximum vertical levels (60+) –Larger domains, especially to include full coverage of North America –More consistent delivery time for GEFS forecast hours –Additional BUFR sounding data for international stations –Simulated radar and satellite data from the GFS –ATCF guidance –Higher spatial resolution from the GEFS –Post-processed grids from NCEP centers –NOMADS-like slice/dice features –Any format is fine, but GRIB2 is preferred How long to evaluate new datasets –A few weeks –Two years