NOAA NWS CIO Report APSDEU-9 Tokyo, Feb 2009 Office of the Chief Information Officer NOAA’s National Weather Service Office of the Chief Information Officer NOAA’s National Weather Service

“Taking the pulse of the planet” NWSTG Functional Overview  The NWSTG is:  the central communication facility of the NWS;  the primary acquisition and distribution center for NWS data and products;  the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;  A major data exchange hub for NOAA and other agency data and products.  The NWSTG is:  the central communication facility of the NWS;  the primary acquisition and distribution center for NWS data and products;  the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;  A major data exchange hub for NOAA and other agency data and products.

“Taking the pulse of the planet” NWSTG Functional Overview The NWSTG includes  WMO Regional Telecommunication Hub (RTH) Washington  ICAO OPMET Databank  the ASOS Operations and Monitoring Center Operational oversight of U.S. federal automated surface observing systems  the AWIPS Network Control Facility AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement The NCF is the central communications hub and technical support center for AWIPS The NWSTG includes  WMO Regional Telecommunication Hub (RTH) Washington  ICAO OPMET Databank  the ASOS Operations and Monitoring Center Operational oversight of U.S. federal automated surface observing systems  the AWIPS Network Control Facility AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement The NCF is the central communications hub and technical support center for AWIPS

“Taking the pulse of the planet” Data Input to the NWSTG  MPLS  IP / Sockets  X.25  Asynchronous  FTP –  –  Web -  MPLS  IP / Sockets  X.25  Asynchronous  FTP –  –  Web -

“Taking the pulse of the planet” Dissemination Systems  GTS  ISCS  EMWIN  Internet  GMDSS  NWS Telecom. Gateway  GTS  ISCS  EMWIN  Internet  GMDSS  NWS Telecom. Gateway  NWWS  NWR  LDAD  Family Of Services (FOS)  Interagency Connections  AWIPS SBN / NOAAPORT International National

“Taking the pulse of the planet” Dissemination and Distribution NWSTG FOS SBN/NOAAPORT GOES NCEP Product Suite NWWS (10 sec) WAN NESDIS Internet-Based Dissemination EMWIN (> 1 min) Telecommunications Operations Center Satellite Legend: EMWIN NOAAPORT NWWS Commercial Satellite GOES NWR WSR-88D Field Offices NCEP Data Servers NCF Media & Other Customers Public Specialized Customers Other Agencies LDAD Designated Local Customers Commercial Weather Services Research Institutions FAA, etc. ISCS GTSEMWIN Private line Dissemination Observations Imagery WMO ICAO

“Taking the pulse of the planet”

Geographically separated backup system Replacement / Backup TG Description Worldwide users ICAO / WMO Govt. agencies Family of services Internet users Foreign countries Emergency mgrs NWS users Nat’l Centers for Environ. Prediction NWS Regional Offices Domestic/int’l observation & forecast offices AWIPS NWS Telecom Gateway Located in Silver Spring New message switching system allows future upgrades 2x upgradeability Much improved response time Redundancy ensures uninterrupted service Full configuration management 120GB/ GB 80GB/ GB 90GB/ GB 700GB/ GB Daily Throughput/# of circuits Legacy over Replacement

“Taking the pulse of the planet” RTH Washington  Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent  Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day  On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS) –GTS delivers tsunami data and warnings to connected MTN centers within two minutes  Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent  Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day  On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS) –GTS delivers tsunami data and warnings to connected MTN centers within two minutes

“Taking the pulse of the planet” Replacement NWSTG  Full functional replacement of existing capabilities  Expanded capacity and capability –Input data growth from 200 to 800 GB/day –Output data growth from 800 to 2400 GB/day  Transition to new technology –Middleware for internal transport –Network-centric systems interconnectivity –Central switching engine with relational database –SAN and NAS storage solutions  Highly scaleable architecture  Hardware refresh  Full functional replacement of existing capabilities  Expanded capacity and capability –Input data growth from 200 to 800 GB/day –Output data growth from 800 to 2400 GB/day  Transition to new technology –Middleware for internal transport –Network-centric systems interconnectivity –Central switching engine with relational database –SAN and NAS storage solutions  Highly scaleable architecture  Hardware refresh

“Taking the pulse of the planet” RTG/BTG Performance Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds Routine Message Latency (averaged monthly) 60 seconds Daily Traffic Volume (averaged monthly) 1.2TB NWSTG Performance Results (Oct, 2008) Performance Category Threshold Actual System Availability 99.90%100% Warning Message Latency10 seconds< 1 second Routine Message Latency60 seconds< 1 second Daily Traffic Volume1.2TB1.3TB (plus.2TB in last year) Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds Routine Message Latency (averaged monthly) 60 seconds Daily Traffic Volume (averaged monthly) 1.2TB NWSTG Performance Results (Oct, 2008) Performance Category Threshold Actual System Availability 99.90%100% Warning Message Latency10 seconds< 1 second Routine Message Latency60 seconds< 1 second Daily Traffic Volume1.2TB1.3TB (plus.2TB in last year)

“Taking the pulse of the planet” Government Networking Requirements - NOAANet IP-based networking solution − Any-to-any connectivity − High degree of bandwidth scalability − Optimum redundancy and survivability − IP convergence (i.e., voice, video and data over IP) − High-end performance Network security remains paramount, particularly in light of today’s socio-political threats Segmentation from the public Internet Minimizes risk of security or privacy breaches

“Taking the pulse of the planet” Logical Design

“Taking the pulse of the planet” Primary / Backup NWSTG Access

“Taking the pulse of the planet”

Major Software Components Core software components of the NWSTG replaced in 2006  Data switching software –Core software rewritten and implemented in IBM WebSphere Message Broker –Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services  Data transport infrastructure (middleware) –Shared data stores replaced with IBM MQSeries Middleware –Applications send data to each other without concern for target location –Data delivery guaranteed even during failures  Data storage infrastructure (relational database) –All data stores moved into Sybase relational database Core software components of the NWSTG replaced in 2006  Data switching software –Core software rewritten and implemented in IBM WebSphere Message Broker –Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services  Data transport infrastructure (middleware) –Shared data stores replaced with IBM MQSeries Middleware –Applications send data to each other without concern for target location –Data delivery guaranteed even during failures  Data storage infrastructure (relational database) –All data stores moved into Sybase relational database

“Taking the pulse of the planet” Internal Redundancy Internal redundancy implemented for critical processes  Active – Warm Standby –Standby system running at all times, system disks moved to standby system during failover –EXAMPLES: Individual front-end processors, application servers  Active – Hot Standby –Data replicated to Hot Standby system in realtime, automatic failover –EXAMPLE: Sybase relational database servers  Active - Active –Multiple systems active in parallel, data shared/routed between systems groups –EXAMPLES – Switching system, HTTP/FTP server groups Internal redundancy implemented for critical processes  Active – Warm Standby –Standby system running at all times, system disks moved to standby system during failover –EXAMPLES: Individual front-end processors, application servers  Active – Hot Standby –Data replicated to Hot Standby system in realtime, automatic failover –EXAMPLE: Sybase relational database servers  Active - Active –Multiple systems active in parallel, data shared/routed between systems groups –EXAMPLES – Switching system, HTTP/FTP server groups

“Taking the pulse of the planet” Architecture Overview

“Taking the pulse of the planet” Architecture Overview

“Taking the pulse of the planet” System Extensibility Virtualized server hardware platform selected for extensibility  Virtualization –System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame –Individual resources selected to create virtual servers –Major storage implemented in Storage Area Network –Virtual disks allocated to virtual servers  Extensibility –Unused resources can be allocated to virtual servers, as needed –Underused resources can be removed from a virtual server –Virtual disks can be extended or reduced as storage requirements change Virtualized server hardware platform selected for extensibility  Virtualization –System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame –Individual resources selected to create virtual servers –Major storage implemented in Storage Area Network –Virtual disks allocated to virtual servers  Extensibility –Unused resources can be allocated to virtual servers, as needed –Underused resources can be removed from a virtual server –Virtual disks can be extended or reduced as storage requirements change

“Taking the pulse of the planet” System Extensibility Virtualized server hardware platform selected for extensibility  Advanced Virtualization –Network and SAN I/O interfaces can be shared through virtual I/O –CPUs can be carved into fractional components down to 1/10 th of a CPU  Advanced Extensibility –Resources (CPU, Memory) included within system that were not purchased –If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available Virtualized server hardware platform selected for extensibility  Advanced Virtualization –Network and SAN I/O interfaces can be shared through virtual I/O –CPUs can be carved into fractional components down to 1/10 th of a CPU  Advanced Extensibility –Resources (CPU, Memory) included within system that were not purchased –If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available

“Taking the pulse of the planet” System Extensibility (Currently migrating to P595 systems)

“Taking the pulse of the planet” RTH Washington WIS approach RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements –Production and Replicated core systems All functions not essential to mission critical operations (i.e. switching) execute against replicant –SOA Approach to design –Integral file, message and application switching –RDBS core design ~ 2000 destinations defined ~ 400K defined products ~ 10M entries (rows) in switching table –Many tables used for managing bulletin, file and report storage including dissemination and services metadata. RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements –Production and Replicated core systems All functions not essential to mission critical operations (i.e. switching) execute against replicant –SOA Approach to design –Integral file, message and application switching –RDBS core design ~ 2000 destinations defined ~ 400K defined products ~ 10M entries (rows) in switching table –Many tables used for managing bulletin, file and report storage including dissemination and services metadata.

“Taking the pulse of the planet” RTH Washington WIS approach View Data Access & Retrieval DAR service as the only truly new service provision –Plan to lay this service on to existed architecture –Will break up DAR functions DAR catalog creation & update discovery & access servicing –DAR will not execute on primary production system (at least in early stages of implementation View Data Access & Retrieval DAR service as the only truly new service provision –Plan to lay this service on to existed architecture –Will break up DAR functions DAR catalog creation & update discovery & access servicing –DAR will not execute on primary production system (at least in early stages of implementation

“Taking the pulse of the planet” RTH Washington WIS approach Develop metadata catalog first in conjunction with national & regional partners –Regional WIGOS Development Project RA-IV Integrated Atmosphere Observing System With initial emphasis on RADAR observations –Will leverage knowledge of partner expertise –Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners. –Work will be made available to all Members –Will collaborate with WIS Project Team, WMO ETs and developers in other Regions Develop metadata catalog first in conjunction with national & regional partners –Regional WIGOS Development Project RA-IV Integrated Atmosphere Observing System With initial emphasis on RADAR observations –Will leverage knowledge of partner expertise –Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners. –Work will be made available to all Members –Will collaborate with WIS Project Team, WMO ETs and developers in other Regions

“Taking the pulse of the planet” Meteorological Assimilation Data Ingest System (MADIS) Transition to Operations SFC-LAND SFC-MARINE U/A-IN SITU U/A-REMOTE SENSING SATELLITE GRIDS METADATA MADIS Collection, QC, and Distribution Information Bases (QCed Datasets) Quality Control Information (Data QC Flags) And Metadata PM 12/12/05

“Taking the pulse of the planet” MADIS Ingests and Performs QC on Meteorological Observational Data  Meteorological Surface –METAR –Airways –Maritime –Modernized NWS Cooperative Observer –UrbaNet –Integrated Mesonet State DOT Mesonets AWS  RSAS Surface Grids  Meteorological Surface –METAR –Airways –Maritime –Modernized NWS Cooperative Observer –UrbaNet –Integrated Mesonet State DOT Mesonets AWS  RSAS Surface Grids  NOAA Profiler Network  Hydrological Surface  Automated Aircraft  Multi-Agency Profiler  Cooperative Agency Profiler  Radiosonde  Radiometer  Satellite Wind –NOAA GOES Products  Satellite Radiances and Soundings –NOAA POES  Snow

“Taking the pulse of the planet” TOC Environment Gateway MADIS-T FTP External ISP LDM FTP Storage (SAN) Automated QC processing Gateway Customers (NCDC, NCF, GTS etc) Data Processing LDM NOAA Customers Non-NOAA Observations LDM Integrated Datasets Web Server Internet Web Server ? ? ? Individual Data Products Tunnel

“Taking the pulse of the planet” Weather Impacts on Society

“Taking the pulse of the planet” Data, Products, and Services to:  Greatly reduce loss of life and injury  Enable communities to mitigate property loss well in advance of threatening conditions  Alert economic sectors to environmental risks with sufficient lead time to limit or avoid impacts Where We’re Headed

“Taking the pulse of the planet”  Integrated Surface Observing System (ISOS) –ASOS –Cooperative Observers (COOP) –COOP-Modernization –National Mesonet Where We’re Headed

“Taking the pulse of the planet” Climate Services  Enhance products and services with climate change information  Local outlooks for temperature, precipitation, El Niño/La Niña  Seamless Climate, Water, and Weather products and services Reduce losses from wildfires Save lives and reduce property damages caused by major climate anomalies including drought Meet new needs with end-to-end suite of climate products and services Reduce losses from wildfires Save lives and reduce property damages caused by major climate anomalies including drought Meet new needs with end-to-end suite of climate products and services Where We’re Headed

“Taking the pulse of the planet” Water Resource Services  Provide high-resolution water quantity, quality, and soil moisture forecasts –Emergency and resource managers mitigate losses for conditions ranging from droughts to floods Enables NOAA to meet our Nation's growing needs for water forecasts Provides forecasts for consumption resources Provides important resource protection capabilities Enables NOAA to meet our Nation's growing needs for water forecasts Provides forecasts for consumption resources Provides important resource protection capabilities Where We’re Headed

“Taking the pulse of the planet” Ecosystem Impact Information  Provide forecasts of weather, water, and climate impacts for management decisions –Management decisions reflect relationships among humans, nonhuman species, and the environments in which they live Benefits through innovative approaches to spill preparedness, response, damage assessments and restoration NOAA contributes approximately $75 million annual to the U.S. economy. Benefits through innovative approaches to spill preparedness, response, damage assessments and restoration NOAA contributes approximately $75 million annual to the U.S. economy. Where We’re Headed

“Taking the pulse of the planet” Forecast Uncertainty Information  Integral and essential part of all forecasts  Enterprise-wide partnership to generate and communicate forecast uncertainty to decision makers and public  Expressed in terms of probabilities Users decide whether to take action and appropriate level of response Thresholds unique to decision maker – based on mission risk Users decide whether to take action and appropriate level of response Thresholds unique to decision maker – based on mission risk Where We’re Headed

“Taking the pulse of the planet” Future Activities – RTH/GISC Washington  Continuing tech refresh  Differences between TG & BTG –Not affecting functional backup –Is a maintenance issue  Next round of Continuity of Operations Planning (COOP) –Backup strategy – hot, warm. cold?  Continuing tech refresh  Differences between TG & BTG –Not affecting functional backup –Is a maintenance issue  Next round of Continuity of Operations Planning (COOP) –Backup strategy – hot, warm. cold?

“Taking the pulse of the planet” Future Activities  GTS – WIS evolution –MTN & GISC core network? –HAZNET VPN  GISC planning and Implementation  WIGOS integral with WIS  Our national strategy is integral with the regional strategy  WDP - proof of concept prior to wider architecture implementation  GTS – WIS evolution –MTN & GISC core network? –HAZNET VPN  GISC planning and Implementation  WIGOS integral with WIS  Our national strategy is integral with the regional strategy  WDP - proof of concept prior to wider architecture implementation A flexible services oriented framework based on a regular requirements review process responding to and feeding into the strategic planning process, at national, regional and WMO levels

“Taking the pulse of the planet” IGDDS GTS Data pull Data push WIS WIS DATA-COMMUNICATIONS FUNCTIONS AND SERVICES WIS/GTS: for time and operation-critical data & products WIS/IGDDS: for space-based data & products WIS/DAR: data discovery, access and retrieval DAR HAZ-NET VPN How do we serve the People?

“Taking the pulse of the planet” Food for thought Services Delivery Data Processing Data Exchange Observations Danger of fracturing or stove piping GEO, WMO & other data sharing initiatives Services Delivery Data Processing Data Exchange Observations Danger of fracturing or stove piping GEO, WMO & other data sharing initiatives

“Taking the pulse of the planet” Challenges To peer into the future and have a vision of future needs The volume of observations from satellites, RADARs, mesonets, and non-traditional domains continues to grow well beyond levels at which we are able to keep up with. NWP presents a two fold problem to the data exchange community. Insatiable Hunger - Profuse Output What demands will climate services and multi-domain models bring? To peer into the future and have a vision of future needs The volume of observations from satellites, RADARs, mesonets, and non-traditional domains continues to grow well beyond levels at which we are able to keep up with. NWP presents a two fold problem to the data exchange community. Insatiable Hunger - Profuse Output What demands will climate services and multi-domain models bring?

“Taking the pulse of the planet” Challenges I want to scare you, at least a little bit Our community is struggling to handle these demands We are often so busy planning for today’s needs that future requirements are a best a casual afterthought. As a result we stay behind curve, always playing the catch-up game. We need to embrace new technologies and develop adaptive, agile exchange architectures Services Orientation SCALABILITY EXPANDABILITY EXTENSIBILITY I want to scare you, at least a little bit Our community is struggling to handle these demands We are often so busy planning for today’s needs that future requirements are a best a casual afterthought. As a result we stay behind curve, always playing the catch-up game. We need to embrace new technologies and develop adaptive, agile exchange architectures Services Orientation SCALABILITY EXPANDABILITY EXTENSIBILITY