1. Roadmap for Satellite Data in the Advanced Weather Interactive Processing System (AWIPS) GOES Users’ Conference Madison, WI
Deirdre Jones
NOAA’s National Weather Service
Office of Science and Technology
Systems Engineering Center (SEC)
November 5, 2009

2. Outline Challenges meeting new customer weather needs
Architectural Vision
AWIPS Plan to Achieve Architecture
Next Steps
Summary

3. Customers Need Improved Weather Services
Speed at which decisions are made
Demand for decision support services is increasing
US industry needs the most accurate, accessible, timely and reliable weather data to make critical decisions that impact our national economy
Aviation weather impacts were $41B in 2007
U.S. modeling and data assimilation lags other nations. We must do better to give the U.S. a competitive advantage in the global economy
Federal deficits and resource constraints
Integrated observations
More efficient R-T-O (projects, modeling)
Every dollar counts!

4. Environmental Data Challenges
Huge data explosion
Rapid data assimilation requirements (e.g., NextGen)—people, models
Demands on data management architecture
Data access on-demand within resource constraints
Integrating all observing data sources to achieve desired effect and outcome

5. NESDIS Archive Data Rate Projections
Data Volume
(non-cumulative view)
AWIPS SBN
3-18 Mbps
w/MPAR
w/GOES-S
w/GOES-R
w/NPOESS C2
w/NPOESS C1
w/NPP

6. Importance of Weather Information Database (WIDB)
Weather Industry
Observations
Private Industry
Forecasting
Private Sector
Numerical
Prediction
Systems
Network Enabled Operations
Postprocessed
Probabilistic
Output
NWS
Forecaster
Radars
Data Integration
WIDB Cube
Aircraft
Automated Forecast
Systems
Surface
Forecast Integration
Soundings
Grids
Decision
Support
Systems
Custom
Graphic
Generators
Custom
Alphanumeric
Generators
Governmental Decision Making

7. Infrastructure Readiness Approach
View requirements in larger context of all future programs
Develop strategic enterprise infrastructure plan and roadmap for building infrastructure capability over next years
Near term tasking to complete analysis of the GOES-R impact on NWS architecture: AWIPS, NCEP and NWS Telecommunications Gateway (NWSTG) (FY08-FY10)
Assess other aspects of the enterprise, as well as other system changes (FY10)
Select architecture for efficient access to new satellite data and products for forecast and warning operations (1QFY11)
Initiate acquisition (FY 11-12)

8. NOAA CIO's Architectural Vision
Web Service Interfaces- Utilizing Open Standards & Protocols
Customers
Experts
Informed
Novice
Presentation Tier (i.e., Web Presence)
Portals
Mobile Devices
Collaboration
Service
Registry
(e.g., SLAs)
Governance
Data Architecture
(e.g., GEO-IDE, IOOS)
Infrastructure Services
Commodity e.g., Authentication, Discovery
Core e.g., Common, Storage, Compute, Enterprise Mgmt
Transport e.g., LAN / MAN / WAN/Internet
Mission Services
Ecosystem Management & Protection
Climate Preparation & Response
Commerce & Transportation Efficiency, Safety, and Environmental Soundness
Weather & Water
Preparation & Response
Critical Support (Satellites & Mission Support)
Modeling & Observing Infrastructure
Security
e.g., Threat Detection, Network Scanning

9. NOAA Infrastructure Framework
Consider environmental data changes in context of NOAA’s full infrastructure and in concert with each other
Service oriented enterprise focused on data and information
Infrastructure services: core, commodity, transport, and security
Transport migration: consolidate NOAA’s disparate networks into NOAANet architecture
Mission services: monitoring and observing, modeling and forecasting, disseminating, ecosystem management, etc.
Data architecture: data description, context, and sharing
Governance: service level agreements, data management
Dissemination Services
To include emerging technologies and Internet services: portals, mobile devices, and collaboration

10. Notional Architecture
Talking point to slide 9: Enterprise considerations on the NESDIS side. For example, refer indirectly to next-generation ESPC environment (ESPDS, which could include GAS and NDE). It might be best to avoid referring to ESPDS by name (unless it's talked about during the week). Our vision includes coordinated evolution in NESDIS and NWS.

11. AWIPS in the GOES-R Era Fitting the Pieces Together
SEC leading effort to prepare the enterprise as well as planning AWIPS improvements to support future data
Robust infrastructure with capacity and throughput for larger volume/higher resolution data
Flexible software infrastructure
Communications bandwidth -- satellite broadcast and terrestrial networks
Data distribution paradigm -- push and pull

12. AWIPS II Software Re-Architecture Approach
Perform “black-box” conversion
Preserve existing functionality,
Preserve look and feel of today’s system
Maintain functionality
Deployed system current with deployed AWIPS capability (i.e., OB9)
Use open source projects - No proprietary code
JAVA and open source projects enable AWIPS II to be platform and OS independent
No plans to move from Linux
AWIPS II enables collaborative development
Operating System and platform independence allows non-Linux based research to be easily integrated into AWIPS II

13. AWIPS II Software Re-Architecture Outcome
Improved software architecture and capabilities to accommodate new data and services
Services oriented architecture
More flexible in production/delivery
Increased access to data for decision making
Improved RTO efficiency
through use of AWIPS
Development Environment
Enables future capabilities
(e.g. flexible data delivery,
thin client, collaboration)

14. AWIPS SOA Architecture Logical Layered View
Client/Presentation Services
Platform Layer
Mission Services Layer
Data Access Layer
Metadata
Index
Data Persistence
Store
Enterprise Service Bus - Communication
Security Services /Demilitarized Zone (DMZ)
Spatial
Hydro Models
LAPS
FORTRAN/C/C++
Command Line
Programs
External Programs
JMX
<<Java>>
DataLayer
PostgreSQL
HDF5
<<abstract>>
BaseDao
Hibernate
HDF5DataStore
HDF5 API
IngestSrv
PersistSrv
IndexSrv
ProductSrv
AdapterSrv
NotifySrv
SubscribeSrv
AutoBldSrv
PurgeSrv
Mbean
CAVE
StagingSrv
UtilitySrv
Localization
The rendering shown here does not provide insight in how this state will be realized or cite its advantages.
It is similar to an architectural rendering of a building. To get a true picture of how the house will be built, we need the blueprints for construction.
User functions generally access from the top down.
The hardware is at the lowest layer, then basic services, then mission services, then user interface
Note that the term “services” is used for three of the layers.
The next few slides discuss SOA basics.

15. AWIPS Product Suite On NOAAPort Today*
* NOAAPort disseminates over one million products (30+GBytes) every day

16. GOES Data in AWIPS Today
Product Name
WMO
Header
Dissem.
Path
Format (AWIPS ID)
Approx.
Frequency
Approx. Daily
Volume
TIG? ii
TIC?
Sounder-based Imagery 1
GINI
hourly
216 Mbits
DPI
TIGN
280 Mbits
Soundings
JUTX 2
BUFR
30 min
312 Mbits
AE SPE Imagery
ZETA98
Manual SPE Imagery
ZEGA98
Text
SPENES
Hi Den Winds (BUFR)
J?CX
1-3 hourly
136 Mbits
TWNA
TWSA
(none)
SCP?R i
WBCSAT5/6
NOUS71
NOUS72
ADMNES
CONUS RTMA (ECA)
LAMA98
GRIB2
Hourly
56 Mbits
Imager-based Imagery
GOES Products Flowing into AWIPS
30/15/7 min
27200 Mbits
GRIB1/Xmrg
hourly & event
driven
Mbits
0 Mbits
Hi Den Winds (Text)
80 Mbits
SPE Text
TXUS20
event driven
1.6 Mbits
Geodetic Subpnt Predictn.
TBUS
daily
ASOS Cloud Products
TCUS
Administrative Messages

17. Future NOAAPort Product Suite*
Environmental Satellite Product Additions
POES AVHRR Imagery (2010)
NPP & NPOESS (From NPP, beginning 2011)
GOES-R Series (2015)
Currently defining process for building, evaluating and implementing new products in the AWIPS II framework
NCEP Model Product Additions
High Resolution Window Model (2010)
GFS-Based DNG (2010)
Bias-corrected SREF (2011)
Numerous additional enhancements to NCEP (ongoing)
Radar (WSR-88D) Product Additions
Dual Polarization (2011)
Higher Pixel Resolution (2013)
* Only selected major planned additions are shown; pending bandwidth expansion; dates tentative

18. AWIPS Communications Improvements
New paradigm for data distribution
User readiness working group to work with users/ customers to define product access and delivery reqts.
Migrate AWIPS wide area network (WAN) to NOAAnet
Increased reliability
Increased bandwidth
Enables any-to-any network topology
Increase AWIPS SBN bandwidth to 45 Mbps (full transponder)
5 times current capacity
Enables flexible delivery over satellite

19. AWIPS Data Delivery Push/Pull Concept
Robust infrastructure to support “intelligent” access (push/pull) to non-local datasets
Key Benefits
Mitigates impacts on SBN by addressing significant growth in data volumes, e.g., ensembles, GOES-R, NPOESS
Allows users to access just the data they need by space, time, parameter
Enables synergy with NextGen net-enabled data access requirements

20. NWS Infrastructure Architecture Implementation ScheduleCY
2009
2010
2011
2012
2013
2014
2015
2016
NWS GOES-R Activities
Assess Impact on NWS Infrastructure
Architectural Requirement Development
Infrastructure Design and Development
Stand up New Architecture
Test and Evaluation of New Architecture
Dual Polarization Milestone
GOES-R and NDE Milestones
NextGen Milestones
Deployment
January 2011
NDE “Distribution
Build” Testing,
NPP Launch
Oct 2015
GOES-R
Launch
April 2013
NPOESS
C1 Launch
July 2013
GOES-R “Data
Operations Tests”
IOC
MOC

21. Summary
NOAA is working on the enterprise infrastructure to accept, manipulate, and use data from new capabilities
Ensure all NWS systems on track for user readiness
Complete assessment
Address in design solution
Build toward NOAA target architecture
Efficiencies to reduce cost needed; seek out new technologies for computing and data integration architectures

22. Back-up

23. AWIPS Workstations and
What is AWIPS?
NEXRAD
GOES/POES/NPOESS
169 separate AWIPS systems at
137 geographical locations
Forecasts
Warnings
Advisories
Watches
NCEP Models
AWIPS
Communications
Buoys,
River Gauges
ASOS
AWIPS Workstations and
Servers
Service provided to
3066 US Counties
24 hrs/day,
365 days/yr.
~900 Workstations (total)
~1200 Servers (total)

24. NOAA’s Satellite Broadcast Network Products (e.g., radar, ASOS,
NOAAPort
NOAA’s Satellite Broadcast Network
GOES
POES
AMC-2
Satellite
Other Environmental
Satellite Products
DCS
HIRS/AMSU
AVHRR
GVAR
NOAA/NWS
NCEP
Guidance &
Model
Products
Satellite
Product
Generation
WFO Centric Architecture
Little AWIPS/NAWIPS Integration
High software Maintenance Costs
Poor RTO efficiency
Fire Hose Data Distribution
AWIPS
NCF
TOC
NWSTG
NOAAPORT
Uplink
Facility (NY)
Satellite imagery
NOAA/NESDIS
WFO/RFC
Forecasts, Watches
Warnings
NOAAPORT Users
Includes Many Users –
Inside and Outside of NOAA
Hydromet Sensor
Products (e.g., radar, ASOS,
Profiler, River Gages)

25. Technical Specifications
NOAAPort
Technical Specifications
Product Stream Assembly: AWIPS Network Control Facility (Silver Spring Md)
NOAAPort Satellite: SES Americom (AMC-2), Transponder: 13C
NOAAPort Satellite Location: 101◦ West
Downlink Signal: C Band Network Capacity: 10 Mbits/sec (approximate)
Transmission Protocol: DVB/S - Digital Video Broadcast by Satellite
Product Identification: Headings (NOAAPort, WMO/AWIPS & product)
Major Product Formats: GRIB, BUFR, GINI, Radar Level III, Text, Redbook
Compression: zlib for satellite imagery and most radar products; GRIB2 supports JPEG2000 compression
There are four NOAAPort “data streams” or “channels”– each with a unique DVB PID.
GOES/NESDIS
NCEP/NWSTG
NCEP/NWSTG2
OCONUS Imagery/Model/DCP

26. Other Satellite Data in AWIPS

27. Glossary of Terms
AWIPS: Advanced Weather Interactive Processing System
CLASS: Comprehensive Large Array-data Stewardship System
DMSP: Defense Meteorological Satellites Program
GOES: Geostationary Operational Environmental Satellite
GOES-R: Geostationary Operational Environmental Satellite R-Series
GOES-R: Geostationary Operational Environmental Satellite S-Series
IMETS: Incident Meteorologists
IOC: Initial Operating Capability
MetOp: Meteorological Operational Satellite
MPAR: Multi-function Phased Array Radar
Mid Operating Capability
NDE: NPOESS Data Exploitation
NESDIS: National Environmental Satellite, Data, and Information Service
NextGen: Next Generation Air Transportation System
NEXRAD: Next-Generation Radar
NPOESS: National Polar-orbiting Operational Environmental Satellite System
NPP: NPOESS Preparatory Project
POES: Polar Operational Environmental Satellite
R-T-O: Research to Operations
SBN: Satellite Broadcast Network