Cyberinfrastructure to Support Real-time, End-to-End Local Forecasting Mohan Ramamurthy Tom Baltzer, Doug Lindholm, and Ben Domenico Unidata/UCAR AGU Fall Meeting December 16, 2004

Local NWP: A Growing Activity Mesoscale forecast models are being run by universities, in real time, at dozens of sites around the country, often in collaboration with local NWS offices –Tremendous value –Leading to the notion of “distributed” NWP Yet only a few (OU and U of Wash) are actually assimilating local observations – which is one of the fundamental reasons for such models! Applied Modeling Inc. (Vietnam) MM5MM5 Atmospheric and Environmental Research MM5MM5 Colorado State University RAMSRAMS Florida Division of Forestry MM5MM5 Geophysical Institute of Peru MM5MM5 Hong Kong University of Science and Technology MM5MM5 IMTA/SMN, Mexico MM5MM5 India's NCMRWF MM5MM5 Iowa State University MM5MM5 Jackson State University MM5MM5 Korea Meteorological Administration MM5MM5 Maui High Performance Computing Center MM5MM5 MESO, Inc. MM5MM5 Mexico / CCA-UNAM MM5MM5 NASA/MSFC Global Hydrology and Climate Center, Huntsville, AL MM5 MM5 National Observatory of AthensMM5MM5 Naval Postgraduate School MM5MM5 Naval Research Laboratory COAMPSCOAMPS National Taiwan Normal University MM5MM5 NOAA Air Resources Laboratory RAMSRAMS NOAA Forecast Systems Laboratory LAPS, MM5, RAMSLAPS, MM5, RAMS NCAR/MMM MM5MM5 North Carolina State University MASSMASS Environmental Modeling Center of MCNC MM5 MM5MM5 NSSL MM5MM5 NWS-BGM MM5MM5 NWS-BUF (COMET) MM5MM5 NWS-CTP (Penn State) MM5MM5 NWS-LBB RAMSRAMS Ohio State University MM5MM5 Penn State University MM5MM5 Penn State University MM5 Tropical Prediction SystemMM5 Tropical Prediction System RED IBERICA MM5 (Consortium of Iberic modelers) MM5 (click on Aplicaciones)MM5 (click on Aplicaciones) Saint Louis University MASSMASS State University of New York - Stony Brook MM5MM5 Taiwan Civil Aeronautics AdministrationMM5MM5 Texas A\&M UniversityMM5MM5 Technical University of MadridMM5MM5 United States Air Force, Air Force Weather Agency MM5MM5 University of L'Aquila MM5MM5 University of Alaska MM5MM5 University of Arizona / NWS-TUS MM5MM5 University of British Columbia UW-NMS/MC2UW-NMS/MC2 University of California, Santa Barbara MM5MM5 Universidad de Chile, Department of Geophysics MM5MM5 University of Hawaii MM5MM5 University of Hawaii RSMRSM University of Hawaii MM5MM5 University of Illinois MM5, workstation Eta, RSM, and WRFMM5, workstation Eta, RSM, and WRF University of Maryland MM5MM5 University of Northern Iowa EtaEta University of Oklahoma/CAPS ARPSARPS University of Utah MM5MM5 University of Washington MM5 36km, 12km, 4kmMM5 36km, 12km, 4km University of Wisconsin-Madison UW-NMSUW-NMS University of Wisconsin-Madison MM5MM5 University of Wisconsin-Milwaukee MM5MM5

Science Drivers for Local Modeling Many weather phenomena that affect society and commerce occur on the mesoscale. E.g., squall-lines, snowbands, hurricanes; downslope windstorms, lake-effect snowfall, etc. Need high-resolution local modelling to accurately resolve and predict these phenomena; Utilize dense local observations (e.g., Mesonets); Resolve local topography Collaboration with local NWS forecast offices; Show examples

Technology Trends Enabling A New Generation of Local NWP Activities  Commodity microprocessors & inexpensive but powerful workstations/clusters  High-bandwidth networks (e.g., Internet 2)  Transparent data access and delivery  Community Models (MM5, WRF)  Local observatories (e.g., mesonets)  Community codes for data assimilation (e.g., 3DVAR, ADAS)

Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction PCs to Teraflop Systems Product Generation, Visualization, Dissemination End Users NWS Private Companies Students Numerical Weather Prediction: Key Steps Observations & Previous Model Forecast Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Radar Data Wind Profilers GPS/Met instruments

Unidata Technologies [That can be Used] in Local Modeling  Local Data Manager – data transport  Data streams: IDD and CONDUIT – Relaying and accessing data  Decoders – Data conversion  NetCDF libraries and tools – Data infrastructure  OPeNDAP – Remote data access (Collaborator)  THREDDS – Cataloging data  GEMPAK and IDV - Visualization  GIS Integration tools (in future)

Real-time Data Distribution Radar Model Satellite There are over 150 university sites in North and South America, Europe, and Asia that receive real-time data using the Unidata Local Data Manager; Plus there are over 300+ LDM sites in NWS, NOAA, NASA, KMA, Taiwan, and Spain that are not part of the “open” IDD.

LDM in Action LDM is providing a variety of real-time meteorological observations and model output from operational prediction systems for local NWP initialization WSR 88-D Data SuomiNet

Meteorologica l AssimilationSystem User running local analysis and display tools Regional Model Hosted on local hardware Decoders National Forecast Model Output Today’s Local NWP Process at Many Universities Assimilated Data For Initial ConditionsReal-time Weather Data Decoders

Meteorologica l AssimilationSystem User running local analysis and display tools Regional Model Hosted on local hardware Decoders National Forecast Model Output Today’s Local NWP Process at Many Universities Assimilated Data For Initial ConditionsReal-time Weather Data Decoders There is no Data Sharing (other than with local NWS offices)

OPeNDAP Servers Unidata Motherlode ServerUnidata LEAD Testbed There are many OPeNDAP servers for operational and historical data, but none outside of Unidata & LEAD for real-time local NWP output

Remote Data Access and Catalogs Developed for real-time WRF predictions from University of Illinois. Courtesy: Brian Jewett

Integrated Data Viewer Unidata’s newest scientific analysis and visualization tool Provides 2, 3 and 4-D displays of geoscientific data Stand-alone or networked application, providing client- server data access via multiple protocols Java-based tools: Runs on Windows, Macs and Unix machines

Remote Visualization of Local NWP Output Developed for real-time WRF predictions from University of Illinois. Courtesy: Brian Jewett

GEMPAK Example Some sites convert their forecast output into a format compatible with GEMPAK analysis and visualization tool Enables integration of local model output with other operational data sets

Thematic Real-time Environmental Distributed Data Services (THREDDS) Combines “push” with several forms of “pull” and digital library discovery To make it possible to publish, locate, analyze, visualize, and integrate a variety of environmental data Connecting People with Documents and Data THREDDS Middleware

LEAD: A Large-ITR Effort Linked Environments for Atmospheric Discovery –Identify, Access, Assimilate, Predict, Manage, Mine, and Visualize a broad array of meteorological data and model output, independent of format and physical location –A range of Grid and Web Services will be developed for dynamic, on-demand, end-to-end weather prediction –Institutions: U. Oklahoma, Unidata, U. Alabama, U. Illinois, U. Indiana, Millersville U., Howard U. and Colorado State U.

Web Services They are self-contained, self-describing, modular applications that can be published, located, and invoked across the Web. Web Services are emerging as tools for creating next generation distributed systems that are expected to facilitate program-to-program interaction without the user-to-program interaction. Besides recognizing the heterogeneity as a fundamental ingredient, these web services, independent of platform and environment, can be packaged and published and they can communicate with other systems using the common protocols.

User running local analysis and display tools Data Service Decoder Service Assimilation Service Regional Model Service User Orchestrates Web Services to Create Regional Forecast Product Generation & Data Mining Service LEAD Vision