Presentation is loading. Please wait.

Presentation is loading. Please wait.

Meteorological Assimilation Data Ingest System (MADIS)

Published byHendra Kurnia Modified over 6 years ago

Similar presentations

Presentation on theme: "Meteorological Assimilation Data Ingest System (MADIS)"— Presentation transcript:

1 Meteorological Assimilation Data Ingest System (MADIS)
Surface Data Density Before MADIS Surface Data Density After MADIS Patty Miller Unidata Webcast April 28, 2009 November 18, 2018

2 MADIS Background History Goal
MADIS was established in 2001 to prototype new observation ingest, integration, quality control, and distribution techniques for real time and saved real-time data Goal To integrate and quality control NOAA and other-agency observations and make them easily accessible and usable for operations, research, and commercial purposes November 18, 2018

3 MADIS Background (continued)
Overall Benefits A more usable, complete, accurate, timely, and higher density observational infrastructure for use in local weather warnings and products, model predictions, and hazardous situations NWS-Specific Benefits Improved observational functionality for… enhancing forecaster situational awareness reducing data access costs for Forecast Offices supporting higher-resolution global and regional data assimilation systems improving the National Digital Forecast Database November 18, 2018

4 MADIS Function Features
Observation access, integration, quality control, and distribution system with software support Features Access to real-time and saved real-time data sets Observational quality control Application Program Interface (API) Multiple network-enabled data distribution mechanisms (ftp, http, ldm) Documentation and user support, including customization packages for NWS’s Advanced Weather Interactive Processing System (AWIPS) November 18, 2018

5 MADIS System Capabilities
Seamless access to real-time and saved datasets Continuous database updates triggered by arriving observations Uniform observation formats, units, and time stamps Automated quality control algorithms Station monitoring for network maintenance Secure authentication for proprietary data Web-enabled push/pull distribution capabilities, with server-side slice and dice capabilities On-the-fly data reformatting, variable transformation, and sounding generations November 18, 2018

6 Observational Datasets Hundreds of MADIS Users, Including:
MADIS Current Status Observational Datasets MADIS supports the collection, integration, quality control, and distribution of thousands of NOAA and non-NOAA observations, including over 50K surface stations from local, state, and federal agencies, and private networks, as well as upper-air datasets including multi-agency profiler, radiosonde, radiometer, selected satellite observations, and commercial aircraft observations. Scope 56,864 Surface Stations producing over 12,800,000 observations/day 154 Profiler Sites (> 200,000 obs/day) Over 450,000 aircraft observations/day Plus global radiosonde and satellite obs Hundreds of MADIS Users, Including: NWS Forecast Offices, National Centers NSSL, AOML, ARL, NESDIS, NOS, + NCAR and NASA over 100 universities DOE laboratories Accuweather WSI Corporation DTN Meteorlogix AWS/WeatherBug Baron Services Weather Underground Profiler data includes NOAA Profiler Network and Cooperating Agency Profilers Aircraft data includes MDCRS, AMDAR, TAMDAR, and WVSS-2 Surface data includes METAR, maritime, snow, UrbaNet, and other mesonet November 18, 2018

7 Observing System Portfolio
Current Surface Sites METAR = 6,397 Maritime = 192 Meteorological Mesonet = 27,920 Hydrological Mesonet = 20,885 UrbaNet = 1,470 Total = 56,864 Networks Processed > 170 Data Variables = 144 Metadata Variables = 55 Observations / Day > 12,800,000 November 18, 2018

8 For more information see
MADIS QC Capabilities by Observation Type For more information see November 18, 2018

9 MADIS SLP (MB) Errors November 18, 2018

10 Standard Surface Observations
MADIS Northeast Standard Surface Network Standard Surface Observations Meteorological Aviation Reports (METARs) Maritime Surface Aviation Observations (SAOs) November 18, 2018

11 Additional Surface Observations
MADIS Northeast Surface Network Additional Surface Observations - New England Pilot Project (NEPP) - AWS Convergence Technologies, Inc. - Citizen Weather Observer Program - Remote Automated Weather Stations - ESRL Ground-Based GPS Meteorology - Weather for You.com - Anything Weather - Soil Climate Analysis Network (SCAN) - Gulf of Maine Ocean Observing System National Ocean Service Physical Oceanographic Real-Time System (PORTS) and National Water Level Observation Network (NWLON) - Aberdeen Proving Grounds (APG) OAR – DCNet UrbaNet Hydrometeorological Automated Data System (HADS) North Carolina ECONet - New Jersey Weather and Climate Network - DoTs: GA, KY, ME, MD, NH, OH, VA, VT November 18, 2018

12 MADIS Northeast Regional Domain Observations
November 18, 2018

13 Standard Surface Observations
MADIS Midwest Standard Surface Network Standard Surface Observations Meteorological Aviation Reports (METARs) Maritime Surface Aviation Observations (SAOs) November 18, 2018

14 Additional Surface Observations
MADIS Midwest Surface Network Additional Surface Observations - AWS Convergence Technologies, Inc. - Citizen Weather Observer Program - Remote Automated Weather Stations - ESRL Ground-Based GPS Meteorology - Weather for You.com - Anything Weather - Soil Climate Analysis Network (SCAN) - National Ocean Service Physical Oceanographic Real-Time System (PORTS) and National Water Level Observation Network (NWLON) - UrbaNet - Oklahoma Mesonet - DOTs: IA, IN, KS, KY, MN, ND, OH, WI Marquette Mesonet Union Pacific Railroad Non-Federal AWOS NERRS (National Estuarine Research Reserve System) CoCoRaHS Hydrometeorological Automated Data System (HADS) - North Carolina ECONet November 18, 2018

15 MADIS Midwest Regional Domain Observations
November 18, 2018

16 Standard Surface Observations
MADIS West Coast Standard Surface Network Standard Surface Observations Meteorological Aviation Reports (METARs) Maritime November 18, 2018

17 Additional Surface Observations
MADIS West Coast Surface Network Additional Surface Observations - AWS Convergence Technologies, Inc. - Citizen Weather Observer Program - Remote Automated Weather Stations - Weather for You.com - Anything Weather - Soil Climate Analysis Network (SCAN) - National Ocean Service Physical Oceanographic Real-Time System (PORTS) and National Water Level Observation Network (NWLON) - Union Pacific Railroad - Snow Information and Technology (SNOTEL) - CA River/Nevada River Forecast Center - CoCoRaHS - U.S. Bureau of Reclamation Agrimet - Pacific Northwest National Laboratory - CO River Basin Forecast Center Dugway Proving Grounds Non-Federal AWOS UrbaNet - DOTs: CA, CO, ID, MT, NV, UT, WY and many more mesonets… November 18, 2018

18 MADIS West Coast Regional Domain Observations
November 18, 2018

19 MADIS Research to Operations
The NOAA MADIS Independent Review Team Purpose To assist NOAA management in making decisions on how best to transition MADIS into NOAA operations IRT Members NESDIS Al Powell (IRT Chair/Director, Center for Satellite Applications and Research) NWS David Caldwell (Director, Office of Climate, Water, and Weather Services) Allan Darling (Chief, Software Branch/Telecommunications Operations Center) Brent Gordon (Chief, NCEP Central Operations/Systems Integration Branch) OAR James Kimpel (Director, National Severe Storms Laboratory) Eddie Bernard (Director, Pacific Marine Environmental Laboratory) Jeremy Warren (Deputy Chief Information Officer) November 18, 2018

20 MADIS Research to Operations (continued)
The NOAA MADIS Independent Review Team unanimously selected a joint OAR/NWS distributed processing solution Transition Goals Expedite the transition of current GSD capabilities to operations Maintain the continuity of MADIS data streams and services before, during, and after the transition Pre-plan for product improvements and technology infusion Summary Statement “The partnership between OAR and NWS led to a solid technical solution and provided a smoother transition from research to operations.” November 18, 2018

21 MADIS Compute Environment
MADIS Research to Operations (continued) The NOAA MADIS Independent Review Team Technical Recommendation Port the existing GSD MADIS software to an integrated NWS TOC and NCO distributed environment, with a supporting research-to-operation test environment at GSD MADIS Compute Environment NCEP TOC Decoders Ingest Distribution Integration and Quality Control November 18, 2018

22 MADIS Transition Plans Time Table to IOC
FY2009 FY2010 October 1, 2008 2010 2009 Port Real-Time Processing to NCEP Install I/O Software and Hardware at the TOC October 1, 2009 End-to-End Testing Training / C&A OT&E April 1, Resolve OT&E Issues Port I/O Software and Hardware to the TOC IOC June 3, 2010 November 18, 2018

23 Port Data Recovery Process to NCEP
MADIS Transition Plans Time Table to FOC FY2010 FY2011 FY2012 June 1, 2010 April 1, 2011 April 1, 2012 Port Data Recovery Process to NCEP Port Web Displays to TOC MADIS Servers CAP Port to TOC Install CAP Hardware and Software January 14, 2011 … Prepare Archive for Transfer to NCDC End-to-End Testing OT&E (4/14/11 – 4/29/11) FOC June 14, 2011 Resolve OT&E October 1, 2010 October 1, 2011 November 18, 2018

24 Post-FOC Product Improvement
Product improvements such as: 1) advanced data query and web services; 2) expanded metadata fields; 3) additional datasets; and 4) improved and expanded observation QC will serve: NOAA Operations NextGen – includes high frequency ASOS National Surface Weather Observing System (NSWOS)/FHWA support Historic Climate Network – Modernized (HCN-M) UrbaNet, National Mesonet Next Generation NOAA Profiler Network (NGNPN) Testbeds (HMT, DTC, Severe Weather Testbed) Fire weather mobile observations DHS research support UAS data management NOAA Research November 18, 2018

25 Network Enabled Operations
Observations: MADIS NextGen Services Data Integration 4D Wx Cube Network Enabled Operations Web Services Data Integration Quality Controlled Observations Metadata Accuracy and Availability Statistics Net-Enabled Observation Subsetting and Web Responses Radiosonde Vector Winds Soundings Radiance Satellites ASOS/AWOS/SAO Marine DoT/RWIS Integrated Mesonet Climate Mobile Platforms Snow Fall/Depth Surface NOAA Profilers Cooperative Profilers Radiometers Ceilometers Sodars Sounding Systems AMDAR TAMDAR UAS Aircraft NexRad TDWR/ARSR Private Radars November 18, 2018

26 Questions? Patricia.A.Miller@noaa.gov MADIS Home Page URL
November 18, 2018

27 MADIS Supplemental Slides Follow November 18, 2018

28 MADIS URLs Home Page Real-Time Surface Observation Display
madis.noaa.gov/ Real-Time Surface Observation Display www-frd.fsl.noaa.gov/mesonet/ Real-Time Profiler Display Real-Time Aircraft Display acweb.fsl.noaa.gov Real-Time Upper Air Soundings www-frd.fsl.noaa.gov/soundings/java November 18, 2018

29 Surface Observation Web Page
November 18, 2018

Download ppt "Meteorological Assimilation Data Ingest System (MADIS)"

Similar presentations

Ken Crawford, Program Director COOP Modernization and Integrated Surface Observing Systems NWS Office of Science and Technology

Ken Crawford, Program Director COOP Modernization and Integrated Surface Observing Systems NWS Office of Science and Technology
1st National Weather and Climate Enterprise Partnership Summit John Horel NOAA Cooperative Institute for Regional Prediction Department of Meteorology.

1st National Weather and Climate Enterprise Partnership Summit John Horel NOAA Cooperative Institute for Regional Prediction Department of Meteorology.
6/12/20141 NoN Recommendations: Measurements and Infrastructure Meteorological Assimilation Data Ingest System (MADIS) –Update Current Services and Capabilities.

6/12/20141 NoN Recommendations: Measurements and Infrastructure Meteorological Assimilation Data Ingest System (MADIS) –Update Current Services and Capabilities.
3rd GOES User’s Conference Report on the NESDIS Data User’s Conference Ken Knapp GOES User’s Conference 10-13 May 2004.

3rd GOES User’s Conference Report on the NESDIS Data User’s Conference Ken Knapp GOES User’s Conference May 2004.
Dissemination Portfolio 101

Dissemination Portfolio 101
SMOS SAG, Villafranca November 2-3, 2006 Development of a Global In-Situ Soil Moisture Network: A SMOS Project Contribution P.J. van Oevelen.

SMOS SAG, Villafranca November 2-3, 2006 Development of a Global In-Situ Soil Moisture Network: A SMOS Project Contribution P.J. van Oevelen.
Atmospheric Sciences 370 Observing Systems January 2007.

Atmospheric Sciences 370 Observing Systems January 2007.
The Gulf of Maine Ocean Observing System. Technical Program Real time monitoring and forecasts of: Weather - surface ocean winds, air temperature, visibility.

The Gulf of Maine Ocean Observing System. Technical Program Real time monitoring and forecasts of: Weather - surface ocean winds, air temperature, visibility.
Plans for AWIPS Next Generation 1April 12, 2005 AWIPS Technology Infusion Darien Davis NOAA/OAR Forecast Systems Laboratory Systems Development Division.

Plans for AWIPS Next Generation 1April 12, 2005 AWIPS Technology Infusion Darien Davis NOAA/OAR Forecast Systems Laboratory Systems Development Division.
GOES Users’ Conference The Role of Geostationary Satellites in WMO’s Global Observing System Dr. Donald E. Hinsman Senior Scientific Officer WMO Satellite.

GOES Users’ Conference The Role of Geostationary Satellites in WMO’s Global Observing System Dr. Donald E. Hinsman Senior Scientific Officer WMO Satellite.
March 14, 2006Intl FFF Workshop, Costa Rica Weather Decision Technologies, Inc. Hydro-Meteorological Decision Support System Bill Conway, Vice President.

March 14, 2006Intl FFF Workshop, Costa Rica Weather Decision Technologies, Inc. Hydro-Meteorological Decision Support System Bill Conway, Vice President.
Recovery Act Smart Grid Funds: $4.5 Billion Office of Electricity Delivery and Energy Reliability$ Millions Smart Grid Investment Grant Program; ≤3 years.

Recovery Act Smart Grid Funds: $4.5 Billion Office of Electricity Delivery and Energy Reliability$ Millions Smart Grid Investment Grant Program; ≤3 years.
NOAA PORTS ® Partnerships MARACOOS Annual Meeting 12/15/2011 Darren Wright Maritime Services Program Manager.

NOAA PORTS ® Partnerships MARACOOS Annual Meeting 12/15/2011 Darren Wright Maritime Services Program Manager.
MADIS to LITTLE_R Converter MADIS observation types MADIS to LITTE_R converter Future work.

MADIS to LITTLE_R Converter MADIS observation types MADIS to LITTE_R converter Future work.
Workshop on DEVELOPING REGIONAL COOPERATION FOR SHARED KARST AQUIFER MANAGEMENT IN SEE 27-28 June 2008 Thessaloniki, Greece Monitoring data and existing.

Workshop on DEVELOPING REGIONAL COOPERATION FOR SHARED KARST AQUIFER MANAGEMENT IN SEE June 2008 Thessaloniki, Greece Monitoring data and existing.
MADIS – The Meteorological Assimilation Data Ingest System Current Operational Status and Future Plans Greg Pratt, Leon Benjamin, Thomas Kent, Gopa Padmanabhan,

MADIS – The Meteorological Assimilation Data Ingest System Current Operational Status and Future Plans Greg Pratt, Leon Benjamin, Thomas Kent, Gopa Padmanabhan,
1 Status of NERON/HCN-M for The Committee for Climate Analysis, Monitoring, and Services (CCAMS) John Hahn NWS Office of Science and Technology.

1 Status of NERON/HCN-M for The Committee for Climate Analysis, Monitoring, and Services (CCAMS) John Hahn NWS Office of Science and Technology.
Accessing and Utilizing AMDAR Data for NWP Applications David R. Helms Office of Science and Technology NOAA National Weather Service AMDAR Regional Workshop.

Accessing and Utilizing AMDAR Data for NWP Applications David R. Helms Office of Science and Technology NOAA National Weather Service AMDAR Regional Workshop.
1 Global Systems Division (GSD) Earth System Research Laboratory (ESRL) NextGen Weather Data Cube Chris MacDermaid October, 2010.

1 Global Systems Division (GSD) Earth System Research Laboratory (ESRL) NextGen Weather Data Cube Chris MacDermaid October, 2010.
IMGW – Poland Presentation U.S. Approach to Integrated Water Resources and Flood Emergency Management March,2003 U.S. Approach to Integrated Water Resources.

IMGW – Poland Presentation U.S. Approach to Integrated Water Resources and Flood Emergency Management March,2003 U.S. Approach to Integrated Water Resources.

Similar presentations

Ads by Google