Concept of Uncertainty - Resources and NRC’s report

Slides:



Advertisements
Similar presentations
Chapter 13 – Weather Analysis and Forecasting
Advertisements

Meteorological Service of Canada – Update Meteorological Service of Canada – Update NOAA Satellite Proving Ground/User-Readiness June 2, 2014 David Bradley.
Dr. Eve Gruntfest Dr. Heather Lazrus American Meteorological Society Summer Community Meeting Norman, OK - August 11, the National Weather Center.
EISWG The Environmental Information Services Working Group of the NOAA Science Advisory Board Raymond J. Ban April 22, 2009.
Unisys Weather Information Services Presentation for NWS Partners Meeting Partner Perspective June 2010 Ron Guy, Director Unisys Weather
1 of Introduction to Forecasts and Verification.
Upcoming Changes in Winter Weather Operations at the Weather Prediction Center (WPC) Great Lakes Operational Meteorological Workshop Dan Petersen, Wallace.
February High Impact Weather Workshop 1 JCSDA-HFIP and -ECMWF Workshop Recommendations Lars Peter Riishojgaard and Sid Boukabara Joint Center for.
Predictability and Chaos EPS and Probability Forecasting.
Verification and evaluation of a national probabilistic prediction system Barbara Brown NCAR 23 September 2009.
Probabilistic QPF Steve Amburn, SOO WFO Tulsa, Oklahoma.
2012: Hurricane Sandy 125 dead, 60+ billion dollars damage.
1 WRF Development Test Center A NOAA Perspective WRF ExOB Meeting U.S. Naval Observatory, Washington, D.C. 28 April 2006 Fred Toepfer NOAA Environmental.
NOAA Climate Program – An Update NOAA Science Advisory Board March 19, 2003 NOAA Science Advisory Board March 19, 2003 Mary M. Glackin NOAA Assistant Administrator.
Introduction to Numerical Weather Prediction and Ensemble Weather Forecasting Tom Hamill NOAA-CIRES Climate Diagnostics Center Boulder, Colorado USA.
Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.
UCAR These organizations have created guidance for the next Administration and Congress on making our nation more resilient to the impacts of severe weather.
Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.
Application of seasonal climate forecasts to predict regional scale crop yields in South Africa Trevor Lumsden and Roland Schulze School of Bioresources.
MDSS Challenges, Research, and Managing User Expectations - Weather Issues - Bill Mahoney & Kevin Petty National Center for Atmospheric Research (NCAR)
Medium Range Forecast - Global System Out To 14 Days Yuejian Zhu Ensemble Team Leader EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March.
1 Ensemble Reforecasts Presented By: Yuejian Zhu (NWS/NCEP) Contributors:
1 Global Ensemble Strategy Presented By: Yuejian Zhu (NWS/NCEP) Contributors: EMC staff.
CORDEX Scope, or What is CORDEX?  Provide a set of regional climate scenarios (including uncertainties) covering the period , for the majority.
1 Addressing Critical Skills Shortages at the NWS Environmental Modeling Center S. Lord and EMC Staff OFCM Workshop 23 April 2009.
1 COMPLETING THE FORECAST: ASSESSING AND COMMUNICATING FORECAST UNCERTAINTY Zoltan Toth Environmental Modeling Center NOAA/NWS/NCEP Acknowledgements: NRC.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss High-resolution data assimilation in COSMO: Status and.
NOAA’s National Weather Service National Digital Forecast Database: Status Update LeRoy Spayd Chief, Meteorological Services Division Unidata Policy Committee.
Identifying Grand Challenges in Climate Change Research: Guiding DOE’s Strategic Planning: Report on the DOE/BERAC workshop March Crystal City For.
Products and Forecasts - Examples and experiences Yuejian Zhu and Hong Guan Ensemble Team EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March.
The Australian Community Climate Earth-System Simulator The Australian Community Climate and Earth System Simulator Kamal Puri (ACCESS Group Leader)
1 Precipitation verification Precipitation verification is still in a testing stage due to the lack of station observation data in some regions
Collaborating on the Development of Warn-On-Forecast Mike Foster / David Andra WFO Norman OK Feb. 18, 2010 Mike Foster / David Andra WFO Norman OK Feb.
Communicating Climate and Weather Information Chris Elfring, Director Board on Atmospheric Sciences and Climate The National Academies 500 Fifth Street.
1 Agenda Topic: National Blend Presented By: Kathryn Gilbert (NWS/NCEP) Team Leads: Dave Myrick, David Ruth (NWS/OSTI/MDL), Dave Novak (NCEP/WPC), Jeff.
David Mogk Dept. of Earth Sciences Montana State University April 8, 2015 Webinar SAGE/GAGE FACILITIES SUPPORTING BROADER EDUCATIONAL IMPACTS: SOME CONTEXTS.
Ensemble Forecasting and You The very basics Richard H. Grumm National Weather Service State College PA
Global Observing System Simulation Experiments (Global OSSEs) How It Works Nature Run 13-month uninterrupted forecast produces alternative atmosphere.
PARTNERING WITH THE NATIONAL SCIENCE FOUNDATION Michael C. Morgan Director, Division of Atmospheric and Geospace Sciences National Science Foundation.
3 rd Annual WRF Users Workshop Promote closer ties between research and operations Develop an advanced mesoscale forecast and assimilation system   Design.
The NOAA Operational Model Archive and Distribution System NOMADS The NOAA Operational Model Archive and Distribution System NOMADS Dave Clark for Glenn.
The Challenge of Mesoscale Probabilistic Weather Prediction and Communication Cliff Mass University of Washington.
NOAA Intra-Seasonal to Interannual Prediction (ISIP) and Climate Prediction Program for Americas (CPPA) Jin Huang NOAA Office of Global Programs November.
1 Partnership Roles NWS Goes Digital Edward Johnson, Director Strategic Planning and Policy.
CBRFC Stakeholder Forum February 24, 2014 Ashley Nielson Kevin Werner NWS Colorado Basin River Forecast Center 1 CBRFC Forecast Verification.
Communicating Weather Forecast Uncertainty Information Andrea Bleistein and Julie Demuth Summer WAS*IS, July 18, 2007.
Climate Services: The Pacific Climate Information System (PaCIS) Approach Eileen L. Shea NOAA IDEA Center 31 st Climate Diagnostics.
RFC Climate Requirements 2 nd NOAA Climate NWS Dialogue Meeting January 4, 2006 Kevin Werner.
CIRES-CDC Institutional Partnership Western Water Assessment Review Randall M. Dole, Director Climate Diagnostics Center September 2001.
4th IPWG Workshop Chinese Meteorological Agency, Beijing, China, October, 2008 MAINSTREAMING THE OPERATIONAL USE OF SATELLITE PRECIPITATION DATA.
Wanda R. Ferrell, Ph.D. Acting Director Climate and Environmental Sciences Division February 24, 2010 BERAC Meeting Atmospheric System Research Science.
Nathalie Voisin 1, Florian Pappenberger 2, Dennis Lettenmaier 1, Roberto Buizza 2, and John Schaake 3 1 University of Washington 2 ECMWF 3 National Weather.
Proposed THORPEX/HEPEX Hydrologic Ensemble Project (THEPS) Presentation for 3 rd THORPEX Science Symposium September 14-18, 2009 Prepared by John Schaake,
Fly - Fight - Win 2 d Weather Group Mr. Evan Kuchera HQ AFWA 2 WXG/WEA Template: 28 Feb 06 Approved for Public Release - Distribution Unlimited AFWA Ensemble.
Focus areas of the NWS Missouri/Souris River Floods of May-August 2011 Service Assessment – Per the NOAA and NWS Strategic Plans, gather stakeholder input.
Findings and Recommendations from the Hydrologic Verification System Requirements Team Peter Gabrielsen, Julie Demargne, Mary Mullusky, Kevin Werner, Bill.
U N I T E D S T A T E S D E P A R T M E N T O F C O M M E R C E N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N.
NOAA Northeast Regional Climate Center Dr. Lee Tryhorn NOAA Climate Literacy Workshop April 2010 NOAA Northeast Regional Climate.
CGMS-43-NOAA-WP-25 Coordination Group for Meteorological Satellites - CGMS NOAA Use of Soil Moisture Products Presented to CGMS-43 Working Group 2 session,
RFC Verification Workshop What to Expect Julie Demargne and Mary Mullusky RFC Verification Workshop, August
Future directions of CEOS WGCV: Incoming chair perspective
Overview of Downscaling
Dynamical Models - Purposes and Limits
CLIVAR/WCRP Issues Imperatives WCRP Restructuring
Andy Wood and Dennis P. Lettenmaier
Joint Planning and Development Office “Where new ideas are welcome”
MOGREPS developments and TIGGE
Summary of the 7th NCEP Ensemble User Workshop
Presentation transcript:

Concept of Uncertainty - Resources and NRC’s report Yuejian Zhu Ensemble Team Leader EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 30, 2015, Fuzhou, Fujian, China

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Lorenz Equation ( Chaos in ODE) X-Z Plane X-Y Plane Lorenz Equation ( Chaos in ODE) The Lorenz model is defined by three differential equations given the time evolution of variables X(t), Y(t), Z(t) http://crossgroup.caltech.edu/Chaos_Course/Lesson1/Demos.html Power Spectrum

Errors? Errors? Errors? Errors? Errors? Errors?

Ensemble Forecast System Describe Forecast Uncertainty Arising Due To Chaos Buizza 2002

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Forecast Uncertainty 预报的不确定性 预报误差 不确定性 Instrument 仪器 Assimilation 同化 Observation 观测 Assimilation 同化 Analysis 分析 预报误差 Forecast Errors Uncertainty 不确定性 Parameterization 模式参数化 Computation 计算过程

Earlier Ensemble Forecast System One day advantage Due to model imperfection

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Ensemble Forecasts Deterministic forecast Initial uncertainty Forecast probability Verified analysis

Real example for bimodality

Top: 2-m temperature probabilistic forecast (10% and 90%) verification 2-m temp 10/90 probability forecast verification Northern Hem, period of Dec. 2007 – Feb. 2008 90% 3-month verifications 10% Top: 2-m temperature probabilistic forecast (10% and 90%) verification red: perfect, blue: raw, green: NAEFS Left: example of probabilistic forecasts (meteogram) for Washington DC, every 6-hr out to 16 days from 2008042300

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Ensemble forecast is widely used in daily weather forecast Uncertainties & disagreements Ensemble forecast is widely used in daily weather forecast

Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds

Example of forecast precipitation (next 24 hours) (assume all forecasts are bias free – after bias correction or calibration) 100 75 50 50 50 50 80% 80% mean mean 25 deterministic probabilistic probabilistic Fcst 1 Fcst 2 Fcst 3 MM/24hrs Which forecast is relatively easier? Or more confident! Can we identify all these information?

Resolution makes difference for Typhoon Morakot Ini: 2009080600 T126 ensemble T190 ensemble Most models do not make right forecasts Ini: 2009080700 T126 ensemble T190 ensemble 17 Is this a good example for uncertainty forecast???

Is this a good example to discuss forecast consistency and accuracy? Typhoon Megi 2010101512 2010101612 NCEP CMC ECMWF 3-ENS 2010101712 2010101812 Is this a good example to discuss forecast consistency and accuracy?

Thick blue: ensemble mean 00UTC Bimodality? Thick blue: ensemble mean Opr: T254L42 (55km) Para: T574L64 (33km) 20121022 (8 days) Red arrow means good forecast 06UTC

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Completing the Forecast Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts Coming soon in Chinese…… US NRC report (174 pages) The National Academic Press 2006

Completing the Forecast Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts Board on Atmospheric Sciences and Climate

Task Provide guidance on how to identify and characterize needs for uncertainty information among various users of forecasts. Support this with examples of needs. Identify limitations in current methods for estimating and validating forecast uncertainty, relating these limitations to users’ needs and recommend improvements or new methods and approaches. Identify sources of misunderstanding in communicating forecast uncertainty, including vulnerabilities dependent on the means of communication used, and recommend improvements in the ways used to communicate forecast uncertainty. Recognizing the breadth and depth of this task, NWS advised the committee at its opening meeting to "teach us how to fish as opposed to giving us a fish."

Report Contents Summary Introduction Uncertainty in Decision Making Estimating and Validating Uncertainty Communicating Forecast Uncertainty Overarching Recommendations

Recommendations The entire Enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information. NWS should take a leadership role in this effort. NWS should improve its product development process by collaborating with users and partners in the Enterprise from the outset and engaging and using social and behavioral science expertise. All sectors and professional organizations of the Enterprise should cooperate in educational initiatives that will improve communication and use of uncertainty information. NWS should develop and maintain the ability to produce objective uncertainty information from the global to the regional scale.

Recommendations To ensure widespread use of uncertainty information, NWS should make all raw and post-processed probabilistic products easily accessible to the enterprise at full spatial and temporal resolution. Sufficient computer and communications resources should be acquired to ensure effective access by external users and NWS personnel. NWS should expand verification of its uncertainty products and make this information easily available to all users in near real time. A variety of verification measures and approaches (measuring multiple aspects of forecast quality that are relevant for users) should be used to appropriately represent the complexity and dimensionality of the verification problem. Verification statistics should be computed for meaningful subsets of the forecasts (e.g., by season, region) and should be presented in formats that are understandable by forecast users. Archival verification information on probabilistic forecasts, including model-generated and objectively generated forecasts and verifying observations, should be accessible so users can produce their own evaluation of the forecasts.

Recommendations To enhance development of new methods in estimation, communication, and use of forecast uncertainty information throughout the Enterprise, and to foster and maintain collaboration, confidence, and goodwill with Enterprise partners, NWS should more effectively use testbeds by involving all sectors of the Enterprise. The committee endorses the recommendation by the NRC “Fair Weather” report to establish an independent advisory committee and encourages NOAA to bring its evaluation of the recommendation to a speedy and positive conclusion. NWS should dedicate executive attention to coordinating the estimation and communication of uncertainty information within NWS and with Enterprise partners.

Key points not to forget Broad applicability within NOAA and other agencies This will take time (turn the aircraft carrier) Let’s get started now NWS evolution - time is right to incorporate uncertainty information into process Uncertainty is another component of improving the forecast – it’s not detached from the primary mission of forecast improvement Uncertainty is not an add-on – it is in the nature of the product