Concept of Uncertainty - Resources and NRC’s report Yuejian Zhu Ensemble Team Leader EMC/NCEP/NWS/NOAA Presents for NWP Forecast Training Class March 30, 2015, Fuzhou, Fujian, China

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Lorenz Equation ( Chaos in ODE) X-Z Plane X-Y Plane Lorenz Equation ( Chaos in ODE) The Lorenz model is defined by three differential equations given the time evolution of variables X(t), Y(t), Z(t) http://crossgroup.caltech.edu/Chaos_Course/Lesson1/Demos.html Power Spectrum

Errors? Errors? Errors? Errors? Errors? Errors?

Ensemble Forecast System Describe Forecast Uncertainty Arising Due To Chaos Buizza 2002

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Forecast Uncertainty 预报的不确定性 预报误差 不确定性 Instrument 仪器 Assimilation 同化 Observation 观测 Assimilation 同化 Analysis 分析 预报误差 Forecast Errors Uncertainty 不确定性 Parameterization 模式参数化 Computation 计算过程

Earlier Ensemble Forecast System One day advantage Due to model imperfection

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Ensemble Forecasts Deterministic forecast Initial uncertainty Forecast probability Verified analysis

Real example for bimodality

Top: 2-m temperature probabilistic forecast (10% and 90%) verification 2-m temp 10/90 probability forecast verification Northern Hem, period of Dec. 2007 – Feb. 2008 90% 3-month verifications 10% Top: 2-m temperature probabilistic forecast (10% and 90%) verification red: perfect, blue: raw, green: NAEFS Left: example of probabilistic forecasts (meteogram) for Washington DC, every 6-hr out to 16 days from 2008042300

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Ensemble forecast is widely used in daily weather forecast Uncertainties & disagreements Ensemble forecast is widely used in daily weather forecast

Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds

Example of forecast precipitation (next 24 hours) (assume all forecasts are bias free – after bias correction or calibration) 100 75 50 50 50 50 80% 80% mean mean 25 deterministic probabilistic probabilistic Fcst 1 Fcst 2 Fcst 3 MM/24hrs Which forecast is relatively easier? Or more confident! Can we identify all these information?

Resolution makes difference for Typhoon Morakot Ini: 2009080600 T126 ensemble T190 ensemble Most models do not make right forecasts Ini: 2009080700 T126 ensemble T190 ensemble 17 Is this a good example for uncertainty forecast???

Is this a good example to discuss forecast consistency and accuracy? Typhoon Megi 2010101512 2010101612 NCEP CMC ECMWF 3-ENS 2010101712 2010101812 Is this a good example to discuss forecast consistency and accuracy?

Thick blue: ensemble mean 00UTC Bimodality? Thick blue: ensemble mean Opr: T254L42 (55km) Para: T574L64 (33km) 20121022 (8 days) Red arrow means good forecast 06UTC

Concept of Uncertainty – Resources and NRC’s Report Definition of uncertainty Uncertainty is kind of natural Uncertainty is kind of error Errors are from: Observation instruments Reading/recording observations Initial analysis – data assimilation Model physics and parameterizations Computations Model developer: Using ensemble to simulate the uncertainty maximally/accurately Find out analysis uncertainty Find out model uncertainty (dynamic and physic) Forecasters: Using ensemble information to make uncertainty forecast Example 1: probability forecast Example 2: deterministic forecast with uncertainty level Example 3: uncertainty forecast Completing forecast US NRC’s report – fundamental education material

Completing the Forecast Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts Coming soon in Chinese…… US NRC report (174 pages) The National Academic Press 2006

Completing the Forecast Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts Board on Atmospheric Sciences and Climate

Task Provide guidance on how to identify and characterize needs for uncertainty information among various users of forecasts. Support this with examples of needs. Identify limitations in current methods for estimating and validating forecast uncertainty, relating these limitations to users’ needs and recommend improvements or new methods and approaches. Identify sources of misunderstanding in communicating forecast uncertainty, including vulnerabilities dependent on the means of communication used, and recommend improvements in the ways used to communicate forecast uncertainty. Recognizing the breadth and depth of this task, NWS advised the committee at its opening meeting to "teach us how to fish as opposed to giving us a fish."

Report Contents Summary Introduction Uncertainty in Decision Making Estimating and Validating Uncertainty Communicating Forecast Uncertainty Overarching Recommendations

Recommendations The entire Enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information. NWS should take a leadership role in this effort. NWS should improve its product development process by collaborating with users and partners in the Enterprise from the outset and engaging and using social and behavioral science expertise. All sectors and professional organizations of the Enterprise should cooperate in educational initiatives that will improve communication and use of uncertainty information. NWS should develop and maintain the ability to produce objective uncertainty information from the global to the regional scale.

Recommendations To ensure widespread use of uncertainty information, NWS should make all raw and post-processed probabilistic products easily accessible to the enterprise at full spatial and temporal resolution. Sufficient computer and communications resources should be acquired to ensure effective access by external users and NWS personnel. NWS should expand verification of its uncertainty products and make this information easily available to all users in near real time. A variety of verification measures and approaches (measuring multiple aspects of forecast quality that are relevant for users) should be used to appropriately represent the complexity and dimensionality of the verification problem. Verification statistics should be computed for meaningful subsets of the forecasts (e.g., by season, region) and should be presented in formats that are understandable by forecast users. Archival verification information on probabilistic forecasts, including model-generated and objectively generated forecasts and verifying observations, should be accessible so users can produce their own evaluation of the forecasts.

Recommendations To enhance development of new methods in estimation, communication, and use of forecast uncertainty information throughout the Enterprise, and to foster and maintain collaboration, confidence, and goodwill with Enterprise partners, NWS should more effectively use testbeds by involving all sectors of the Enterprise. The committee endorses the recommendation by the NRC “Fair Weather” report to establish an independent advisory committee and encourages NOAA to bring its evaluation of the recommendation to a speedy and positive conclusion. NWS should dedicate executive attention to coordinating the estimation and communication of uncertainty information within NWS and with Enterprise partners.

Key points not to forget Broad applicability within NOAA and other agencies This will take time (turn the aircraft carrier) Let’s get started now NWS evolution - time is right to incorporate uncertainty information into process Uncertainty is another component of improving the forecast – it’s not detached from the primary mission of forecast improvement Uncertainty is not an add-on – it is in the nature of the product