Ensembles and Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This.

Slides:



Advertisements
Similar presentations
Chapter 13 – Weather Analysis and Forecasting
Advertisements

Report of the Q2 Short Range QPF Discussion Group Jon Ahlquist Curtis Marshall John McGinley - lead Dan Petersen D. J. Seo Jean Vieux.
PRESENTS: FORECASTING FOR OPERATIONS AND DESIGN February 16 th 2011 – Aberdeen.
A Brief Guide to MDL's SREF Winter Guidance (SWinG) Version 1.0 January 2013.
Page 1© Crown copyright 2006ESWWIII, Royal Library of Belgium, Brussels, Nov 15 th 2006 Forecasting uncertainty: the ensemble solution Mike Keil, Ken Mylne,
Statistical post-processing using reforecasts to improve medium- range renewable energy forecasts Tom Hamill and Jeff Whitaker NOAA Earth System Research.
Mesoscale Probabilistic Prediction over the Northwest: An Overview Cliff Mass Adrian Raftery, Susan Joslyn, Tilmann Gneiting and others University of Washington.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with.
Introduction to Probability and Probabilistic Forecasting L i n k i n g S c i e n c e t o S o c i e t y Simon Mason International Research Institute for.
2012: Hurricane Sandy 125 dead, 60+ billion dollars damage.
Instituting Reforecasting at NCEP/EMC Tom Hamill (ESRL) Yuejian Zhu (EMC) Tom Workoff (WPC) Kathryn Gilbert (MDL) Mike Charles (CPC) Hank Herr (OHD) Trevor.
EPS Training Module 1: Introduction Richard Verret (Normand Gagnon) Meteorological Service of Canada The illusion of determinism…
Description and Preliminary Evaluation of the Expanded UW S hort R ange E nsemble F orecast System Maj. Tony Eckel, USAF University of Washington Atmospheric.
Revolutions in Remote Sensing Greatly Enhanced Weather Prediction from the 1950s Through Today.
Introduction to Weather Forecasting Cliff Mass Department of Atmospheric Sciences University of Washington.
MOS Developed by and Run at the NWS Meteorological Development Lab (MDL) Full range of products available at:
EG1204: Earth Systems: an introduction Meteorology and Climate Lecture 7 Climate: prediction & change.
Forecasting Boot Camp. Major Steps in the Forecast Process Data Collection Quality Control Data Assimilation Model Integration Post Processing of Model.
Ensembles and Probabilistic Forecasting. Probabilistic Prediction Because of forecast uncertainties, predictions must be provided in a probabilistic framework,
MOS Performance MOS significantly improves on the skill of model output. National Weather Service verification statistics have shown a narrowing gap between.
A Brief History of Weather Forecasting. The Beginning: Weather Sayings "Red Sky at night, sailor's delight. Red sky in the morning, sailor take warning."
Introduction to Numerical Weather Prediction and Ensemble Weather Forecasting Tom Hamill NOAA-CIRES Climate Diagnostics Center Boulder, Colorado USA.
The Expanded UW SREF System and Statistical Inference STAT 592 Presentation Eric Grimit 1. Description of the Expanded UW SREF System (How is this thing.
Uncertainties about Uncertainty Cliff Mass. Major Issues.
A Brief History of Weather Forecasting. The Stone Age Prior to approximately 1955, forecasting was basically a subjective art, and not very skillful.
Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.
A Regression Model for Ensemble Forecasts David Unger Climate Prediction Center.
Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.
Using Short Range Ensemble Model Data in National Fire Weather Outlooks Sarah J. Taylor David Bright, Greg Carbin, Phillip Bothwell NWS/Storm Prediction.
Forecasting Boot Camp.
Probabilistic forecasts of (severe) thunderstorms for the purpose of issuing a weather alarm Maurice Schmeits, Kees Kok, Daan Vogelezang and Rudolf van.
Forecasting and Numerical Weather Prediction (NWP) NOWcasting Description of atmospheric models Specific Models Types of variables and how to determine.
Probabilistic Prediction. Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that.
A History of Modern Weather Forecasting. The Beginning: Weather Sayings "Red Sky at night, sailor's delight. Red sky in the morning, sailor take warning."
How can LAMEPS * help you to make a better forecast for extreme weather Henrik Feddersen, DMI * LAMEPS =Limited-Area Model Ensemble Prediction.
WORKSHOP ON SHORT-RANGE ENSEMBLE PREDICTION USING LIMITED-AREA MODELS Instituto National de Meteorologia, Madrid, 3-4 October 2002 Limited-Area Ensemble.
Improving Ensemble QPF in NMC Dr. Dai Kan National Meteorological Center of China (NMC) International Training Course for Weather Forecasters 11/1, 2012,
Celeste Saulo and Juan Ruiz CIMA (CONICET/UBA) – DCAO (FCEN –UBA)
Notes on reforecasting and the computational capacity needed for future SREF systems Tom Hamill NOAA Earth System Research Lab presentation for 2009 National.
Numerical Prediction of High-Impact Local Weather: How Good Can It Get? Kelvin K. Droegemeier Regents’ Professor of Meteorology Vice President for Research.
Ensemble Forecasting and You The very basics Richard H. Grumm National Weather Service State College PA
Toward Probabilistic Seasonal Prediction Nir Krakauer, Hannah Aizenman, Michael Grossberg, Irina Gladkova Department of Civil Engineering and CUNY Remote.
© Crown copyright Met Office Probabilistic turbulence forecasts from ensemble models and verification Philip Gill and Piers Buchanan NCAR Aviation Turbulence.
1 An overview of the use of reforecasts for improving probabilistic weather forecasts Tom Hamill NOAA / ESRL, Physical Sciences Div.
Gouge – Navy slang for the bare essential knowledge to get by.
Probabilistic Forecasting. pdfs and Histograms Probability density functions (pdfs) are unobservable. They can only be estimated. They tell us the density,
Model Post Processing. Model Output Can Usually Be Improved with Post Processing Can remove systematic bias Can produce probabilistic information from.
Plans for Short-Range Ensemble Forecast at INM José A. García-Moya SMNT – INM Workshop on Short Range Ensemble Forecast Madrid, October,
A Brief History of Weather Forecasting The Beginning: Weather Sayings "Red Sky at night, sailor's delight. Red sky in the morning, sailor take warning."
The Challenge of Mesoscale Probabilistic Weather Prediction and Communication Cliff Mass University of Washington.
Probabilistic Prediction Cliff Mass University of Washington.
Verification of ensemble precipitation forecasts using the TIGGE dataset Laurence J. Wilson Environment Canada Anna Ghelli ECMWF GIFS-TIGGE Meeting, Feb.
2015 Production Suite Review: Report from NHC 2015 Production Suite Review: Report from NHC Eric S. Blake, Richard J. Pasch, Andrew Penny NCEP Production.
MPO 674 Lecture 2 1/20/15. Timeline (continued from Class 1) 1960s: Lorenz papers: finite limit of predictability? 1966: First primitive equations model.
Extracting probabilistic severe weather guidance from convection-allowing model forecasts Ryan Sobash 4 December 2009 Convection/NWP Seminar Series Ryan.
Figures from “The ECMWF Ensemble Prediction System”
Forecasting Boot Camp.
A Guide to Tropical Cyclone Guidance
LEPS VERIFICATION ON MAP CASES
Course Evaluation Now online You should have gotten an with link.
Overview of Deterministic Computer Models
Ensembles and Probabilistic Prediction
Course Evaluation Now online You should have gotten an with link.
Verifying and interpreting ensemble products
Dan Petersen Bruce Veenhuis Greg Carbin Mark Klein Mike Bodner
Probabilistic Prediction
A Review of the CSTAR Ensemble Tools Available for Operations
The Stone Age Prior to approximately 1960, forecasting was basically a subjective art, and not very skillful. Observations were sparse, with only a few.
Course Evaluation Now online You should have gotten an with link.
The Technology and Future of Weather Forecasting ATMS 490
Presentation transcript:

Ensembles and Probabilistic Prediction

Uncertainty in Forecasting All of the model forecasts I have talked about reflect a deterministic approach. This means that we do the best job we can for a single forecast and do not consider uncertainties in the model, initial conditions, or the very nature of the atmosphere. These uncertainties are often very significant. Traditionally, this has been the way forecasting has been done, but that is changing now.

The work of Lorenz (1963, 1965, 1968) demonstrated that the atmosphere is a chaotic system, in which small differences in the initialization, well within observational error, can have large impacts on the forecasts, particularly for longer forecasts. In a series of experiments found that small errors in initial conditions can grow so that all deterministic forecast skill is lost at about two weeks. A Fundamental Issue

Butterfly Effect: a small change at one place in a complex system can have large effects elsewhere

Uncertainty Extends Beyond Initial Conditions Also uncertainty in our model physics. And further uncertainty produced by our numerical methods (e.g., finite differencing truncation error, etc.).

Probabilistic NWP To deal with forecast uncertainty, Epstein (1969) suggested stochastic-dynamic forecasting, in which forecast errors are explicitly considered during model integration. Essentially, uncertainty estimates were added to each term in the primitive equation. This stochastic method was not computationally practical, since it added many additional terms.

Probabilistic-Ensemble NWP Another approach, ensemble prediction, was proposed by Leith (1974), who suggested that prediction centers run a collection (ensemble) of forecasts, each starting from a different initial state. The variations in the resulting forecasts could be used to estimate the uncertainty of the prediction. But even the ensemble approach was not possible at this time due to limited computer resources. Became practical in the late 1980s as computer power increased.

Ensemble Prediction Can use ensembles to estimate the probabilities that some weather feature will occur. The ensemble mean is more accurate on average than any individual ensemble member. Forecast skill of the ensemble mean is related to the spread of the ensembles When ensemble forecasts are similar, ensemble mean skill is higher. When forecasts differ greatly, ensemble mean forecast skill is less.

e a u c j t g n M T T Analysis Region 48h forecast Region 12h forecast 36h forecast 24h forecast phase space

A critical issue is the development of ensemble systems that provide probabilistic guidance that is both reliable and sharp.

Elements of a Good Probability Forecast Reliability (also known as calibration) –A probability forecast p, ought to verify with relative frequency p. –Forecasts from climatology are reliable (by definition), so calibration alone is not enough.

Sharpness We are trying to predict a probability density function (PDF)

Elements of a Good Probability Forecast Sharpness (a.k.a. resolution) –The variance or confidence interval of the predicted distribution should be as small as possible. Probability Density Function (PDF) for some forecast quantity Sharp Less Sharp

PDFs are created by fitting gaussian or other curves to ensemble members

More ensembles are generally better Can better explore uncertainty in initial conditions Can better explore uncertainty in model physics and numerics

Ensmbles can be calibrated

Variety of Ways to View Ensembles and Their Output

The Thanksgiving Forecast h forecast (valid Thu 10AM) 13: avn* 11: ngps* 12: cmcg* 10: tcwb* 9: ukmo* 8: eta* Verification 1: cent 7: avn 5: ngps 6: cmcg 4: tcwb 3: ukmo 2: eta - Reveals high uncertainty in storm track and intensity - Indicates low probability of Puget Sound wind event SLP and winds

Box and Whiskers NAEFS

Early Forecasting Started Probabilistically Early forecasters, faced with large gaps in their nascent science, understood the uncertain nature of the weather prediction process and were comfortable with a probabilistic approach to forecasting. Cleveland Abbe, who organized the first forecast group in the United States as part of the U.S. Signal Corp, did not use the term “forecast” for his first prediction in 1871, but rather used the term “probabilities,” resulting in him being known as “Old Probabilities” or “Old Probs” to the public. A few years later, the term ‘‘indications’’ was substituted for probabilities and by 1889 the term ‘‘forecasts’’ received official sanction (Murphy 1997).

“Ol Probs” Professor Cleveland Abbe, who issued the first public “Weather Synopsis and Probabilities” on February 19, 1871 Cleveland Abbe (“Ol’ Probabilities”), who led the establishment of a weather forecasting division within the U.S. Army Signal Corps, Produced the first known communication of a weather probability to users and the public.

History of Probabilistic Prediction The first operational probabilistic forecasts in the United States were produced in These forecasts, for the probability of precipitation, were produced by human weather forecasters and thus were subjective predictions. The first objective probabilistic forecasts were produced as part of the Model Output Statistics (MOS) system that began in 1969.

Ensemble Prediction Ensemble prediction began at NCEP in the early 1990s. ECMWF rapidly joined the club. During the past decades the size and sophistication of the NCEP and ECMWF ensemble systems have grown considerably, with the medium-range, global ensemble system becoming an integral tool for many forecasters. Also during this period, NCEP has constructed a higher resolution, short-range ensemble system (SREF) that uses breeding to create initial condition variations.

Up to date listing: u/nwp/pcu2/ens_matrix/index.ht mwww.meted.ucar.ed u/nwp/pcu2/ens_matrix/index.ht m

Major Global Ensembles NCEP GEFS (Global Ensemble Forecasting System): GFS, 21 members every 6 hr, T254 (roughly 50 km resolution), 64 levels Canadian CEFS: GEM Model, 21 members, 100 km grid spacing, 0 and 12Z ECMWF: 51 members, 62 levels, 0 and 12Z, T399 (roughly 27 km) eps/ eps/

Major International Global/Continental Ensembles Systems North American Ensemble Forecasting Systems (NAEFS): Combines Canadian and U.S. Global Ensembles: Sgrams_e.html

NCEP Short-Range Ensembles (SREF) Resolution of 16 km Out to 87 h twice a day (09 and 21 UTC initialization) Uses both initial condition uncertainty (breeding) and physics uncertainty. Uses the NMM, NMM-B, and WRF-ARW models (21 total members) US/web_js/html/mean_surface_prs.htmlhttp:// US/web_js/html/mean_surface_prs.html

ef/fplumes/

Lessons of the NE Snowstorm 5/01/forecast-lessons-from- northeast.html 5/01/forecast-lessons-from- northeast.html

SREF

NARRE (N. American Rapid Refresh Ensemble)

British Met Office MOGREPS 24 members, 18 km

Ensemble Post-Processing Ensemble output can be post-processed to get better probabilistic predictions Can weight better ensemble members more. Correct biases Improve the width of probabilistic distributions (pdfs)

BMA (Bayesian Model Averaging) is One Example

There is a whole theory on using probabilistic information for economic savings C= cost of protection L= loss if bad event event occurs Decision theory says you should protect if the probability of occurrence is greater than C/L

Critical Event: sfc winds > 50kt Cost (of protecting): $150K Loss (if damage ): $1M Hit False Alarm Miss Correct Rejection YES NO YES NO Forecast? Observed? Decision Theory Example $150K$1000K $150K$0K Optimal Threshold = 15%

The Most Difficult Part: Communication of Uncertainty

Deterministic Nature? People seem to prefer deterministic products: “tell me exactly what is going to happen” People complain they find probabilistic information confusing. Many don’t understand POP. Media and internet not moving forward very quickly on this.

Commercial sector is no better

A great deal of research and development is required to develop effective approaches for communicating probabilistic forecasts which will not overwhelm people and allow them to get value out of them.