Verification of Probabilistic Forecast J.P. Céron – Direction de la Climatologie S. Mason - IRI.

Slides:

Advertisements

Similar presentations

Quantitative Methods Topic 5 Probability Distributions

Advertisements

Extra Credit Challenge III Chapter 18 How it works Your group is your own table If you participate and avoid disqualification, you automatically get.

Chapter 7 EM Math Probability.

Fill in missing numbers or operations

© Crown copyright Met Office WAFC CAT verification Objective verification of GRIB CAT forecasts Dr Philip G Gill, WAFC Science Meeting, Washington, 20.

Norwegian Meteorological Institute met.no LAMEPS – Limited area ensemble forecasting in Norway, using targeted EPS Marit Helene Jensen, Inger-Lise Frogner,

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Extended range forecasts at MeteoSwiss: User experience.

Page 1© Crown copyright 2004 Presentation to ECMWF Forecast Product User Meeting 16th June 2005.

ECMWF Slide 1Met Op training course – Reading, March 2004 Forecast verification: probabilistic aspects Anna Ghelli, ECMWF.

Slide 1ECMWF forecast User Meeting -- Reading, June 2006 Verification of weather parameters Anna Ghelli, ECMWF.

Slide 1ECMWF forecast products users meeting – Reading, June 2005 Verification of weather parameters Anna Ghelli, ECMWF.

ECMWF User Meeting / 1 Pertti Nurmi Juha Kilpinen Annakaisa Sarkanen ( Finnish Meteorological Institute ) Probabilistic Forecasts.

Ensemble Verification I

Year 6 mental test 15 second questions Calculation Addition.

© 2002 Prentice-Hall, Inc.Chap 17-1 Basic Business Statistics (8 th Edition) Chapter 17 Decision Making.

Chi Square Interpretation. Examples of Presentations The following are examples of presentations of chi-square tables and their interpretations. These.

Measures of disease frequency (II). Calculation of incidence Strategy #2 ANALYSIS BASED ON PERSON-TIME CALCULATION OF PERSON-TIME AND INCIDENCE RATES.

Sl No Top-up Amount No Of Affiliate Ads Payment Per Day By Affiliate Ad Total Affiliate Ad Income 1.5,000/- Daily 2 ad for 100 days 100/- Affiliate.

The basics for simulations

Operations Management For Competitive Advantage © The McGraw-Hill Companies, Inc., 2001 C HASE A QUILANO J ACOBS ninth edition 1 Strategic Capacity Management.

1 00 Month 0000 Are we ready for small screen surfing? Marek Sláčík Chief Marketing Officer Telenor Serbia.

Meteorological Training Course, 20 March /25 Using Combined Prediction Systems (CPS) for wind energy applications European Centre for Medium-Range.

Happy Wednesday! April 18, What is a probability distribution? -A table or an equation that links each outcome of a statistical experiment with.

Area under curves Consider the curve y = f(x) for x  [a, b] The actual area under the curve is units 2 The approximate area is the sum of areas.

Lets play bingo!!. Calculate: MEAN Calculate: MEDIAN

Panel II Efficient Court Administration SURVEY Commercial Enforcement and Insolvency Systems Legal Vice Presidency The World Bank.

Model and Relationships 6 M 1 M M M M M M M M M M M M M M M M

Subtraction: Adding UP

Number bonds to 10,

Splinter Session – ‘Space Weather Metrics, Verification & Validation.’ Thursday 20 th Nov., 16: :00 Splinter session - Space weather metrics, verification.

Signal Detection Theory. The classical psychophysicists believed in fixed thresholds Ideally, one would obtain a step-like change from no detection to.

Caio A. S. Coelho Centro de Previsão de Tempo e Estudos Climáticos (CPTEC) Instituto Nacional de Pesquisas Espaciais (INPE) 1 st EUROBRISA.

14 May 2001QPF Verification Workshop Verification of Probability Forecasts at Points WMO QPF Verification Workshop Prague, Czech Republic May 2001.

Verification of probability and ensemble forecasts

Seasonal Predictability in East Asian Region Targeted Training Activity: Seasonal Predictability in Tropical Regions: Research and Applications 『 East.

Introduction to Probability and Probabilistic Forecasting L i n k i n g S c i e n c e t o S o c i e t y Simon Mason International Research Institute for.

1 Intercomparison of low visibility prediction methods COST-722 (WG-i) Frédéric Atger & Thierry Bergot (Météo-France)

El Nino and La Nina An important atmospheric variation that has an average period of three to seven years. Goes between El Nino, Neutral, and La Nina (ENSO.

Probabilistic forecasts of (severe) thunderstorms for the purpose of issuing a weather alarm Maurice Schmeits, Kees Kok, Daan Vogelezang and Rudolf van.

Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using forecasts and observations at all 205 UK civil and defence aerodromes.

Tutorial. Other post-processing approaches … 1) Bayesian Model Averaging (BMA) – Raftery et al (1997) 2) Analogue approaches – Hopson and Webster, J.

Assessing Predictability of Seasonal Precipitation for May-June-July in Kazakhstan Tony Barnston, IRI, New York, US.

Building Community Resilience in the Face of Increased Vulnerability to Extreme Weather & Water Events Andy Bryant National Weather Service Portland, OR.

A Preliminary Verification of the National Hurricane Center’s Tropical Cyclone Wind Probability Forecast Product Jackie Shafer Scitor Corporation Florida.

Measuring forecast skill: is it real skill or is it the varying climatology? Tom Hamill NOAA Earth System Research Lab, Boulder, Colorado

Heidke Skill Score (for deterministic categorical forecasts) Heidke score = Example: Suppose for OND 1997, rainfall forecasts are made for 15 stations.

Model validation Simon Mason Seasonal Forecasting Using the Climate Predictability Tool Bangkok, Thailand, 12 – 16 January 2015.

Latest results in verification over Poland Katarzyna Starosta, Joanna Linkowska Institute of Meteorology and Water Management, Warsaw 9th COSMO General.

Verification of IRI Forecasts Tony Barnston and Shuhua Li.

61 st IHC, New Orleans, LA Verification of the Monte Carlo Tropical Cyclone Wind Speed Probabilities: A Joint Hurricane Testbed Project Update John A.

© 2005 Accurate Environmental Forecasting Climate and Hurricane Risk Dr. Dail Rowe Accurate Environmental Forecasting

Can we distinguish wet years from dry years? Simon Mason Seasonal Forecasting Using the Climate Predictability Tool Bangkok, Thailand,

Retrospective Evaluation of the Performance of Experimental Long-Lead Columbia River Streamflow Forecasts Climate Forecast and Estimated Initial Soil Moisture.

© Crown copyright Met Office Standard Verification System for Long-range Forecasts (SVSLRF) Richard Graham, Met Office Hadley Centre. With acknowledgements.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Verification and Metrics (CAWCR)

Verification of ensemble systems Chiara Marsigli ARPA-SIMC.

SASCOF 2010 Météo-France GCM forecasts JP. Céron – Météo-France

El Nino and La Nina An important atmospheric variation that has an average period of three to seven years. Goes between El Nino, Neutral, and La Nina (ENSO.

Seasonal prediction of South Asian summer monsoon 2010: Met Office

Probabilistic forecasts

Validation-Based Decision Making

Quantitative verification of cloud fraction forecasts

COSMO-LEPS Verification

IRI forecast April 2010 SASCOF-1

Can we distinguish wet years from dry years?

Verification of SPE Probability Forecasts at SEPC

Seasonal Forecasting Using the Climate Predictability Tool

Short Range Ensemble Prediction System Verification over Greece

Presentation transcript:

Verification of Probabilistic Forecast J.P. Céron – Direction de la Climatologie S. Mason - IRI

The Exercise identify the forecasts of El Niño, Neutral, and La Niña years L E E E E E E L

The Exercise identify the forecasts of El Niño, Neutral, and La Niña years

The Exercise calculate forecast probabilities for El Niño, neutral, and La Niña events 0 % 80 % 20 % 100 % 0 % 0 % 20 % 60 % 20 %

The Exercise calculate forecast probabilities for El Niño, neutral, and La Niña events

The Exercise calculate forecast biases 20 % - 33 % The frequency of La Niña forecasts is 18 / (18*5) = 0,20 The observed frequency of La Niña events is 6 / 18 = 0,33

The Exercise calculate forecast biases

The Exercise calculate the Hit Rate and False Alarme Rate 0,17 0,00 0,50 0,00 0,83 0,00 0,83 0,25 1,00

The Exercise calculate the Hit Rate and False Alarme Rate

The Exercise ROC diagram

The Exercise ROC diagram

The Exercise Brier skill scores Differences between the ROC areas and Brier skill scores The ROC areas indicate more optimistic estimates of forecast skill, most notably for the neutral conditions. The Brier skill score indicates negative skill for forecasts of neutral conditions. The Brier skill scores for this category are weak because of the large forecast bias, but the ROC ignores the bias.