Ensemble Experiments based on the convection-allowing model COSMO-DE

Ensemble Experiments based on the convection-allowing model COSMO-DE
Deutscher Wetterdienst Ensemble Experiments based on the convection-allowing model COSMO-DE Susanne Theis Project COSMO-DE-EPS

Deutscher Wetterdienst
Overview Motivation for Project COSMO-DE-EPS Tasks within Project Current Status and Experiments

Motivation for the Project
Deutscher Wetterdienst Motivation for the Project

Numerical Weather Prediction Model COSMO-DE grid box size: 2,8 km without parametrization of deep convection assimilation of radar data lead time: 0-21 hours operational since April 2007 COSMO-DE COSMO-EU GME

Benefits of COSMO-DE COSMO-EU (7 km) COSMO-DE (2,8 km) captures extremes mm/h

Benefits of COSMO-DE COSMO-EU (7 km) COSMO-DE (2,8 km) looks more realistic mm/h

What about Deterministic Predictability? Scale Diagram Predictability synoptic scale 1 week typical time scale convective scale 1 hour Man koennte sagen: Kein Thema fuer den DWD, denn wir schauen nicht auf solch lange Vorhersagezeitraeume! Aber Vorsicht: Vorhersagbarkeit ist skalenabhaengig! (Skalendiagramm beschreiben) (Fehlerwachstum) Vorhersagbarkeit lang fuer grosse Skalen, kurz fuer kleine Skalen Fazit: Auch bei Festhalten der Vorhersagedauer, aber gleichzeitigem Verfeinern der betrachteten Skala  Grenzen determ. Vorhersagbarkeit Mit der hochauflösenden Modellierung laufen wir auch bei 18 Stunden Vorhersagen in Bereiche, wo Vorhersagbarkeit ein Thema ist. 100 m 10 km 1000 km typical spatial scale lead time

COSMO-DE (2,8 km) need for probabilistic approach  so that user really gets benefit mm/h

Project COSMO-DE-EPS: Development of a convection-allowing Ensemble Prediction System Susanne Theis, Christoph Gebhardt, Michael Buchhold, Marcus Paulat, Roland Ohl, Zied Ben Bouallègue, Carlos Peralta

Project COSMO-DE-EPS: Development of a convection-allowing Ensemble Prediction System Aims & Tasks

Setting up an Ensemble System several predictions = „members“ of ensemble  very large investment in computing power „variations“ in prediction system ensemble products: probability density fct - exceedance probs quantiles - „spread“ mean ... Benefit for user: uncertainty range, risk Need to cover the whole chain! (cannot specialize, cannot reduce the complexity) Supercomputer is our most important partner

Aim of the Project How many ensemble members? 20 members, pre-operational 40 members, operational When? 2010 pre-operational 2011 operational

Aim of the Project How many ensemble members? 20 members, pre-operational 40 members, operational When? 2010 pre-operational 2011 operational conditional on supercomputer

Tasks within Project implementation of perturbations verification & diagnostics post-processing visualization early user feedback

Implementation of Perturbations
Deutscher Wetterdienst Implementation of Perturbations

Implementation of Perturbations X X initial conditions boundary conditions model physics

Implementation of Perturbations X X initial conditions boundary conditions model physics start with a simple approach

Perturbation of Boundary Conditions

Perturbation of Boundary Conditions Reminder: operational chain COSMO-DE COSMO-EU GME

Perturbation of Boundary Conditions Part of AEMet Ensemble (AEMet, Spain) Part of COSMO-SREPS (ARPA-SIMC, Bologna) COSMO-DE-EPS (DWD, Germany) (García-Moya et al., 2007) (Marsigli et al., 2008) (Gebhardt et al., 2009)

Perturbation of Boundary Conditions Part of AEMet Ensemble (AEMet, Spain) Part of COSMO-SREPS (ARPA-SIMC, Bologna) COSMO-DE-EPS (DWD, Germany) GME IFS UM GFS COSMO-DE (2,8km) COSMO COSMO COSMO COSMO

Perturbation of Boundary Conditions Part of AEMet Ensemble (AEMet, Spain) Part of COSMO-SREPS (ARPA-SIMC, Bologna) COSMO-DE-EPS (DWD, Germany) GME IFS UM GFS COSMO-DE (2,8km) COSMO COSMO COSMO COSMO long-term Plan: take boundaries from ICON Ensemble

Perturbation of Initial Conditions

Perturbation of Initial Conditions current work: - perturb “nudging” at forecast start - use differences between control and COSMO-SREPS plans: Ensemble Transform Kalman Filter (COSMO project KENDA)

Perturbation of the Model

Perturbation of the Model Alter parameters in physics parametrizations entr_sc=  entrain_sc=0.002 rlam_heat=1.  rlam_heat=10. rlam_heat=1.  rlam_heat=0.1 …q_crit… …tur_len… 1 2 3 4 5 5 4 3 2 1 requires careful tuning  affect forecast, but not long-term quality

Current Experiments experimental set-up - examples

Current Status X initial conditions boundary conditions model physics

Current Experiments 1 2 3 4 5 GME IFS UM NCEP no perturbation of initial conditions

Current Experiments 1 2 3 4 5 IFS GME GFS UM no perturbation of initial conditions

2 July 2007, 00 UTC + 24h 24h accumulations of precipitation [mm] mm/24h

Current Experiments 1 2 3 4 5 IFS GME GFS UM no perturbation of initial conditions

2 July 2007, 00 UTC + 24h 24h accumulations of precipitation [mm] mm/24h

- how do perturbations affect the forecast?
Deutscher Wetterdienst Ensemble Diagnostics - how do perturbations affect the forecast?

Effect of perturbations on precipitation boundaries: dominate after a few hours (not necessarily, also case-dependent) physics: dominate during first few hours (Gebhardt et al., 2009)

Ensemble Dispersion (precipitation) Normalized Variance Difference = variance (BC only) - variance (PHY only) normalized by sum of variances (Gebhardt et al., 2009)

Ensemble Dispersion (precipitation) Normalized Variance Difference = variance (BC only) - variance (PHY only) normalized by sum of variances for individual days (Gebhardt et al., 2009)

Ensemble Verification
Deutscher Wetterdienst Ensemble Verification - how good is the Ensemble (at this stage)?

Verification Data Sets RADAR

Verification Data Sets RADAR Longest Period: 1 July – 16 Sep 2008

Probabilistic Verification Measures Traditional: - Brier Score, Brier Skill Score ROC curve Reliability Diagram Talagrand Diagram Spread-Skill Relation with focus on precipitation, also scale-dependent

Verification Results

Verification Results current setup X without initial condition perturbations postprocessing ! boundary conditions model physics

Verification Results ensemble is superior to single simulation ensemble is underdispersive current setup X without initial condition perturbations postprocessing ! boundary conditions model physics observation

As a Reminder: Tasks within Project implementation of perturbations verification & diagnostics post-processing visualization early user feedback

Early User Feedback - work towards acceptance - work towards useful interpretation

Early User Feedback - Forecasters (DWD)
Deutscher Wetterdienst Early User Feedback - Forecasters (DWD)

EPS Product Example: Probability Maps „Event somewhere in 2,8km-Box“ %

EPS Product Example: Probability Maps „Event somewhere in 2,8km-Box“ Forecasters: Probabilities are too low! %

EPS Product Example: Probability Maps „Event somewhere in 2,8km-Box“ Reason: Forecasters are used to larger reference regions  Experiment: present probabilities on coarser grid Forecasters: Probabilities are too low! %

EPS Product Example: Probability Maps „Event somewhere in 2,8km-Box“ „Event somewhere in 28km-Box“ %

EPS Product Example: Probability Maps „Event somewhere in 2,8km-Box“ „Event somewhere in 28km-Box“ Relevance of reference region! %

Literature about perception of ensemble forecasts: Demeritt, D., Cloke, H., Pappenberger, F., Thielen, J., Bartholmes, J. and M.-H. Ramos (2007): Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting. Environmental Hazards 7, Gigerenzer, G., Hertwig, R., van den Broek, E., Fasolo, B. and K.V.Katsikopoulos (2005): „A 30% Chance of Rain Tomorrow“: How does the public understand probabilistic weather forecasts? Risk Analysis 25(3), Joslyn, S., Pak, K., Jones, D., Pyles, J. and E. Hunt (2007): The effect of probabilistic information on threshold forecasts. Weather and Forecasting 22, Morss, R.E., Demuth, J.L. and J.K. Lazo (2008): Communicating Uncertainty in Weather Forecasts: A Survey of the U.S. Public. Weather and Forecasting 23, Morss, R.E., Wilhelmi, V.W., Downton, M.W. and E. Gruntfest (2005): Flood risk, uncertainty, and scientific information for decision making: Lessons from an interdisciplinary project. Bulletin of the American Meteorological Society 86(11), National Research Council (2006): Completing the Forecast: Characterizing and Communicating Uncertaintiy for Better Decisions Using Weather and Climate Forecasts. National Academies Press, 124 pp. Nobert, S., Demeritt, D. and H. Cloke (2009): Using Ensemble Predictions for Operational Flood Forecasting: Lessons from Sweden. Submitted to Journal of Flood Risk Management. Roulston, M.S., Bolton, G.E., Kleit, A.N. and A.L. Sears-Collins (2006): A laboratory study of the benefits of including uncertainty information in weather forecasts. Weather and Forecasting 21,

Back to the Project COSMO-DE-EPS: Summary and Challenges
Deutscher Wetterdienst Back to the Project COSMO-DE-EPS: Summary and Challenges

Summary of Project COSMO-DE-EPS setting up a convection-permitting ensemble with prospect of becoming operational dealing with all parts of forecast chain (perturbations, products, postprocessing, verification, users) simple approach first, then keep improving

Challenges of Project COSMO-DE-EPS scientific representation of uncertainties e.g. getting appropriate spread technical implementation e.g. amount of data communication to the user e.g. appropriate spatial scale

Thank You For Your Attention
Deutscher Wetterdienst Thank You For Your Attention

Extra Slides

More Diagnostics & Verification
Deutscher Wetterdienst More Diagnostics & Verification

Ensemble Dispersion (precipitation) Spread Ratio #(GPall members) = #(GPat least 1 member) Event: > 0.1 mm/h physics only boundary conditions only (Gebhardt et al., 2009)

Talagrand Diagram 24h precipitation observation Rank

Example of Verification

Example of Verification Frequencies of precipitation amount for ensemble forecasts (members) for observations

Example of Verification Frequencies of precipitation amount for ensemble forecasts (members) for observations Conditional on Categories of Ensemble Mean Forecast: Ens.Mean: no & low precip Ens.Mean: slight precip Ens.Mean: medium & heavy precip

Example of Verification Frequencies of precipitation amount for ensemble forecasts (members) for observations Conditional on Categories of Ensemble Mean Forecast: Ens.Mean: no & low precip Ens.Mean: slight precip Ens.Mean: medium & heavy precip Lead time: 0 - 6 hours

Example of Verification Frequencies of precipitation amount for ensemble forecasts (members) for observations Conditional on Categories of Ensemble Mean Forecast: Ens.Mean: no & low precip Ens.Mean: slight precip Ens.Mean: medium & heavy precip Lead time: hours

Example of Verification Frequencies of precipitation amount for ensemble forecasts (members) for observations Conditional on Categories of Ensemble Mean Forecast: Ens.Mean: no & low precip Ens.Mean: slight precip Ens.Mean: medium & heavy precip add IC perturbations and statistical PP Lead time: hours

ROC Curve Reliability Diagram ROC area: 0.90 Threshold: 0.1mm/24h

ROC Curve Reliability Diagram ROC area: 0.85 Threshold: 10mm/24h

Verification results for 2m-temperature 12UTC threshold: 25°C ROC Reliability area=0.94

Brier Skill Score (BSS) 0.4 0.2 0.0 24 hour precipitation Reference Forecast: Climatology from Period Threshold [mm/24h]

Verification results: single member quality check Frequency Bias Index 24 hours accumulated precipitation red line shows the deterministic COSMO-DE all members are within a reasonable range

Verification results: Brier Skill Score 24 hours accumulated precipitation reference: deterministic COSMO-DE 6 hours accumulated precipitation

Statistical Postprocessing
Deutscher Wetterdienst Statistical Postprocessing

Statistical Postprocessing several predictions = „members“ of ensemble „variations“ in prediction system ensemble products: probability density fct - exceedance probs quantiles - „spread“ mean ... Benefit for user: uncertainty range, risk Need to cover the whole chain! (cannot specialize, cannot reduce the complexity) Supercomputer is our most important partner Aim: improve quality Focus: precipitation

Statistical Postprocessing Methods: Logistic Regression + spatio-temporal neighbourhood + lagged average ensemble 2. Bayesian Approach cooperation with University of Bonn  calibration of probabilities enhance sample  entire pdf

Reaching the User

Reaching the user (= “serve mankind”) make ensemble forecast accessible choose a good format and reduce information work towards acceptance work towards correct (useful) interpretation work towards integration into decision making increase trust in forecast provider

Visualization in NinJo
Deutscher Wetterdienst Visualization in NinJo make forecasts accessible - choose formats

Visualization in NinJo NinJo = visualization tool for forecasters Challenge: amount of data good communication of complex matters must fit into an existing system (NinJo) Visualization is part of project: end-to-end approach

EPS Product Example: Exceedance Probabilities in % Probability of RR > 1 mm/h

EPS Product Example: 1h-sums of precipitation [mm] 2m-temperature [°C] 10% probability (= risk) Quantiles range of 80% probability (= uncertainty)

Further Plans: Ensemble Navigation Window (draft) also possible to look at individual members

Ensemble Experiments based on the convection-allowing model COSMO-DE

Similar presentations

Presentation on theme: "Ensemble Experiments based on the convection-allowing model COSMO-DE"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Ensemble Experiments based on the convection-allowing model COSMO-DE

Similar presentations

Presentation on theme: "Ensemble Experiments based on the convection-allowing model COSMO-DE"— Presentation transcript:

Similar presentations

About project

Feedback