1. Development of a LAM ensemble for the 2014 Winter Olympic Games Andrea Montani, C. Marsigli, T. Paccagnella ARPA Emilia-Romagna Servizio IdroMeteoClima, Bologna (I) Servizio IdroMeteoClima COSMO meeting Lugano, 10-13 September 2012

2. A.Montani; The COSMO-LEPS system. Outline Introduction to COSMO-LEPS system:  methodology and implementation. Limited-area ensemble activity for FROST-2014:  relocation of COSMO-LEPS: COSMO-FRost-EPS. Performance of COSMO-FROST-EPS:  time-series verification using SYNOP reports and comparison against ECMWF;  case-study assessment; Conclusions and open issues.

3. A.Montani; The COSMO-LEPS system. COSMO-LEPS (developed at ARPA-SIMC) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale Modelling, including Germany, Greece, Italy, Poland, Romania, Russia, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of high- impact and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)  generation of COSMO-LEPS to improve the forecast of high-impact weather in the short and early-medium range.

4. Dim 2 Initial conditions Dim 1 Dim 2 Possible evolution scenarios Dim 1 Initial conditions ensemble size reduction Cluster members chosen as representative members (RMs) LAM integrations driven by RMs LAM scenario COSMO-LEPS methodology

5. A.Montani; The COSMO-LEPS system. Outline Limited-area ensemble activity for FROST-2014:  relocation of COSMO-LEPS: COSMO-FRost-EPS. As for Sochi-2014 Winter Olympics, Roshydromet is organizing a blended RDP/FDP (Research and Development Project / Forecast Demonstration Project) under the auspices of WMO: FROST-2014 (Forecasting and Research: the Olympic Sochi Testbed; http://frost2014.meteoinfo.ru).

6. A.Montani; The COSMO-LEPS system. COSMO activities for FROST-2014 COSMO is performing a number of activities related to FROST-2014 WWRP RDP/FDP, coordinated in the framework of the Priority Project CORSO: a)deterministic forecasting, b)probabilistic forecasting: b1) FDP initiatives (COSMO-FROST-EPS: relocation of COSMO-LEPS over the Sochi area), b2) RDP initiatives (development of a convective-scale ensemble system for the Sochi area), c)post-processing and product generation, d)verification (development of VERSUS software for probabilistic verification). Relocation of COSMO-LEPS system over the Sochi area so as to provide: probabilistic forecasting at high resolution for the Olympic competitions, support to the deterministic forecasting, initial and boundary conditions for the development of a convective-resolving EPS.

7. A.Montani; The COSMO-LEPS system. Main milestones of COSMO-FROST-EPS 11/3/2011. Nothing present. 2/5/2011. Submission of a new ECMWF Special Project (Title: “Implementation of a limited-area ensemble prediction system for Sochi Olympic Games”; Project investigators: Majewski, Montani, Steiner; duration: 3 years) for provision of computer time to run the system on ECMWF super-computers; 5-9/9/2011. Discussion during the COSMO meeting about the system set-up; 6/12/2011. Approval of the Special Project by ECMWF Council; 6-9/9/2011. Visit of Russian colleagues at ARPA-SIMC to define specifics of the new ensemble system; 19/12/2011. Beginning of provision of COSMO-FROST-EPS products on a daily basis.

8. A.Montani; The COSMO-LEPS system. COSMO-FROST-EPS @ ECMWF: present status d+3dd+2d+1 ECMWF EPS clustering interval Cluster Analysis and RM identification 4 variables Z U V Q 3 levels 500 700 850 hPa 2 time steps Cluster Analysis and RM identification Black-Sea area Complete Linkage 10 Representative Members driving the 10 COSMO-model integrations (weighted according to the cluster populations) employing either Tiedtke or Kain-Fristch convection scheme (randomly choosen) + perturbations in turbulence scheme and in physical parameterisations clustering area Δx ~ 7 km; 40 ML; fc+72h; initial time: 00/12 UTC; At the moment, computer time (~ 2 million BUs for 2012) is provided by an ECMWF Special Project; suite managed by ARPA-SIMC; contributions from ECMWF member states could be needed in the future. Integration Domain

9. A.Montani; The COSMO-LEPS system. Disseminated products All post-processing is done using with COSMO-software fieldextra:  probability fields for the exceedance of thresholds for surface fields;  ensemble mean and ensemble standard deviation for some fields;  individual ensemble members for the generation of point-forecasts;  hourly boundary conditions (from fc+0h to fc+48h) for convective- resolving ensemble (RDP part).  10 perturbed COSMO-model runs (ICs and BCs from 10 selected EPS members): start at 00UTC and 12UTC;  t = 72h;  1 deterministic run (ICs and BCs from the deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 00UTC and 12UTC;  t = 72h;

10. A.Montani; The COSMO-LEPS system. Outline Performance of COSMO-FROST-EPS:  verification using SYNOP reports and comparison against ECMWF-EPS;  case-study assessment;

11. A.Montani; The COSMO-LEPS system. COSMO-FROST-EPS vs ECMWF-EPS Main features of verification: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Jan 2012 to Mar 2012; region: 40-50N, 35-45 (SOCHIDOM); method: nearest grid point; no-weighted fcst; obs: synop reports (about 60 stations/day); fcst ranges: 6-18h, 18-30h, 30-42h, 42-54h, 54-66h; thresholds: 1, 5, 10, 15, 25, 50 mm/12h; scores: ROC area, BSS, RPSS, Outliers, … systems: - COSMO-FROST-EPS(10m, 7 km, 40 ML) - ECMWF-EPS (51m, 25 km, 62 ML) “Large-scale” verification tends to smooth out differences between higher and lower resolution systems  ECMWF-EPS should be favoured.

12. A.Montani; The COSMO-LEPS system. Impact of higher resolution -40500010300020001000500100 ECMWF-EPS orography (25 km)COSMO-FROST-EPS orography (7 km) 7 km still not enough to resolve some scale-scale features  need of statistical post-processing to downscale the information

13. A.Montani; The COSMO-LEPS system. ROC area  Area under the curve in the HIT rate vs FAR diagram; the higher, the better …  Valuable forecast systems have ROC area values > 0.6.  Consider the event: 12-hour precipitation exceeding 10 mm  COSMO-FROST-EPS outperforms ECMWF-EPS for all forecast ranges.  12-hour cycle of the score for both systems, which better predict precipitation occurring during daytime (6-18Z).  ROC area values show little dependence on the threshold (not shown). These results need to be confirmed over higher-resolution observational networks.

14. A.Montani; The COSMO-LEPS system. Brier Skill Score  BSS is written as 1-BS/BS ref. Sample climate is the reference system. Useful forecast systems if BSS > 0.  BS is equivalent to MSE for deterministic forecast.  BS measures the mean squared difference between forecast and observation in probability space.  Lower threshold: COSMO-FROST-EPS wins for all forecast ranges.  Higher thresholds: similar performance of the two systems, but fewer obs are available. Performance of the systems shown for 2 events: total precipitation exceeding 1 mm in 12 hours (dashed lines), total precipitation exceeding 5 mm in 12 hours (solid lines).

15. A.Montani; The COSMO-LEPS system. Ranked Probability Skill Score  BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.  Useful forecast systems for RPSS > 0.  RPSS depends on the ensemble size and penalises small ensemble sizes.  Consider debiased RPSS: RPSS D = 1 –(RPS/(RPS ref + RPS ref /N))  In either case (RPSS or RPSS D ) COSMO-FROST-EPS has higher scores than ECMWF-EPS for all forecast ranges, despite the lower ensemble size. RPSS RPSS D

16. A.Montani; The COSMO-LEPS system. Outliers  How many times the analysis is out of the forecast interval spanned by the ensemble members.  … the lower the better …  In a “well-constructed” ensemble system of N members, the percentage of outliers should asymptotically approach its theoretical values (= 2/N+1).  COSMO-FROST-EPS has fewer outliers than ECMWF-EPS, especially for shorter forecast ranges.  COSMO-FROST-EPS approaches quickly the theoretical limit; ECMWF-EPS is away from its limit.

17. A.Montani; The COSMO-LEPS system. Case-study assessment 1.Heavy precipitation event: snowfall started on 31 January, at 00UTC in the mountains and at 3UTC on the coast. Snowfall lasted 2 days (31 Jan and 1 Feb). In the first day, 18 cm of snow in Sochi, 5 cm in Adler. 2.Foehn event: started on 3 February in the evening (local time) and lasted until 6 February. Sochi local time = GMT + 4 hours GMT = Sochi local time – 4 hours

18. A.Montani; The COSMO-LEPS system. Probability maps of tp > 20 mm eq water / 24h All maps verify on 1 Feb 2012, 12UTC (4pm local time) IC: 31/1, 00UTC +12-36h IC: 30/1, 00UTC +36-60h IC: 30/1, 12UTC +24-48h

19. A.Montani; The COSMO-LEPS system. The new COSMO-based ensemble system over the Sochi-area (COSMO-FROST-EPS) was developed, implemented and runs on a daily basis. Preliminary verification results indicate the added value of the system with respect to ECMWF-EPS; positive results are confirmed also for case-study analysis. 7 km not enough to resolve some small-scale features: –statistical post-processing to downscale the information  FDP? –dynamical downscaling to generate convective-resolving EPS  RDP? Conclusions and open issues COSMO-FROST-EPS is running throughout 2012 (and not only for 6 months), thanks to the Swiss contribution. Improve dissemination: –develop new products “on demand” (in the future “multi-model” products?); –produce meteograms and/or xml output files, –Concentrate on issues related to data format / grid /... Improve probabilistic verification: –COSMO-software VERSUS is being developed to address probabilistic verification, –need of good-quality observations at high resolution.

20. A.Montani; The COSMO-LEPS system. Important ingredients 1.Provide reasonable “numbers”.  addressed 2.Develop experience with probabilities. 3.Feedback on the top-priority products.  being addressed 4.Snow analysis. 5.Soil-field initialisation. 6.Observations to assess the quality of the system..  being addressed 7.Computer time.  addressed 8.Timeliness in product delivery.  addressed 9.… anything to add/remove?

21. A.Montani; The COSMO-LEPS system. Thank you !