1. European Storm Forecast Experiment Verification of Dichotomous Lightning Forecasts at the European Storm Forecast Experiment (ESTOFEX) Pieter Groenemeijer Oscar van der Velde Helge Tuschy Christoph Gatzen Johannes Dahl Nicholas Verge Institute for Meteorology and Climate Research, Research Centre / University Karlsruhe European Severe Storms Laboratory Laboratoire d'Aérologie, Université Paul Sabatier, Toulouse Institute for Atmospheric Physics, German Aerospace Center (DLR)

2. European Storm Forecast Experiment Overview About ESTOFEX ESTOFEX forecasts Verification of lightning forecasts Verification of threat level forecasts

3. European Storm Forecast Experiment about ESTOFEX the name of a web-based platform for the exchange of knowledge about convective storm forecasting the name of a the forecast experiment that is carried out by some of its active members goal of the experiment is... “to find out how convective storms can best be forecast in Europe”

4. European Storm Forecast Experiment about ESTOFEX ESTOFEX issues “convective forecasts” for the next day These forecasts consist of two parts: forecast discussion forecast map Storm Forecast Valid: Sat 02 Jun 2007 06:00 to Sun 03 Jun 2007 06:00 UTC Issued: Sat 02 Jun 2007 09:22 Forecaster: VAN DER VELDE SYNOPSIS An mid/upper cut-off low dwells slowly southward with its center just west of Italy. The associated surface pressure field weakens during the period. High pressure...

5. European Storm Forecast Experiment ESTOFEX is not... a new weather service in Europe (no such centre without an initiative from NMS’s) funded the start of a commercial company

6. European Storm Forecast Experiment thunder area level 1 level 2 level 3 ESTOFEX forecasts The forecast maps show lines that represent level 1, level 2, level 3 are threat levels for convective wind gusts, tornadoes and large hail

7. European Storm Forecast Experiment ESTOFEX forecasts After the forecast has been issued, verification data appears on the map. Hail reports Tornado reports Wind gust reports Heavy rain reports (source: ESSL) Lightning strikes source: UK. Met Office arrival-time-difference system

8. European Storm Forecast Experiment Verification of lightning forecasts Predictand the occurrence of lightning* within 40 km of a point Dichotomous forecast event occurred? yesno event forecast? yeshitfalse alarm nomiss correct negative * = one or more strike detected

9. European Storm Forecast Experiment Forecast lightning (1 Sep. 2006 – 31 Aug. 2007) number of forecast periods (of 346 forecasts) in which lightning was forecast to occur within 40 km of each point

10. European Storm Forecast Experiment Detected lightning (1 Sep. 2006 – 31 Aug. 2007) number of forecast periods (of 346 forecasts) in which lightning occurred within 40 km of a point More lightning is forecast than occurs in reality. The forecasts have a positive bias.

11. European Storm Forecast Experiment Forecast skill indicators False Alarm Rate = false alarms / (hits + false alarms) Probability Of Detection = hits / (hits + misses) Heidke Skill Score

12. European Storm Forecast Experiment Probability Of Detection (1 Sep. 2006 – 31 Aug. 2007) High POD over... the western and central European continent the central and eastern Mediterranean Sea Low POD over... the Atlantic Ocean Scandinavia Eastern Europe Northern Africa POD = hits / (hits + misses)

13. European Storm Forecast Experiment False Alarm Rate (1 Sep. 2006 – 31 Aug. 2007) Low FAR over... central and eastern parts of the European continent High FAR over... western and northern Europe parts of the Mediterranean the Black Sea FAR = false / (hits + false)

14. European Storm Forecast Experiment Heidke Skill Score (1 Sep. 2006 – 31 Aug. 2007) HSS = 0 means that the forecast is no better than chance HSS = 1 means that the forecast is perfect HSS assumes that a miss is just as bad as a false alarm. HSS indicates the forecasts are best over the central and eastern European continent and the southern Mediterranean Forecasts are bad over western and northern Europe

15. European Storm Forecast Experiment Temporal evolution of forecast skill

16. European Storm Forecast Experiment Temporal changes in performance

17. European Storm Forecast Experiment Individual forecaster performance 2006 and 2007 summer periods compared Probability of DetectionFalse Alarm Ratio

18. European Storm Forecast Experiment Individual forecaster performance 2006 and 2007 summer periods compared Probability of DetectionFalse Alarm Ratio

19. European Storm Forecast Experiment Future plans event occurred ? YesNo event forecast? > 60 % 30-60% 15-30% 5 -15% < 5% Polychotomous lightning forecasts

20. European Storm Forecast Experiment Future plans switch to probabilistic forecasting of lightning start rigorous verification of severe weather (hail, tornadoes, severe winds) switch to probabilistic forecasting of severe weather test the skill of model-derived parameters for both lightning and hail, tornadoes winds – and compare with human forecasters (interesting!)

21. European Storm Forecast Experiment Final remarks a discussion of individual storm events is presented on the poster lead-authored by Helge Tuschy and in Christoph Gatzen’s talk all verification data is available through the forecast archive at www.estofex.orgwww.estofex.org I am happy to answer your questions on ESTOFEX and the verification of its forecasts