1. Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training Workshop on Severe Weather Forecasting Bujumbura, Burundi, Nov 11-16, 2013 Slides courtesy: UKMO Paul Davies and Chris Tubbs 2014-10-101

2. 2 Computer models Forecaster Customer/Public Atmosphere Scientists The forecaster misled me The NWP misled me Errorneous observations misled the NWP The atmosphere is ”chaotic” ”The Blame Game” or ”The Passing of The Buck”

3. 2014-10-103 Understanding chaos

4. Numerical Weather Prediction 2014-10-104 The Atmosphere Model System Data Assimilation Forecast Analysis Error Forecast distribution Diagnostic Tools and Products

5. 2014-10-105 Quantifying uncertainty with ensembles time Forecast uncertainty Climatology Initial Condition Uncertainty X Deterministic Forecast Analysis CHAOS

6. April 2013 Anders Persson6 -12 0+12+24+36+48+60+72+84 h ψ Exchanging the “accurate” forecast with a more “honest” one Dangerous threshold Today’s NWP forecast Today’s EPS forecast

7. Anders Persson7 10/10/20147 ψ Correction for systematic errors -12 0+12+24+36+48+60+72+84 h

8. Anders Persson8 10/10/20148 ψ ● ● ● ● ● ● ● ● ● ● ● ● obs ●● ● The final ensemble forecast – with verification 70%50% -12 0+12+24+36+48+60+72+84 h

9. Anders Persson9 Ψ Probability Investigations show that “jumpiness” correlates badly to the accuracy of the last forecast NWP today NWP yesterday NWP the day before yesterday We might form an opinion by looking at the last NWPs from the same model, so called “lagged” forecasts

10. Anders Persson10 10/10/2014 Medium range forecasting…with deterministic and EPS information Latest three NWP Latest EPS Most common case with good agreement between EPS spread and NWP “jumpiness” Most common case

11. Anders Persson11 10/10/2014 Rather poor agreement between larger EPS spread and small NWP “jumpiness”. The analysis system has obviously managed to avoid possible problems because the NWP is not very “jumpy” Should the forecasters be more certain than the EPS indicates? Rather common case

12. Anders Persson12 10/10/2014 Rather poor agreement between small EPS spread and large NWP “jumpiness”. The perturbations have not been quite able to cover the analysis uncertainties Should the forecasters be more uncertain than the EPS indicates? Not uncommon case

13. Anders Persson13 10/10/2014 Poor agreement between the main directions of the EPS and the NWP This puts the forecasters in a very difficult situation and there is not enough experience or investigations about this situation Rare case Best choice: create a “super ensemble” See TIGGE

14. The Effect of Chaos 2014-10-1014 Tiny errors in our analysis of the current state of the atmosphere lead to large errors in the forecast – these are both equally valid 4-day forecasts. We can usually forecast the general pattern of the weather up to about 3 days ahead. Chaos then becomes a major factor Fine details (eg rainfall) have shorter predictability

15. Ensembles In an ensemble forecast we run the model many times from slightly different initial conditions This provides a range of likely forecast solutions which allows forecasters to: – assess possible outcomes; – estimate risks – gauge confidence. 2014-10-1015

16. 2014-10-1016 Reminder on scales and predictability

17. Temporal Resolution 2014-10-1017 Hail shaft Thunderstor m Front Tropical Cyclone Space Scale Lifetime Predictability? 10mins1 hr12hrs3 days 30mins3 hrs36hrs9 days 1000km 100km 10km 1km MCS

18. Short-range Ensembles ECMWF EPS has transformed the way we do Medium-Range Forecasting Uncertainty also in short-range: – Rapid Cyclogenesis often poorly forecast deterministically – Uncertainty of sub-synoptic systems (eg thunderstorms) – Many customers most interested in short-range Assess ability to estimate uncertainty in local weather – QPF – Cloud Ceiling, Fog – Winds etc 2014-10-1018

19. Initial conditions perturbations Perturbations centred around 4D-Var analysis Transforms calculated using same set of observations as used in 4D-Var (including all satellite obs) within +/- 3 hours of data time Ensemble uses 12 hour cycle (data assimilation uses 6 hour cycle) 2014-10-1019

20. Initial conditions perturbations Differences with ECWMF Singular Vectors: -It focuses on errors growing during the assimilation period, not growing period: - Suitable for Short-range! -Calculated using the same resolution than the forecast -ETKF includes moist processes -Running in conjunction with stochastic physics to propagate effect 2014-10-1020

21. 2014-10-1021 Model error: parameterisations Random parameters ParameterSchememin/std/Max Entrainment rateCONVECTION2 / 3 / 5 Cape timescaleCONVECTION30 / 30 / 120 RH criticalLRG. S. CLOUD0.6 / 0.8 / 0.9 Cloud to rain (land)LRG. S. CLOUD1E-4/8E-4/1E-3 Cloud to rain (sea)LRG. S. CLOUD5E-5/2E-4/5E-4 Ice fallLRG. S. CLOUD17 / 25.2 / 33 Flux profile param.BOUNDARY L.5 / 10 / 20 Neutral mixing lengthBOUNDARY L.0.05 / 0.15 / 0.5 Gravity wave const.GRAVITY W.D.1E-4/7E-4/7.5E-4 Froude numberGRAVITY W.D.2 / 2 / 4  QUMP (Murphy et al., 2004)  Initial stoch. Phys. Scheme for the UM (Arribas, 2004)

22. Using probabilities Recipients of forecasts & warnings are sensitive to different levels of risk: reflecting cost of mitigation vs expected loss An intelligent response to forecasts & warnings depends on risk analysis, requiring knowledge of impact probability Use of ensembles to estimate probability at longer lead times is well established in meteorology 2014-10-1022

23. WMO SWFDP Macau 10 April 2013 Anders Persson 23 10203040 50 Ψ Median 50-50% 25%=12-13 forecasts Ψ= 31-37 25%=12-13 forecasts Ψ= 23-30 25%=12-13 forecasts Ψ= 10-22 25%=12-13 forecasts Ψ= 38-50 10203040 50 Ψ 0% 25% 50% 75% 100% | | ||| ||| || | | | ||| | | || || ||| |||| || || || | | | | | ||| | || | | | |

24. 2014-10-1024 Probability maps

25. 2014-10-1025

26. 2014-10-1026 EC Total ppn prob > 20mm 12z Tue – 12z Wed

27. 2014-10-1027 EPSgrammes

28. 2014-10-1028 Total cloud cover 6 hourly precipitation 10m wind speed 2m temperature EPS control Deterministic median (50%) max min 90% 10% 75% 25%

29. 2014-10-1029 Pretoria

30. Bujumbura 06 Nov 12UTC 2014-10-1030

31. 2014-10-1031

32. 2014-10-1032 The Extreme Forecast Index (EFI)

33. Extreme forecast index (EFI) EFI measures the distance between the EPS cumulative distribution and the model climate distribution Takes values from –1 (all members break climate minimum records) and +1 (all beyond model climate records) The main idea is to have an index that can be conveniently mapped – removing the effect from different climatologies – to use as an “alarm bell” 2014-10-1033

34. EFI Experience suggests that EFI magnitudes of 0.5 - 0.8 (irrespective of sign) can be generally regarded as signifying that "unusual" weather is likely whilst magnitudes above 0.8 usually signify that "very unusual" or extreme weather is likely. Although larger EFI values indicate that an extreme event is more likely, the values do not represent probabilities as such 2014-10-1034

35. 2014-10-1035 0% EPS distribution Climate distribution Clim. mean Temperature EPS mean Advantages with probability density functions Means and asymmetric variances are easily spotted 100%

36. 2014-10-1036 0% 100% EPS distribution Climate distribution Clim. mean Temperature EPS mean Advantages with probability density functions Means and asymmetric variances are easily spotted

37. 2014-10-1037 EFI ~ 70% The EFI generally does not take the probability into account

38. 2014-10-1038 EFI ~ - 50% EFI ~ 50% For temperature the EFI can take values < 0

39. EFI 2m Temp 2014-10-1039

40. EFI 10m gusts 2014-10-1040

41. EFI wind speed 2014-10-1041

42. EFI precip 2014-10-1042

43. Other products 2014-10-1043

44. Tropical strike probability 2014-10-1044

45. Tropical strike 2014-10-1045

46. Ensemble mean 2014-10-1046

47. Forecast Probability 2014-10-1047

48. UKMO African LAM 2014-10-1048

49. UKMO African LAM 2014-10-1049

50. Ensemble forecasts Ensemble forecasts help us 1.To judge the (un)certainty of the weather situation 2.To acquire probability estimates of anomalous events (extreme or high impact) 3.To get the most accurate and least “jumpy” deterministic forecast value 2014-10-1050

51. 2014-10-1051 50% ? Would you guide the ships to go steer into Newcastle harbour??

52. 2014-10-1052 Questions & Answers