Automated weather warning proposals based on post-processed numerical weather forecasts Guido Schröder, Bernhard Reichert, Dirk Heizenreder Deutscher Wetterdienst, Offenbach am Main, Germany 19 August 2014, WWOSC, Montreal, Canada

2 The simplified warning process Observations Synop, Radar, etc. Numerical Models GME, COSMO-DE-EPS, IFS, etc Other forecast products Post-processed model forecasts, NowCastMIX, etc. Warnings Forecasters Guido Schröder, DWD, Germany

3 The simplified warning process with AutoWARN Observations Synop, Radar, etc. Numerical Models GME, COSMO-DE-EPS, IFS, etc. Other forecast products Post-processed model forecasts, NowCastMIX, etc. ModelMIX Statistical Post-processing Automated warnings Forecasters integrate automated warnings Guido Schröder, DWD, Germany Warnings

1) On the characteristics of weather warnings 2) Automatic generation of warning proposals 3) Verification with station observations 4) Summary and outlook Guido Schröder, DWD, Germany 4

1) On the characteristics of weather warnings 2) Automatic generation of warning proposals 3) Verification with station observations 4) Summary and outlook Guido Schröder, DWD, Germany 5

Manual generation of warnings – Example for gust warnings (winter storm Xaver 2013) 6 - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Station observations Height levels: A - > 0m B - > 200m C - > 400m D - > 600m E - > 800m F - > 1000m G - > 1500m H - > 2000m Guido Schröder, DWD, Germany Good weather warnings are:  accurate  significant  meteorologically consistent  overwarn rather than underwarn  homogeneous  simple

Manual generation of warnings – Example for gust warnings (winter storm Xaver 2013) 7 - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Manually generated warnings – here for Bft 8-9 only Using a DWD layer within NinJo the forecasters define (draw) polygons for which a specific warning is valid for a given time. Station observations Guido Schröder, DWD, Germany

Manual generation of warnings – Example for gust warnings (winter storm Xaver 2013) 8 - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Using a DWD layer within NinJo the forecasters define (draw) polygons for which a specific warning is valid for a given time. All manually generated gust warnings Manually generated warnings – here for Bft 8-9 only Guido Schröder, DWD, Germany

Manual generation of warnings – Example for gust warnings (winter storm Xaver 2013) 9 - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Automatically generated warning proposals To support the manual warning generation warning proposals are generated automatically based on post-processed numerical model forecasts. All manually generated gust warnings Guido Schröder, DWD, Germany

1) On the characteristics of weather warnings 2) Automatic generation of warning proposals 3) Verification with station observations 4) Summary and outlook Guido Schröder, DWD, Germany 10

11 Guido Schröder, DWD, Germany  For mountainous areas predefined polygons will be used to generate warning proposals for upper levels. 2a) Mountainous areas

12 Guido Schröder, DWD, Germany 2b) Lower levels – maximum event Lower levels The maximum gust forecast for the event is computed for lower levels on a 20kmx20km grid. In mountainous areas it is the wind speed forecast within the valleys. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

The gridded data is smoothed with the constraint that the gusts can only increase. Overwarning is preferred to underwarning. 13 Guido Schröder, DWD, Germany 2c) Smoothing gridded data - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Smoothing

14 Guido Schröder, DWD, Germany 2d) Generation of warning regions Based on contour lines regions with the same maximum warning are extracted. Too small polygons are removed as they are insignificant. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

15 Summary – temporal fragmentation Guido Schröder, DWD, Germany 2e) Temporal fragmentation - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 Regions with similar starting and end time are clustered together. Stronger warnings are nested into weaker warnings. This way the warning regions are temporally split into fragments.

1) On the characteristics of weather warnings 2) Automatic generation of warning proposals 3) Verification with station observations 4) Summary and outlook Guido Schröder, DWD, Germany 16

Guido Schröder, DWD, Germany , ) Verification result for Oct 13-Feb 14 Bft 12 >12 ForecastersAutomated system Each horizontal bar (normalized to the same width) corresponds to all observed events for a given wind speed category. The colors indicate which fraction of the observed events were observed correctly (green) or incorrectly (other colors). The numbers in each box is the number of actual events underwarningoverwarningunderwarningoverwarning

1) On the characteristics of weather warnings 2) Automatic generation of warning proposals 3) Verification with station observations 4) Summary and outlook Guido Schröder, DWD, Germany 18

19 Guido Schröder, DWD, Germany 4) Summary  The increased amount of data in the warning process requires more and more automization  The automated system tries to generate warning proposals the same way the forecaster would do it. That implies Separate treatment of mountainous areas Maximum of the event: It is more important to get the location right than the timing Significance: Smoothing the gridded data  The automated warnings give slightly better verification results than the manually generated warnings  For low wind speeds (Bft 7) the automated system has the tendency to underwarn – the system needs to be optimized to generate automated warning at lower thresholds for Bft 7

20 Guido Schröder, DWD, Germany 4) Outlook  For wind gusts the AutoWARN system is already being tested  Tools needs to be developed to better integrate the warning proposals into the actual warnings  More research is needed in how to integrate several more numerical models (e.g. ICON, ICON-EPS, IFS-EPS) and data sources  Probabilities for the events to occur need to be issued along with the warnings

Appendix Guido Schröder, DWD, Germany 21

Strategic Goals of DWD for the Weather Warning Service DWD Strategy :  Development of a system for an optimal decision support in the warning process  Automated support for the warning service and for the production of customer specific warn products  Stepwise centralization of the warning service from current regional centers to the DWD headquarter in Offenbach  Development of AutoWARN: An automated decision support system with manual monitoring and decision capabilities for the forecaster 22 Guido Schröder, DWD, Germany

IFS COSMO-DE-EPS Data sources for warnings generation 23 Guido Schröder, DWD, Germany  Numerical models (DWD-models GME and COSMO-DE-EPS, ECMWF- model IFS)  Statistically post-processed numerical models (ModelMIX, Hirsch et al., WWOSC 2014)  Observations (station data, radar observations) GME ModelMIX IFS COSMO-DE-EPS GME Automated warnings

Manual generation of warnings – Example for gust warnings (winter storm Xaver 2013) 24 - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12 All warnings issued The actual warnings are issued on County (“Landkreis”) level by an automated system that takes the manually generated warnings as input. All manually generated gust warnings Guido Schröder, DWD, Germany

B. Reichert, FE ZE, DWD Manually Edited Future Warn Status Time interval Elevation interval Warning Event Attributes Indicator: New Significant Warn Proposal Automatic Warn Proposal

Guido Schröder, DWD, Germany 26 2a) Event extraction The raw observed time series is inflated to reduce fluctuations. Between two adjacent local maxima the observations are artificially increased. This is achieved applying the above formula twice to the data. Inflated observation Raw observation

Guido Schröder, DWD, Germany , The inflated wind speeds are converted into the event categories. Distinct maximum events are extracted. Only events with at least 3h duration are used for verification. 2a) Event extraction Bft

28 Warning polygons for mountainous areas Guido Schröder, DWD, Germany 1h) Merging of warning polygons for mountainous areas The maximum event in the mountainous areas is determined. Redundant polygons are removed. The remaining polygons are spatially merged where possible.

Appendix Smoothing Guido Schröder, FEZE-B, DWD 29

30 Smoothing Guido Schröder, FEZE-B, DWD Temporal clustering

Filter for smoothing  Higher order filters (Shapiro 1971, also Schlünzen et al., 1996) Non-filtered value at grid point i Filtered value at grid point i 3-Points 5-Points 7-Points

Dots are model data The blue line is the filtered model data  Higher order filters (Shapiro 1971, also Schlünzen et al., 1996) Filter for smoorhing

Multi-linear Regression for smoothing  Assuming model values at, these can be approximated using a multi- linear regression with the regression function where. Here where is generated as linear combination of with the coefficients The error is minimized.  The base functions can be fields that describe the main structures in the data.  It is impossible to attain a perfect fit everywhere – this is why a local regression is done – for each grid point with n surrounding grid points.  For each grid point there are n values where k = 1, …,n. Of these a weighted average is computed while taking into account the error corresponding to the regression function. The smoothed value is then On this slide k and g are no exponents!

Multi-linear regression for smoothing  The example below uses the base functions with n=11  The model data (dots) is discontinuous – the optimal regression curve is taking that into account. Individual local regression curves (red) don‘t. Example regression for i=41

Schröder / FEZE-B – 10/2012 Multi-linear regression for smoothing  The example below uses the base functions with n=11  The model data (dots) is discontinuous – the optimal regression curve is taking that into account. Individual local regression curves (red) don‘t. Example regression for i=41 All regressions for i=41

Schröder / FEZE-B – 10/2012 Multi-linear regression for smoothing Dots are model data The blue line is the smooth data The green line is derived with the 7-point-filter

Schröder / FEZE-B – 10/2012 Quadratic programming for smoothing  Multi-linear regression does not preserve maxima – which could mean that extreme warnings are smoothed away – which is not acceptable  This problem can be solved by indroducing a constraint to the linear regression  This is a quadratic programming problem and can be solved with standard solvers  The above constraint ensures that all values can only increase. It is also possible to preserve only selected local maxima.

Schröder / FEZE-B – 10/2012 Multi-lineare regression for smoothing Glätter  If all values can only increase, this will lead to overwarning  If only selected maxima are pteserved, the overwarned area is smaller No constraint Values can only increase Only some maxima are preserved Dots are model data The blue line is the smooth data The green line is derived with the 7-point-filter

Appendix temporal clustering Guido Schröder, FEZE-B, DWD 39

40 Preperation for temporal fragmentation Guido Schröder, DWD, Germany 1e) Temporal fragmentation The polygons are fragmented with a regular hexagon grid. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

41 Guido Schröder, DWD, Germany For each polygon the starting and end time within each hexagon is computed. Starting time End time h 1f) Temporal fragmentation

42 Clustering of hexagons with similar starting and end time Guido Schröder, DWD, Germany Regions with similar starting and end time are clustered together. Stronger warnings are nested into weaker warnings. 1g) Temporal fragmentation - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

43 Intersection with the original contour polygons Guido Schröder, DWD, Germany The hexagon clusters are smoothed and interseted with the original 1h) Temporal fragmentation - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

44 Guido Schröder, FEZE-B, DWD Temporal clustering The warning regions are split into hexagons. Each hexagon is assigned a starting and end time. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

45 Guido Schröder, FEZE-B, DWD Temporal clustering The starting and end time for each hexagon is determined Starting time End time h

46 Guido Schröder, FEZE-B, DWD Temporal clustering The hexagons are clustered together taking into account allowed error tolerances. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12

47 Guido Schröder, FEZE-B, DWD Temporal clustering The hexagon clusters are smoothed and then intersected with the original warning regions. - Bft 7 Bft 8,9 Bft 10 Bft 11 Bft 12 > Bft 12