Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Limbering up VERSUS for operational use COSMO-GM, September 2011, Roma Vanessa Stauch (MeteoSwiss)

2 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, The Vision: furthest possible automation of VERSUS Each month/season/year, the calculation and visualisation of the desired verification scores for the desired forecast models is executed with a “jump”

3 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Necessary steps for oper-verif Prerequisites: 1.configuration of the desired verification types (per model, parameter and type) » once (and if there are additional needs) 2.Regular ingestion of the newest data into the database (forecasts, observations, weather types) » e.g. cronjob Operational run: 1.Execute batch jobs (one per model) that saves results and plots 2.Distribute plots for example into an intranet 3.INTERPRETATION OK

4 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Questions parameter/verifications Continuous scores for precipitation? Precipitation temporal aggregates: which hours are aggregated? Is this independent from the start time of the model run? e.g. 12hourly sums standard: 06UTC- 18UTC, 18UTC-06UTC. Gridpoint association: changed to land fraction < 0.5 (instead of 1?) Suspect observations, general default/standard definition? Standard verification of wind direction conditional on wind speed (e.g. > 3m/s)?

5 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Other operational duties: test chains Prerequisites: 1.configuration of the desired verification types (per model, parameter and type) » on the basis of the operational templates 2.Event-based ingestion of the data for the chosen time period into the database (forecasts reference, forecasts experiment, observations, weather types) Run: 1.Execute batch jobs for a specified time period (?) 2.Distribute plots for example into an intranet 3.INTERPRETATION OK TIPS?

6 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Prerequisites: Regular ingestion of the newest data into the database VERSUSRussiaPolandGreece Load obs~1h ~3h~30min Load fcst~2h~3.5h~7h~1h verif Temperature 22.5sec21.4sec54sec13.8sec verif Precipitation 127sec19.9sec58.7sec107.1sec Results of the performance tests (Nov 2010): Results not exactly as expected Switzerland: no actual figures but happy with the loading performance Germany:

7 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Necessary steps for oper-verif Prerequisites: 1.configuration of the desired verification types (per model, parameter and type) » once (and if there are additional needs) 2.Regular ingestion of the newest data into the database (forecasts, observations, weather types) » e.g. cronjob Operational run: 1.Execute batch jobs (one per model) that saves results and plots 2.Distribute plots for example into an intranet 3.INTERPRETATION

8 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Batch jobs: wishes Very important feature of VERUS! Performance improvement necessary: most reported limitations (Russia, Greece, Switzerland) when many jobs, or demanding jobs are started Running jobs (data loading and verification jobs) affect the entire machine (limits other work) Include annual time period (particularly important for weather type dependent verification, COSI) Include automatic storage of figures? Include cross model graphics? Filodealix

9 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Distribution of plots for analysis Individual, but this is one possibility: INTERPRETATION

10 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Technical problems & suggestions Administration: (batch) Several jobs (e.g. verifications and data ingestions) at the same time leads to time outs Time outs most often associated with WTD verification with 2 conditions (e.g. complicated verifications) When time out problems occur: /var/lib/mysql/.tmp directory is getting full and stops the VERSUS activities at some point. Solution is to manually empty it and reboot the system Log files accumulate unhampered, automatically zip files? Configuration: avoid confusion of forecast parameters (e.g. wind speed, table 2 vs. table 69) Fcst parameter list when registering a verification is mcuh smaller than the list in configuration, why? Verification: From cross model graphic results back to menu is missing, download graphic does not work Precipitation graphs: add number of obs-fcst pairs Precipitation graphs: lead time dependent graphic registration: choice to show or not show lead time? COSI: wind vector compared to which obs? Wind verif is missing in scores list COSI: only monthly available, UK MetOffice applies moving averages (30 months windows), possible? Cross model graphics: registration shows too little detail about the models to compare (particularly necessary for comparison of test chains) WTD verification, display number of cases per weather class in the analysed period Report: selection of e.g. parameters: tick on several pages in one go Cloud cover forecasts database ingestion: no errors reported but no data available Forecast of hour +33h lands in error directory, but no error message in log file …

11 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Management of bug reports Need an information system (e.g. alert) if there are new entries on this forum

12 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Open questions Current status of: Test chains: grib definition should be decided in WG6 Temp: feedback files? Verification of assimilation cycle (do what is possible in VERSUS not with forecasts but with analyses » no lead time)? Rain gauges of Uli? Do we want to include them? Radar data? Multiple users/ several jobs at the same time

13 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Conclusions Limbering up VERSUS is (apart from its extension with further functionalities) a key activity to ensure that it can be used for operational purposes A good performance is a key property of an accepted system that is used on a day-to-day basis The reporting is best organised on a VERSUS forum

14 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Problem precip 0.1mm/ 12h 1mm /12h 10mm/ 12h

15 Local wind forecasts | ECAC/EMS 2011, Berlin Vanessa Stauch, Problem VMAX