1. CIAO Priority Task Andrea Montani
Arpae Emilia-Romagna Servizio IdroMeteoClima (Arpae-SIMC), Bologna, Italy
COSMO GM
Jerusalem, September 2017

2. COSMO Priority Task CIAO:
implementation of the Bechtold Convection scheme In COSMO model: deterministic And ensemble-mOde tests
Main goal:
To assess the sensitivity of COSMO forecast skill to the use of the newly implemented Bechtold convection scheme

3. Project structure SubTask1: benchmark of COSMO-B
COSMO-T: COSMO run performed with Tiedtke convection scheme
COSMO-B: COSMO run performed with Bechtold convection scheme
SubTask1: benchmark of COSMO-B
SubTask2: tests of COSMO-B in deterministic mode
SubTask3: test of COSMO-B in ensemble mode
SubTask4: COSMO-B and COSMO-T in ensemble mode
Duration: –

4. SubTask1: benchmark of COSMO-B
(Bechtold scheme available since version 5.04b)
Perform a benchmark of the COSMO integrations in deterministic mode to assess and refine the technical details of COSMO-B....
Deliverables: Identification of the optimal configuration of COSMO-B runs for the different set-ups.
Start: 04/17 - End: 06/17

5. Overview of the work Massimo Milelli Valeria Garbero
Model version: 5.04e
&PHYCTL CTRL
lsuper_coolw=.true.,
itype_conv=0,
lseaice=.FALSE.,
ltkesso=.true.,
lconf_avg=.false.,
nincrad=55,
itype_albedo=1,
icapdcycl=2,
icpl_aero_conv=0,
/END
&PHYCTL BECHT
lsuper_coolw=.true.,
itype_conv=2,
lseaice=.FALSE.,
ltkesso=.true.,
lconf_avg=.false.,
nincrad=55,
itype_albedo=1,
icapdcycl=3,
icpl_aero_conv=1,
/END
Massimo Milelli
Valeria Garbero
11/09/2017 – 19th COSMO General Meeting, Jerusalem

6. Massimo Milelli Valeria Garbero
12h tot_prec, UTC, +36h FORECAST
CTRL (TIED) RADAR BECHT
Massimo Milelli
Valeria Garbero
11/09/2017 – 19th COSMO General Meeting, Jerusalem

7. SubTask2: tests of COSMO-B in deterministic mode
For past cases of heavy precipitation, the performance of COSMO-B and COSMO-T will be investigated in deterministic mode….
“Standard” verification scores will be used …..
Novel spatial verification techniques will be also used …. (SAL, INSPECT results)
Deliverables: for the investigated case studies, assessment of the skill of COSMO-B and COSMO-T in terms of the above-mentioned scores so as to have a detail description of the potential strengths and weaknesses of the individual convection schemes.
Start: 06/17 - End: 10/17

8. Model version: 5.04d COSMO-ME (5 km) Marco Alemanno Domain size
1083 x 559
Grid spacing
0.045° (5 km) since May 2017
0.0625° (7 km) earlier
Number of layers / top
45 / ~22 km since May 2017
40 / ~22 km earlier
Time step
60 s
Forecast range
72 hrs
Initial time of model run
00/06/12/18 UTC
Lateral bound. condit.
IFS
L.B.C. update freq.
3 hrs
Initial state
Interpol. LETKF
Initialization
None
External analysis
T,u,v, qv,ps, snow mask
Special features
Filtered topography
Status
Operational
Marco Alemanno

9. FBI and TS values for the event: «6h tot. prec. > 0. 1 mm» (01. 09
In autumn-winter period, Bechtold scheme over forecasts precipitation events with threshold of 0.1 mm (FBI>1). Better FBI values for Tiedtke scheme. In this case, Tiedtke scheme predicts the event with an higher percentage of success (higher values of TS for Tiedtke).
Marco Alemanno

10. FBI and TS values for the event: «6h tot. prec. > 2. 0 mm» (01. 09
In autumn-winter period, Bechtold scheme over forecasts precipitation events with threshold of 2.0 mm (FBI>1). Better FBI values for Tiedtke scheme. In this case, Bechtold scheme predicts the event with an higher percentage of success (higher values of TS for Bechtold).
Marco Alemanno

11. SubTask3: test of COSMO-B in ensemble mode
Implement cleps_10b:
10-member ensemble (7km, 40 ML, ….) starting at 00UTC,
all members are run with COSMO-B,
IC/BCs (+ soil conditions) are the same as those from members of COSMO-LEPS (cleps_10t).
Compare cleps_10b vs cleps_10t in terms of surface variables (e.g. TP, T2M, TD2M) over a long period as well as for case studies .
Deliverables: Assessment of the individual skill of cleps_10b and cleps_10t for different verification times, computed over the full verification period as well as for particular cases.
Start: 06/17 - End: 12/17

12. SubTask4: COSMO-B and COSMO-T in ensemble mode
Implement cleps_20bt:
cleps_20bt has 20 members, 10 members run with Bechtold (cleps_10b) + 10 members run with Tiedtke (members 1-10 of COSMO-LEPS).
Compare cleps_20bt vs COSMO-LEPS in terms of surface variables (e.g. TP, T2M, TD2M) over a long period as well as for case studies .
The use of the Versus package is also envisaged…
Deliverables: Assessment of the skill of cleps_20bt and COSMO-LEPS….
Start: 08/17 - End: 08/18
First tests performed at Arpae-SIMC
variable: 6h cumulated precip (00-06, 06-12, ... UTC);
period : from 28 March to 31 May 2017 (~ 60 days);
region: Northern Italy;
method: nearest grid point; no-weighted fcst;
obs: non-GTS network (~ 1000 stations/day);
fcst ranges: 0-6h, 6-12h, …, h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
systems: cleps_20bt vs cosmo-leps;
scores: ROC area, BSS, RPSS, Outliers, …

13. cleps_20bt vs cosmo-leps
BSS is written as 1-BS/BSref. Sample climate is the reference system.
BS measures the mean squared difference between forecast and observation in probability space.
Consider three events: 6-hour precipitation exceeding 1, 5 and 15 mm.
RPSS: BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. RPS is the extension of the Brier Score to the multi-event situation. Useful forecast systems for BSS > 0, RPSS > 0.
1mm
5 mm
15 mm
cleps_20bt slightly outperforms COSMO-LEPS, more clearly after fc+48h.
Matteo Vasconi (Univ. Bologna)

14. First conclusions and open issues:
subTask 1: benchmark of COSMO-B
Computational time and scaling comparable.
As for total precipitation, results are comparable in space, but smoothed intensity in Bechtold.
subTask 2: tests of COSMO-B in deterministic mode (COSMO-ME results)
The switching from Tiedtke to Bechtold scheme mainly affects forecasts of total precipitation.
Bechtold scheme underestimates high-threshold total precipitation amounts, overpredicts light precipitation events and underpredicts heavy precipitation events.
COSMO-ME with Bechtold scheme is more capable to highlight the local/isolated convection events (poor performances of Tiedtke scheme in these cases).
Further tuning/improvements of Bechtold scheme for high-thresholds is needed.
subTask 4: COSMO-B and COSMO-T in ensemble mode
Encouraging results by the use of a “multi-scheme” ensemble system.
More thorough tests once COSMO v5.05 is available so as to perform clean experiments.
Understand namelists switches (link to WG3a? ICON people?).
Look at vertical profile differences between COSMO-B and COSMO-T runs: which variables to store for diagnostics?

15. Ciao and thanks for your attention

16. About namelists (and docu)
&PHYCTL
itype_conv=2,  with "2" the Bechtold scheme is chosen
icpl_aero_conv=1,
icapdcycl=3,
…….
New namelist switches:
     Name                       Description                                       Default:     icpl_aero_conv       : type of coupling between aerosols and convection scheme   0         =0:         =1:     icapdcycl            :                                                           0         =0 no CAPE diurnal cycle correction (IFS default prior to cy40r1, i.e )         =1    CAPE - surface buoyancy flux (intermediate testing option)         =2    CAPE - subcloud CAPE (IFS default starting with cy40r1)         =3    Apply CAPE modification of (2) over land only, with additional restriction to the tropics

17. More on namelists /TUNING/ --------
  For Tiedtke-Bechtold, the following new namelist variables have been implemented as tuning parameters:      Name                       Description                                        Default:     tune_capdcfac_et     : fraction of CAPE diurnal cycle correction applied                                    in the extratropics     tune_rhebc_land      : relative humidity threshold for onset of evaporation                                 below cloud base over land     tune_rhebc_ocean   : relative humidity threshold for onset of evaporation      0.85                             below cloud base over sea     tune_texc            : excess value for temperature used in test                                            parcel ascent     tune_qexc            : excess fraction of grid-scale QV used in                                              test parcel ascent     tune_rcucov          : convective area fraction                                     tune_entrorg         : entrainment parameter for deep convection valid at                                  dx=20 km     tune_rhebc_land_trop : relative humidity threshold for onset of evaporation                                 below cloud base over land in the tropics     tune_rhebc_ocean_trop: relative humidity threshold for onset of evaporation                                 below cloud base over sea in the tropics     tune_rcucov_trop     : convective area fraction in the tropics                0.05
No documentation

18. SubTask3: test of COSMO-B in ensemble mode
Implement cleps_10b:
10-member ensemble (7km, 40 ML, ….) starting at 00UTC,
all members are run with COSMO-B,
IC/BCs (+ soil conditions) are the same as those from members of COSMO-LEPS (cleps_10t).
Compare cleps_10b vs cleps_10t in terms of surface variables (e.g. TP, T2M, TD2M) over a long period as well as for case studies .
Deliverables: Assessment of the individual skill of cleps_10b and cleps_10t for different verification times, computed over the full verification period as well as for particular cases.
First evaluations performed over Northern Italy
variable: 6h cumulated precip (00-06, 06-12, ... UTC);
period : from 28 March to 31 May 2017 (~ 60 days);
region: Northern Italy;
method: nearest grid point; no-weighted fcst;
obs: non-GTS network (~ 1000 stations/day);
fcst ranges: 0-6h, 6-12h, …, h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
systems: cleps_10b vs cleps_10t;
scores: ROC area, BSS, RPSS, Outliers, …

19. SubTask4: COSMO-B and COSMO-T in ensemble mode
Implement cleps_20bt, :
cleps_10b + members 1-10 of COSMO-LEPS;
cleps_20bt has 20 members: 10 members run with Bechtold plus 10 members run with Tiedtke (no duplication of boundary conditions).
Compare cleps_20bt vs COSMO-LEPS in terms of surface variables (e.g. TP, T2M, TD2M) over a long period as well as for case studies .
The use of the Versus package is also envisaged…
Deliverables: Assessment of the skill of cleps_20bt and COSMO-LEPS….
variable: 6h cumulated precip (00-06, 06-12, ... UTC);
period : from 28 March to 31 May 2017 (~ 60 days);
region: Northern Italy;
method: nearest grid point; no-weighted fcst;
obs: non-GTS network (~ 1000 stations/day);
fcst ranges: 0-6h, 6-12h, …, h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
systems: cleps_20bt vs cosmo-leps;
scores: ROC area, BSS, RPSS, Outliers, …

20. Massimo Milelli Valeria Garbero
CTRL
BECHT
Massimo Milelli
Valeria Garbero
11/09/2017 – 19th COSMO General Meeting, Jerusalem

21. Preliminary conclusions
Computational time comparable
Scaling comparable
Results comparable in space, but smoothed intensity in Bechtold
Massimo Milelli
Valeria Garbero
11/09/2017 – 19th COSMO General Meeting, Jerusalem

22. Overview of the work Model version: 5.04e Domain: operational COSMO-I7
Namelists: operational COSMO-I7
Forecast time: +48h
Boundary frequency: 3h, from IFS
Output frequency: 1h
Data assimilation: no (cold start)
Case study: 18 May 2017
Number of cores: 64 (8x8)
Simulation 1: ctrl (Tiedke scheme)
Simulation 2: becht (Bechtold scheme)
Simulation 3: ctrl_16 (ctrl with 16 cores, 4x4)
Simulation 4: becht_16 (becht with 16 cores, 4x4)
11/09/2017 – 19th COSMO General Meeting, Jerusalem

23. Verification features
SubTask2: case studies over Serchio river basin (~ 2000 km2, Tuscany region)
Verification features
Area: Serchio river basin (~ 2000 km2) .
Measure: 24h area-averaged precipitation.
Obs: about 50 non-GTS stations.
Period: 31/01 – 07/02/2017.
Model features
COSMO v5.04e-beta-corrected.
7km, 40ML (about 50 grid points in the basin).
fc+132h, 00UTC runs only.
COSMO-LEPS integration domain.
BC/IC from ECMWF HRES.
Soil from ICON-Regional.
Matteo Vasconi (Univ. Bologna)

24. 24-h area-average precipitation (fcst 24-48h)
COSMO-T: some forecasts with under-estimation and one forecast with severe over-estimation.
COSMO-B: area-averaged precipitation forecasts are clearly closer to obs than operational COSMO-T, for cases of both under- and over-estimation.
COSMO-T
COSMO-B
MAE
9.3 mm
5.3 mm
BIAS
8.3 mm
3.6 mm
Matteo Vasconi (Univ. Bologna)

25. Precipitation maps (fcst 24-48h, VT: 20170206 00UTC)
COSMO-T
COSMO-B
COSMO-B – COSMO-T
In the Serchio river basin, differences larger than 70 mm

26. cleps_10b vs cleps_10t 1mm 5 mm 15 mm
Brier Skill Score (BSS) is written as 1-BS/BSref. Sample climate is the reference system.
BS measures the mean squared difference between forecast and observation in probability space.
Consider three events: 6-hour precipitation exceeding 1, 5 and 15 mm.
Ranked Probability Skill Score (RPSS): it is sort of BSS “cumulated” over all thresholds.
RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.
Useful forecast systems for RPSS > 0.
1mm
5 mm
15 mm
As for 6-hourly cumulated precipitation, cleps_10t slightly better than cleps_10b especially for rain/no-rain situations.
Some negative impact by cleps_10b for short ranges.
Matteo Vasconi (Univ. Bologna)

27. SubTask4: COSMO-B and COSMO-T in ensemble mode
COSMO runs in ensemble mode (hor. Res. 7 km, 40 ml, 132 fcst range) with the Bechtold scheme; 10 members are run with the same IC/BCs as members 1-10 of operational COSMO-LEPS (all members use Tiedtke scheme)
Verification over the peridod 28/3 - end of May for the 10-member ensemble
Confronto skill 10b vs 10t (cioè primi 10 mebri di COSMO-LEPS
Verification of the new ensemble system so generated:
20 membri: 10 girati con lo schema di Bechtold + 10 girati con lo schema di Tiedtke (quello attualmente in uso in COSMO-LEPS): NB stesse ic e bc di COSMO-LEPS
Confronto tra fc skill del nuovo sistema e quella di COSMO-LEPS su casi di precipitazione sui 2 mesi di verifica

28. SubTask1: benchmark of COSMO-B
Perform a benchmark of the COSMO integrations in deterministic mode to assess and refine the technical details (cost of the runs, set-up to be implemented) of COSMO-B. The runs will be carried out over integration domains covering Central-Southern Europe and Italy, at the horizontal resolutions of about 7 and 5 km.
Deliverables: identification of the optimal configuration of COSMO-B runs for the different set-ups.
Participating scientists: V. Garbero (ARPA-Pie), P. Mercogliano (CIRA), A. Montani (Arpae-SIMC), M. Alemanno (Comet)
Start: 04/17 - End: 06/17

29. SubTask2: tests of COSMO-B in deterministic mode
Verification over the Serchio river basin (~ 2000 km2) in terms of areal average precipitation.
About 70 non-GTS stations
Jkjkkjk
Jkkkj
About 1000 stations (non-GTS obs)