I. Sanchez, M. Amodei and J. Stein Météo-France DPREVI/COMPAS Deterministic and fuzzy verification of the cloudiness of High Resolution operational models I. Sanchez, M. Amodei and J. Stein Météo-France DPREVI/COMPAS GEX-PARME "modèles conceptuels"

Summary Introduction Summer 2008 : Conclusion Verification against satellite data Simulate satellite images Summer 2008 : Illustrative example of SSI : June 11th Deterministic scores Probabilistic scores Comparison with QPF Conclusion

Verification against satellite data 3 data types : ALADIN-FRANCE 0.1 ° and mass flux convection scheme AROME 0.025 ° and explicit convection SEVIRI METEOSAT 9 Verification domain AROME domain with 0.1 ° grid Verification time: every 6 hours, instantaneous Aladin France is nested in the Arpege global model and AROME is the old version presented in the previous QPF slides. It is nested in the ALADIN model and performs a dynamical adaptation(no assimilation). The Seviri data are assimilated in the ALADIN model and their bias can be removed if desired. It does not have a strong impact. GEX-PARME "modèles conceptuels"

Simulated satellite images (SSI) Output of the model forecasts Two-dimensional SSI Temperature RTTOV_CLOUD 8 Nebulosity Specific quantities of vapor Liquid and solid water For AROME only explicit clouds are used but for ALADIN, the radaitive impact of the parameterized clouds are also taken into account.Only one IR channel is presented here but this methodology can be applied to all the channels and instruments, which are emulated by RTTOV AROME : explicit clouds ALADIN : explicit + subgrid clouds Wavelength of the Infrared channel is 10.8 micrometers GEX-PARME "modèles conceptuels"

Simulated satellite images (SSI) 11 June 2008 18 UTC Observation ALADIN No convective subgrid clouds Explicit clouds 213 K 273 K 313 K AROME Correct development of the convection GEX-PARME "modèles conceptuels"

SUMMER 18 UTC BIAS Heidke skill score 2 June – 10 September 2008 BIAS Heidke skill score AROME superiority for high clouds 3% ~5000 events 40% ~91000 events GEX-PARME "modèles conceptuels"

double-penalty and neighborhood High resolution forecast Low resolution forecast The yellow area represents the strong event observed or forecasted. The HR forecast made two errors : A false alarm and a miss if no tolerance is itaken into account. On the contrary, the LR forecast made only one error: the miss. If we tolerate a spatial error whose amplitude is lower than the size of the red squarre, the situation is reversed: the HR forecast is correct and the LR forecast still miss the observed event. Thus, we replace the yes or no forecast of the event by the frequency of the event forecasted in the red squarre i.e. we transform the deterministic forecast into a probabilistic one. Spatial tolerance Observed field Deterministic forescast Probalistic forescast GEX-PARME "modèles conceptuels"

Fuzzy approach Brier Score (BS): Brier Skill Score(BSS): 2 interesting limits : 1- Neighbourhood size = 0 : 2- Neighborhood = simulation domain Ref = persistence Ok is either the local occurence of the phenomena (1 if it happens and 0 else) or the observed frequency in the previous red squarre ( 0<=Ok<=1) Pk is the forecast probability deduced from the deterministic forecast For a null neighbourhood BSS degerates into the HSS and the limit of BS is a kind of squarre mean a the daily biais ponderated by the observed frequency. ( further details in Amodei and Stein (2009) ) GEX-PARME "modèles conceptuels"

SUMMER 18 UTC Brier skill score (NO) Heidke skill score 2 June – 10 September 2008 Neighborhood 76 Km Brier skill score (NO) HSS = BSS (10 km) Heidke skill score Removal of the double penalty for high clouds in AROME GEX-PARME "modèles conceptuels"

Average the data and the models QPF at 0.2°x0.2° QPF verification Average the data and the models QPF at 0.2°x0.2° Climatological state network ~4000 raingauges giving 24 hours accumulated rain every day 100 km GEX-PARME "modèles conceptuels"

QPF verification during SUMMER 2008 Heidke skill score QPF verification during SUMMER 2008 2 June – 10 September 2008 24 hours accumulated rain Known drawback of ALADIN convection scheme Using a fuzzy approch removes the double penalty for heavy rains and cold SSI Thresholds (mm/day) Brier skill score (NO) Neighborhood 76 Km Bias Difference is removed for heavy rains Thresholds (mm/day) Thresholds (mm/day) GEX-PARME "modèles conceptuels"

Daily evolution of SSI BSS SSI every 6 hours high clouds 240°K Neighborhood 76 Km During the night large scale cloudy phenomena replaces convective clouds Difficulties to forecast convective clouds 6 18 30 Hours UTC GEX-PARME "modèles conceptuels"

Conclusion SSI allow to document the forecast quality of the all types of clouds ALADIN and AROME under-estimate low and medium clouds. High-tropospheric clouds are quasi-absent in ALADIN forecasts. The fuzzy approach corrects the double penalty for the convection simulation for AROME but not for ALADIN. QPF and SSI verifications provide complementary information for convective events GEX-PARME "modèles conceptuels"

Future plans Define a temporal tolerance (Theis etal 2005) to reduce the double penalty for temporal misplacement Perform the QPF verification for 6 hours accumulated precipitation. Compare both information provided by these new verifications for SSI and QPF Operational use of both verifications GEX-PARME "modèles conceptuels"

The End GEX-PARME "modèles conceptuels"