1. I. Sanchez, M. Amodei and J. Stein Météo-France DPREVI/COMPAS
Deterministic and fuzzy verification of the cloudiness of High Resolution operational models
I. Sanchez, M. Amodei and J. Stein
Météo-France
DPREVI/COMPAS
GEX-PARME "modèles conceptuels"

2. Summary Introduction Summer 2008 : Conclusion
Verification against satellite data
Simulate satellite images
Summer 2008 :
Illustrative example of SSI : June 11th
Deterministic scores
Probabilistic scores
Comparison with QPF
Conclusion

3. Verification against satellite data
3 data types :
ALADIN-FRANCE 0.1 ° and mass flux convection scheme
AROME ° and explicit convection
SEVIRI METEOSAT 9
Verification domain AROME domain with 0.1 ° grid
Verification time:
every 6 hours, instantaneous
Aladin France is nested in the Arpege global model and AROME is the old version presented in the previous QPF slides. It is nested in the ALADIN model and performs a dynamical adaptation(no assimilation).
The Seviri data are assimilated in the ALADIN model and their bias can be removed if desired. It does not have a strong impact.
GEX-PARME "modèles conceptuels"

4. Simulated satellite images (SSI)
Output of the model forecasts
Two-dimensional SSI
Temperature
RTTOV_CLOUD 8
Nebulosity
Specific quantities of vapor
Liquid and solid water
For AROME only explicit clouds are used but for ALADIN, the radaitive impact of the parameterized clouds are also taken into account.Only one IR channel is presented here but this methodology can be applied to all the channels and instruments, which are emulated by RTTOV
AROME : explicit clouds
ALADIN : explicit + subgrid clouds
Wavelength of the Infrared channel is 10.8 micrometers
GEX-PARME "modèles conceptuels"

5. Simulated satellite images (SSI) 11 June 2008 18 UTC
Observation
ALADIN
No convective subgrid clouds
Explicit clouds
213 K
273 K
313 K
AROME
Correct development of the convection
GEX-PARME "modèles conceptuels"

6. SUMMER 18 UTC BIAS Heidke skill score
2 June – 10 September 2008
BIAS
Heidke skill score
AROME superiority for high clouds 3%
~5000 events
40%
~91000 events
GEX-PARME "modèles conceptuels"

7. double-penalty and neighborhood
High resolution forecast
Low resolution forecast
The yellow area represents the strong event observed or forecasted. The HR forecast made two errors : A false alarm and a miss if no tolerance is itaken into account. On the contrary, the LR forecast made only one error: the miss. If we tolerate a spatial error whose amplitude is lower than the size of the red squarre, the situation is reversed: the HR forecast is correct and the LR forecast still miss the observed event. Thus, we replace the yes or no forecast of the event by the frequency of the event forecasted in the red squarre i.e. we transform the deterministic forecast into a probabilistic one.
Spatial tolerance
Observed
field
Deterministic forescast
Probalistic forescast
GEX-PARME "modèles conceptuels"

8. Fuzzy approach Brier Score (BS): Brier Skill Score(BSS):
2 interesting limits :
1- Neighbourhood size = 0 :
2- Neighborhood = simulation domain
Ref = persistence
Ok is either the local occurence of the phenomena (1 if it happens and 0 else) or the observed frequency in the previous red squarre ( 0<=Ok<=1)
Pk is the forecast probability deduced from the deterministic forecast
For a null neighbourhood BSS degerates into the HSS and the limit of BS is a kind of squarre mean a the daily biais ponderated by the observed frequency.
( further details in Amodei and Stein (2009) )
GEX-PARME "modèles conceptuels"

9. SUMMER 18 UTC Brier skill score (NO) Heidke skill score
2 June – 10 September 2008
Neighborhood 76 Km
Brier skill score (NO)
HSS = BSS (10 km)
Heidke skill score
Removal of the double penalty for high clouds in AROME
GEX-PARME "modèles conceptuels"

10. Average the data and the models QPF at 0.2°x0.2°
QPF verification
Average the data and the models QPF at 0.2°x0.2°
Climatological state network
~4000 raingauges giving 24 hours accumulated rain every day
100 km
GEX-PARME "modèles conceptuels"

11. QPF verification during SUMMER 2008
Heidke skill score
QPF verification during SUMMER 2008
2 June – 10 September 2008
24 hours accumulated rain
Known drawback of ALADIN convection scheme
Using a fuzzy approch removes the double penalty for heavy rains and cold SSI
Thresholds (mm/day)
Brier skill score (NO) Neighborhood 76 Km
Bias
Difference is removed for heavy rains
Thresholds (mm/day)
Thresholds (mm/day)
GEX-PARME "modèles conceptuels"

12. Daily evolution of SSI BSS SSI every 6 hours
high clouds 240°K
Neighborhood 76 Km
During the night large scale cloudy phenomena replaces convective clouds
Difficulties to forecast convective clouds 6 18 30
Hours UTC
GEX-PARME "modèles conceptuels"

13. Conclusion
SSI allow to document the forecast quality of the all types of clouds
ALADIN and AROME under-estimate low and medium clouds.
High-tropospheric clouds are quasi-absent in ALADIN forecasts.
The fuzzy approach corrects the double penalty for the convection simulation for AROME but not for ALADIN.
QPF and SSI verifications provide complementary information for convective events
GEX-PARME "modèles conceptuels"

14. Future plans
Define a temporal tolerance (Theis etal 2005) to reduce the double penalty for temporal misplacement
Perform the QPF verification for 6 hours accumulated precipitation.
Compare both information provided by these new verifications for SSI and QPF
Operational use of both verifications
GEX-PARME "modèles conceptuels"

15. The End
GEX-PARME "modèles conceptuels"