1. Robin Hogan Ewan OConnor Damian Wilson Malcolm Brooks Evaluation statistics of cloud fraction and water content

2. Overview Cloudnet level 3 data A solution to the problem of evaluating high cloud? Summary of errors in model cloud fraction and water content climatologies over Europe –ECMWF model –KNMI Regional atmospheric climate model (RACMO) –Met Office mesoscale and global –SMHI Rossby Centre atmospheric model (RCA) –Meteo France ARPEGE model –DWD Lokal Modell Forecast skill

3. Cloudnet level 3 data Level 3 files summarise the comparison of a observations and model over a certain period: –Long-term mean of a quantity versus height –Separation into freq. of occurrence and amount when present –PDFs in height ranges 0-3 km, 3-7 km, 7-12 km and 12-18 km –Skill scores versus height for different thresholds Separate level-3 files/quicklooks are produced for –Each variable: cloud fraction, LWC, IWC, high cloud fraction –Each site: 4 European, 4 ARM (so far) –Each model: 7 so far, plus persistence/climatology forecasts –Each month and each year –Different forecast lead times (Met Office meso and DWD only) –In principle: different model resolutions / parameterisations Over 5000 files so far!

4. Observations Met Office Mesoscale Model ECMWF Global Model Meteo-France ARPEGE Model KNMI RACMO Model Swedish RCA model Cloud fraction

5. What can we do about high cloud? All models see more cirrus than observed –We use the known radar sensitivity to remove clouds from model that we would not expect to detect (affecting heights > 7 km) –Does not usually remove enough cloud to bring into agreement Are all models wrong? –Or does radar miss more IWC than it thinks due to small particles?

6. ARM Nauru 8 Nov 2003 Night-time Radar 35 GHz MMCR Lidar Merged ceilometer and micropulse lidar

7. October 2003: Normal processing No periods when rain rate > 8 mm/h Large difference between observations and ECMWF model, whether model is modified for radar sensitivity or not

8. …only periods of high lidar sensitivity Consider only night-time and periods when lidar is unobscured by liquid cloud, rain or melting ice Liquid clouds removed from comparison Cloud fraction OK but peak 2 km too high

9. One month later Model grossly overestimates high cloud fraction To evaluate high clouds in models: need a high sensitivity lidar and appropriate sampling of data (both model and observations)

10. ECMWF cloud fraction Cabauw 2002: –Amount when present is good –Mean cloud fraction and frequency of occurrence too high in the boundary layer –Need to treat snow as cloud in the model Chilbolton 2004 (and all mid- latitude sites 2003-2005): –Boundary layer cloud fraction much more accurate –Still need to treat snow as cloud

11. ECMWF water content Mean LWC and IWC accurate to observational uncertainties Freq. of occurrence too high; amount when present too low Inconsistent with cloud frac.? PDF shows occurrence of low values is too high Chilbolton 2004: LWC Chilbolton 2004: IWC

12. RACMO Cloud fraction errors similar to ECMWF before 2003 Water content errors (mean, frequency of occurrency) much as ECMWF Lower IWC in high cirrus

13. Met Office mesoscale cloud fraction Mean amount when present too low through most of atmosphere Largely due to inability of model to simulate 100% cloud fraction, as shown by the PDFs Error in high cloud needs to be checked using high sensitivity lidar Cabauw 2004

14. Met Office global cloud fraction Observations show greater frequency of cloud with increased gridbox size; opposite in model PDF error unchanged Cabauw 2004

15. Met Office mesoscale water content Liquid occurrence very good Boundary layer perhaps too low Mean LWC underestimated above 3 km Similar to previous result found for occurrence of supercooled layers Chilbolton 2004: LWCChilbolton 2004: IWC Mean IWC very good Frequency of ice cloud occurrence too high above 3 km PDFs much better than ECMWF!

16. Met Office global water content Mean LWC similar but frequency of occurrence much lower IWC generally higher Chilbolton 2004: LWCChilbolton 2004: IWC

17. SMHI Rossby Centre model Amount when present reasonable but frequency of occurrence and overall mean much too high Similar picture for LWC/IWC: mean overestimated due to cloud too often Palaiseau 2004

18. Meteo France cloud fraction Before Apr 03 –Amount when present far too low –High values rarely predicted Cabauw 2002 After Apr 03 –Amount when present very good (better than Met Office & ECMWF) –Mean cloud fraction much better –Amazingly, worse agreement with synoptic obs of cloud cover! Cabauw 2004

19. Meteo Fr. water content Boundary-layer LWC too low Frequency of supercooled liquid much too high –Need to change the T-dependent ice/liquid ratio PDF of LWC and IWC too narrow Mean IWC too low in mid-levels Chilbolton 2004: IWC Chilbolton 2004: LWC

20. DWD cloud fraction Cloud fraction generally very good –But frequency of occurrence always overestimated by 20-30% PDFs particularly well simulated Chilbolton 2004

21. DWD water content Frequency of liquid cloud occurrence too high LWC in supercooled clouds too high Frequency of ice cloud occurrence OK Mean IWC and mean amount when present (in-cloud IWC) are both underestimated below 7 km Chilbolton 2004

22. Equitable threat score Measure of skill of forecasting cloud fraction>0.05 Persistence and climatology shown for comparison Lower skill in summer convective events

23. Skill versus lead time Unsurprisingly UK model most accurate in UK, German model most accurate in Germany! Typically 500-mb geopotential height used in operational forecast verification Cloud fraction a more challenging test: more rapid loss of skill with time