1. Adaption of an entity based verification
method for short-range cloud forecasts
utilizing satellite data
Christoph Zingerle, Pertti Nurmi
Toulouse, Sep. 2005
WSN05
High resolution model and observation
Known problems: bad scoes but still useful forecasts
Source of error
– error decomposition to make verification more meaningful
- only then we can start to improve model (e.g. parameterion development)
e.g phase correct but amount wrong – good dynamics
amount good but phase wrong – good physics
error is combination of errors
Idea: adapt a tool sucessfuly applied (CRA Beth) to other data
(in our case satellite data)

2. Outline - Satellite data in verification - Forecast ‘treatment’
- Case
- Conclusion / Outlook
How do we use satellite data in verification, which channels?
What information do satellite data provide for verification?
How to fit the Forecast to the Observation / make them comparable?
WSN05, Toulouse, Sep. 2005

3. Satellite data in verification
- High resolution (temporal and spatial)
- 'No' gaps (national bounderies, sea – land ...)
- Variety of parameters / channels
- Limits?
Different scale: Model GRID – Observation Point (representativeness?)
Satellites: high density of points – representative for a wide number of scales
resampling schemes are simple when comparing fields.
National bounderies are no problem (same measurement device, sampling time, observation density over sea, land, …)
Different wavelengths – different property of the atmosphere (we are looking only at one of them – clouds (IR Tb – Surface / cloud temperature)
Geostat don’t see at far north/south (or viewing angle is to high to get quantitative measurements), Polar orbiters don’t have the high temporal resol.)
WSN05, Toulouse, Sep. 2005

4. after re-sampling, 30.4.2004 (10.8µm)
full resolution, (10.8µm)
after re-sampling, (10.8µm)
WSN05, Toulouse, Sep. 2005

5. Forecast RTTOV-8 - Simulates clear/cloudy multilevel infrared
temperature
humidity
cloud fraction
cloud liquid water
cloud ice water
surface data (p, T, q)
RTTOV-8
- Simulates clear/cloudy multilevel infrared
and microwave radiances
- Consistent random-overlap scheme for
clouds in different levels (as Hirlam)
- Multiphase cloud field: water / ice / mixed,
crystal size distribution / aggregates
RTM transfers the given atmosperhic state (from the model) to what the
Satellite would see.
Done for each profile (grid collumn) – synthetic satellite image.
RTM is very sensitive to cloud fraction and water path – we see effect of
Badly represented cf and cwat in the model!
WSN05, Toulouse, Sep. 2005

6. Verification case 15. – 20. July 2003 Classical methodology:
Daily cycle of error measures (here RMS err.) maximum with convection active
Visible inspection, difference plots, scatterplots for each image and we find out
That model has predicted the system at the right place but not intense
Enough.
Model produces not enough cf and cwat in upper levels – problem with entrain
Ment – detrainment in the convection scheme
Over all, forecast was useful!
Also from modellers perspective: want to know what is the problem: is our
Cloud sceme working correct in case of convection / large scale systems
Physics / dynamics. – error decomposition is answering these questions
- Big errors when
convection is active
- No error-decompositon
only visible inspection
WSN05, Toulouse, Sep. 2005

7. Adapting CRA to CCA
- Entity based error decompostion method introduced
by Ebert & McBride (2000)
- Entities defined by simple thresholding method
- Forecast moved around in the domain to search
for best fit
- Estimates dispalcement, volume (intensity) and pattern
error
Tool that has been spread all over in the verification community
Thanks to Beth, for giving the code to us
Simple thresholding method – we use very simple threshold of 3C for clouds
Not a perfect one but reasonable if we want to catch precipitating clouds
(convection, large scale)
Best fit (miniming RMS or maximizing corr.coeff)
Volume error in our case is the mean brightness temperature * nr of pixels
WSN05, Toulouse, Sep. 2005

8. Different kinds of scores.
See only the areas of fc and obs that show values lower than threshold – so where we
Assume clouds.
View of the whole domain, see the cloud entities (BT), get all contingency table and the
Different kinds of scores.
To high BT produced by the model, in a short range forecast – kind of a spin up problem!
We can take a closer look to the contiguous cloud areas in the observation and fc.
WSN05, Toulouse, Sep. 2005

9. Convection at almost the right place.
Chose another lead time (longer, 30h) for the same verification time. Produced at least some
Convection at almost the right place.
Algorythm moves the center of the fc-cell by this vector indicating a location error
Measure for the displacement error – correlation coeff. Not significant.
scatterplot showes underforecasting – see this in the volume error
Forecst is moved around searching for the best match with observation
WSN05, Toulouse, Sep. 2005

10. CCA 1: synoptic system north
Most importan, we can keep the scores for the moved and unmoved forecast/observation pairs –
Here ist the CCA of the synoptic system at the north of the the domain.
Problem of that ccas at the domain boarder is easily moved out of the domain – it imporves the RMS
See this also in the right hand side – when system moves into the domain , the displacement
Error gets less important and is going down to 0!!
- RMSE reduced when CCA's
are shifted
- Error decomposition helps dis-
tinguish major reason for bad
scores.
WSN05, Toulouse, Sep. 2005

11. Conclusions - Knowledge about the error components is essential to
support the improvement of cloud schemes in a
short range forecasting model
- Satellite data enables verification of 'integral model collumn'
- BT to simple? Cloud mask from cloud classification schemes?
- Tool for early assessment of forecast quality and detection
of regions with problems?
WSN05, Toulouse, Sep. 2005

12. Outlook
WSN05, Toulouse, Sep. 2005