1. Composite-based Verification
Jason Nachamkin
Naval Research Laboratory
Monterey, CA
References
Nachamkin, J. E., 2004: Mesoscale verification using meteorological composites. Mon. Wea. Rev., 132,
Nachamkin, J. E., S. Chen, and J. S.Schmidt 2005: Evaluation of heavy precipitation forecasts using composite-based methods: A distributions-oriented approach. Mon. Wea. Rev.,133,

2. Meteorological Fields
Observations (gridded, point obs, swaths)
SSMI winds
RFC/ST4, BMRC rainfall
Satellite rainfall estimates
Storm reports?
Forecast data
Wind speed/direction
Rainfall (resolved, parameterized, snow)

3. The Composite Method
SSMI Winds
Composite
Forecast
m s-1
27 km COAMPS®* wind speed (2001)
Identify events of interest in the forecasts and observations
Collect coordinated samples
Compare forecast distribution to observed distribution
*COAMPS® is a registered trademark of the Naval Research Laboratory

4. Quantifying the Results
Given FC Event
All Events ≥ 25 mm
24-hr FCST-OBS Bias (mm) mm
Given OB Event
27 km COAMPS® rainfall (summer 2003) mm
810 x 810 km

5. Mistral Statistics Given an event is predicted
All northwest winds ≥ 12.5 m s-1
Given an event is predicted
Given an event is observed
27 km COAMPS® winds (2001)
810 x 810 km
Dist of all known FCST events
Dist of all known OBS events

6. Strengths Simplicity Flexibility Sampling and averages
Minimal data manipulation
Straightforward uncertainty calculations
No dependence on field structure
Flexibility
Many data types accepted
Multiple variables validated
Database capable
Probabilistic statistics applicable

7. Weaknesses Simplicity Dependent on event parameters
Only general systematic biases
No rotation/shape parameters
Difficult to apply to single cases
Dependent on event parameters
Not good for very large, complex shapes (synoptic cloud fields)
No funding

8. Case Study All Events ≥ 0.5 inches
WRF2CAPS 13 May
WRF4NCAR 13 May
WRF4NCAR 1 June
in×100
WRF4NCAR 1 June mm
400 x 400 km
COAMPS® 27 km
400 x 400 km
WRF2CAPS 13 May best overall
WRF4NCAR 1 June missed forecasts
High-res forecasts show improved ability to resolve more observed events at smaller scales