Composite-based Verification Jason Nachamkin Naval Research Laboratory Monterey, CA References Nachamkin, J. E., 2004: Mesoscale verification using meteorological composites. Mon. Wea. Rev., 132, 941-955 Nachamkin, J. E., S. Chen, and J. S.Schmidt 2005: Evaluation of heavy precipitation forecasts using composite-based methods: A distributions-oriented approach. Mon. Wea. Rev.,133, 2163-2177.

Meteorological Fields Observations (gridded, point obs, swaths) SSMI winds RFC/ST4, BMRC rainfall Satellite rainfall estimates Storm reports? Forecast data Wind speed/direction Rainfall (resolved, parameterized, snow)

The Composite Method SSMI Winds Composite Forecast m s-1 27 km COAMPS®* wind speed (2001) Identify events of interest in the forecasts and observations Collect coordinated samples Compare forecast distribution to observed distribution *COAMPS® is a registered trademark of the Naval Research Laboratory

Quantifying the Results Given FC Event All Events ≥ 25 mm 24-hr FCST-OBS Bias (mm) mm Given OB Event 27 km COAMPS® rainfall (summer 2003) mm 810 x 810 km

Mistral Statistics Given an event is predicted All northwest winds ≥ 12.5 m s-1 Given an event is predicted Given an event is observed 27 km COAMPS® winds (2001) 810 x 810 km Dist of all known FCST events Dist of all known OBS events

Strengths Simplicity Flexibility Sampling and averages Minimal data manipulation Straightforward uncertainty calculations No dependence on field structure Flexibility Many data types accepted Multiple variables validated Database capable Probabilistic statistics applicable

Weaknesses Simplicity Dependent on event parameters Only general systematic biases No rotation/shape parameters Difficult to apply to single cases Dependent on event parameters Not good for very large, complex shapes (synoptic cloud fields) No funding

Case Study All Events ≥ 0.5 inches WRF2CAPS 13 May WRF4NCAR 13 May WRF4NCAR 1 June in×100 WRF4NCAR 1 June mm 400 x 400 km COAMPS® 27 km 400 x 400 km WRF2CAPS 13 May best overall WRF4NCAR 1 June missed forecasts High-res forecasts show improved ability to resolve more observed events at smaller scales