Evaluation of a Challenging Warm Season QPF month at HPC: June 2009 Brendon Rubin-Oster Richard Otto (with contributions from Mike Bodner, Keith Brill, David Novak and HPC management) June 3,
Motivation For This Review HPC 1” Threat Score was for June 2009 Lowest since 1999 (0.119) FY 2009 goal is NAM threat score of is lowest since at least 1998 GFS threat score of tied its lowest in 10 years (1999) HPC % Improvement over models: 18% NAM, 43% GFS, 16% ECMWF
(2009) 3
Seek To Answer the Following Was there a common theme associated with the busted forecasts? (MCS activity, scattered convection, synoptic scale forcing, mesoscale boundaries, etc.) What kinds of errors were observed? (timing/duration, placement, magnitude, etc.) How did the models perform? What (if anything) was so different about June 2009? 4 What were the causes of these errors?
Data & Methodology Selected model data archived at HPC for later review deterministic (NAM, GFS, ECMWF, CMC, UKMET) ensembles (SREF, CMCE, ECENS) 2 km base reflectivity radar composite archive (NMAPRAD) VIS/IR satellite & analysis data from internet (SPC archives, UCAR case studies) NCEP/NCAR reanalysis for monthly means Limitations on available data Surface: winds, moisture convergence, T, T d, θ e, moisture transport, 1000 mb frontogenesis, mixing ratio Thickness: mb & mb Instability: CAPE, lifted indices Miscellaneous: Q-vectors, Corfidi vectors And others… 5
6
What (if anything) was so different about June 2009? 7
Source: 500mb Geopotential Height means (m) 8
250 mb mean zonal winds (m/s): June 1-30 Source: 9
Observed Precipitation (in) (June 1-30) Source: 10
Percent of Normal Precipitation (June 1-30) Source: 11
Was there a common theme associated with the busted forecasts?
Defining Modes of Convection ● Mesoscale Convective System (MCS): organized cluster of thunderstorms resides significant distance from synoptic boundaries persists for at least 3 to 6 hours
Defining Modes of Convection ● Convection along a synoptic scale boundary: thunderstorms developing within 150 km of a synoptic boundary
Defining Modes of Convection ● Convection along a mesoscale boundary: thunderstorms developing within 150 km of a mesoscale surface boundary consists of: sea breezes, surface trofs, outflow boundaries
Defining Modes of Convection ● Scattered convection: thunderstorms not appearing to be associated with synoptic/mesoscale boundaries does not meet MCS guidelines
Was there a common theme associated with the busted forecasts? 50% 22% 7% 14% 7%
What kinds of errors were observed? 29% 43% 7% 21%
What were the causes of these errors? 14% 22% 14% 22% 14%
How did the models perform during these 14 cases? 20 RankPerformerThreat Score 1NAM HPC GFS ECMWF0.026 RankPerformerBias 1NAM ECMWF GFS HPC2.810 Source:
45mm / 1.78” PW June 26, 2009 case study
OBSERVEDNAM f036AUTO f036 HPC f036GFS f036EC f036
24-hr radar loop {June 25 (1200Z)- June 26 (1200Z)}
Satellite loop June 25 (1415Z-2315Z)
Final Notes / Future Work? Explore how atypical the June 2009 observed phenomena and forecast performance were relative to other June cases HPC is currently applying MODE (method for object-based diagnostic evaluation) in real-time Use object oriented verification to objectively quantify errors Compare performance of high resolution mesoscale models