Evaluation of the ability of Numerical Weather Prediction models run in support of IHOP to predict the evolution of Mesoscale Convective Systems Steve Koch, Linda Wharton, Andy Loughe Forecast Systems Laboratory Bill Gallus, Jeremy Grams Iowa State University Beth Ebert Australian Bureau of Meteorology

Background ( ) (Weisman, Skamarock, and Klemp, 1997: The resolution dependence of explicitly modeled convective systems, Mon. Wea. Rev., 125, ) 1.Results from 3D midlatitude squall-line simulations suggest that 4 km grid size is needed to reproduce much of the mesoscale structure and evolution of MCSs seen in 1-km simulations. 2.Evolution at resolutions coarser than 4 km is characteristically slower, due largely to a delayed strengthening of the cold pool. 3.Cold pool is crucial to the evolution of an MCS into an upshear-tilted mature system. 4.An overly strong mesoscale circulation and overprediction of precipitation results at resolutions coarser than 4 km. MCS: Mesoscale Convective System

Objectives of this Study 1.Determine forecast precipitation properties for each type of observed MCS for the 12-km NWP models run during IHOP (Eta, MM5, and WRF). Operational models in the U.S. use resolutions of km presently, not 4 km. 2.Traditional “measures-oriented” verification statistics (RMS, bias, etc.) severely penalize an incorrectly located precipitation system that may be forecast with only small positional or shape error, yet have practical forecast utility. We use an “object-oriented” verification technique in this study. 3.For each MCS type, we obtain systematic model performances by using the Ebert-McBride (2000) technique to determine the fractional contribution of forecast precipitation displacement, intensity, and shape errors. 4.The EM technique was implemented in the Real-Time Verification System (RTVS) at FSL and changes were made to the EM code as needed to make it applicable to mesoscale systems in the central U.S.

Ebert-McBride (EM) Verification Technique EM Technique takes maximum of observed and forecasted rain at all points and determines Contiguous Rain Areas (CRAs) exceeding specified isohyet Forecast is permitted to shift within expanded CRA box by user- defined amount, until either RMSE is minimized or correlation coefficient maximized OBS FCST CRA BOX Expanded CRA box

Application of the EM technique to 6h accumulated precipitation ending at 0600 UTC 13 June 2002

Strategy using Ebert-McBride technique Eta, WRF, and MM5 12-km runs from IHOP period were analyzed using the EM Contiguous Rain Area (CRA) technique Observed CRAs were assigned a morphology based on 2-km composite reflectivity radar images (30 minute time resolution) We classified the morphology of MM5 and WRF model convective systems using hourly reflectivity output from the models. Only six- hourly output was available for the Eta model. Required that an MCS meet the following criteria for at least 3 hours: 30 dBZ (~3 mm/h) over at least a 100 x 100 km area 40 dBZ (~13 mm/h) over at least a 50 x 50 km area

Improvements to EM CRA code 1. Percent of grid points allowed to shift off domain reduced from 50% to 0.1% for RMSE minimization and to 25% for correlation coefficient maximization (removes problem of displacements usually being off edge) 2. Plots changed to show entire expanded CRA region, with shifted forecast overlaid on observed rainfall chart 3. Correlation Coefficient maximization seems to produce more reasonable results than minimization of RMSE, but the error decomposition required development of a new decomposition (four terms) based on Murphy (1995) 4. Critical mass threshold for 24 hr periods was reduced by a factor of 4, allowing a greater number of smaller CRAs to be identified

Improvements to EM CRA code 5. Increasing threshold amount from.25 to.50 inch (13 mm) per 6 hours helps to identify distinct systems, but statistics are then computed over too small an area (so we kept the threshold at.25 inch) 6. Because models at best only resolve 6  x features, a Lanczos filter was introduced to filter observations – creating patterns more similar to that forecast in the models 7. Sensitivity tests of tuneable parameters such as expansion of CRA domain, and minimum size (grid points) of CRA/observed rainfall area were performed, but results did not indicate the need for changes 8. CRA statistics for error decomposition continue to be computed over union of observed, forecasted and shifted forecast of rain. Volume and average rain rate are determined just over appropriate portion of CRA

Linear o Linear (CL) o Linear Bowing (CLB) o Linear (CL & CLB) sub-classifications (Parker and Johnson 2000): Trailing Stratiform (TS), Leading Stratiform (LS), Parallel Stratiform (PS) o Squall Line Developmental Types (Bluestein and Jane 1985): Broken Areal (BA), Broken Line (BL), Backbuilding (BB), Embedded Areal (EA) Non-linear o Continuous Non-Linear (CNL) o Discontinuous Areal (DA) o Isolated Cells (IC) Orographically Fixed (OF) In the 6-hour CRA window, if multiple types were observed, then the type dominating most of the time was used. If multiple systems were observed in one CRA, then the system with greater temporal, spatial, and rain volume was used. MCS Morphology Classification

Number of Linear MCSs in Radar Data CL CLB TS LS PS TS_PS LS_PS BA BL BB EA

Number of Nonlinear MCSs in Radar Data

Rain Volume The version of the MM5 model (“Pre-MM5”) run in real-time during IHOP by FSL exhibited very large wet biases. These results compelled FSL to make major changes to the “Hot Start” diabatic initialization, both during the field phase and for several months thereafter. Those changes resulted in a removal of the bias for linear MCSs, but a slight wet bias was maintained with non-linear MCSs

Maximum Rainfall The post-MM5 overpredicts rainfall maxima for all MCS categories except CNL Conversely, the Eta underpredicts rainfall maxima for all categories except IC

Frequency Classification of MCSs MCS TypeRadar (%)Eta (%)MM5 (%)WRF (%) CNL DA IC10431 Total Nonlinear CL CLB5598 Total Linear

Distribution of Forecast Rainfall Errors by Error Type for the Five Basic MCS categories

Displacement Errors: ETA

Displacement Errors: MM5

Displacement Errors: WRF

Conclusions Displacement vectors (polar plots): oNone of the models displays a strongly preferred (or systematic) direction and magnitude of displacement vectors, either for any particular MCS or between MCS classes, except for the linear CL and CLB types, which were forecast too slowly (north or northwest) by the WRF and MM5 models oThis may suggest cold pools for squall line systems were forecast to be too weak, which if true is consistent with the idealized study of squall lines by Weisman et al. (1997) Decomposition of rainfall errors (histograms): oOverall, for all three models and all MCS types, the largest contributors to total MSE are conditional bias and pattern errors, with volume error consistently being smallest oThis was a major problem for CLB type, suggesting poorly forecast bowing structures and with the wrong intensity for the convective lines oNonlinear systems (CNL and DA) were forecast with the least error, though all three models did display a significant bias for these systems