1. QPF ISSUES IN NWP William A. Gallus, Jr. Dept. of Geological & Atmospheric Science Iowa State University

2. ETA MM5 24HR PRECIPITATION 4/00-5/00 OBS PREC:3/12-4/12OBS PREC:4/12-5/12

3. ETA MM5 48 HR PRECIPITATION (4/00-6/00) OBS:3/12-4/12OBS:4/12-5/12 OBS:5/12-6/12

4. What is “TRUTH” for QPF verification?

5. Increased computer resources have allowed better parameterization schemes and model resolution 2-day precipitation forecast today is now as accurate as 1-day forecast in 1974 Each resolution improvement in NCEP Eta model improves skill scores GOOD NEWS: QPF is improving!!

6. MRF has some skill compared to persistence, even out to 7-8 days: Roads et al. 1991 (WAF)

7. This skill is even more apparent for heavy rainfall cases

8. BAD NEWS: Problems abound Most improvement in QPF scores occurs during cold season - little improvement in warm season Flash flooding kills more people than any other convective-related event QPF problems have several potential sources Skill scores themselves may be misleading or of little “real” value

9. Roads and Maisel 1991 WAF: MRF has regional biases in precipitation over long periods

10. Example of human improvements on numerical QPF (Olson et al. 1995, WAF) NGM Manual OBS

11. Slow improvement in skill for human forecasters, but less skill for heavier amounts (Olson et al. 1995, WAF)

12. Annual bias has also improved slowly, but interestingly, is better for Day 2 than Day 1 (Olson et al. 1995, WAF)

13. QPF skill is better is winter than in summer, even when forecasters adjust the NWP guidance

14. What are sources of QPF error? Resolution inadequacies Parameterization errors Initialization deficiencies Observational errors in verification

15. If vertical motion is directly constrained by horizontal resolution….. Shouldn’t forecasts for heavy rain events be greatly improved with finer resolution? Is there a “magic” resolution where model QPF will approach observed peak values

16. Gallus 1999 found QPF-horizontal resolution dependence is case-dependent and varies with convective parameterization 6/16/96 6/14/987/28/97 7/17/96 5/27/97 BMJ -shaded KF - clear Mx obs: 225Mx obs: 330 Mx obs: 250 Mx obs: 300 Mx obs: 102

17. Extreme example of unexpected results and Conv. Param. Impacts: 7/17/96 00UTC surface conditions

18. 00 UTC 17 JUL 1996 - OMAHA Betts-Miller- Janjic Reference T, Td profiles shown

19. Large MCS drops up to 300 mm of rain, causing record river crests and severe flash flooding in far eastern NE and western IA.

20. 7/17/96 BMJ simulations with 78,39,22 and 12 km horizontal resolution NOTE: actual reduction in peak QPF amounts as resolution improves MX: 46MX: 45 MX: 32

21. 7/17/96 KF simulations: NOTE: very strong QPF sensitivity to horizontal resolution. Precipitation area shifted much farther north than in BMJ runs, or observations MX: 11 MX: 70 MX: 135 MX: 186

22. Daytime precipitation (12-00 UTC 7/16-17/96) BMJ produces much larger area and amounts

23. BMJ KF Convective scheme influences cold pool strength, which in turn, affects evolution of events outside initial rain region

24. Impacts of convective schemes may be felt outside region of precipitation. Here, stronger downdrafts in KF scheme result in greater northward transport of instability into Minnesota - leading to more intense subsequent development. BMJ KF

25. Another case: Iowa flood of June 1996 Large-scale region looked favorable for excessive rains Heaviest rains (225 mm) fell in small area in warm sector Impacts of horizontal resolution changes strongly depend on convective scheme used

26. Tropical-like soundings with very deep moisture Td at 850 mb = 18 C Td at 700 mb = 8 C

27. BMJ simulations: Almost no horizontal resolution-QPF dependence No hint of C IA maximum

28. 12 UTC 6/16 cold pool affecting Iowa

29. 12 UTC 6/16 Eta model 00 hr - initialization NOTE: cold pool is missing: winds are southerly, without E component

30. 21 UTC 6/16 Observed Surface Moisture Convergence Flood-producing storms would form on C IA enhancement

31. Simulated Moisture Convergence -21 UTC - BMJ run with 12 km resolution Despite poor initial wind field, model does show enhancement in W IA

32. BMJ simulation: No general clearing into Iowa by 1 pm - Less destabilization than actually occurred

33. KF simulations: Strong horizontal resolution-QPF dependence Some evidence of C IA enhancement with 22 and 12 km resolution

34. KF 6 hr forecast: Some clearing into SW Iowa more agreement with obs.

35. June case shows: Moist low-mid troposphere allows BMJ scheme to be aggressive Even high resolution may not improve simulation of small QPF maxima if other simulated parameters are incorrect Generation of QPF upstream due to resolution changes may affect QPF downstream

36. Changes within a specific convective parameterization can also have a very pronounced effect on QPF Spencer and Stensrud (1998) show this using MM4 with KF scheme

37. Spencer & Stensrud variations in KF scheme Permit Precipitation Efficiency to remain at maximum (90%) instead of varying from 10-90% Neglect convective downdrafts Delay convective downdrafts

38. Max. Prec for 4 tests Maximum QPF in 4 KF MM4 runs From Spencer and Stensrud 1998 - MWR

39. Microphysicalschemes may be the next challenge - once resolution improves so that convective parameterization is no longer necessary Microphysical schemes may be the next challenge - once resolution improves so that convective parameterization is no longer necessary Colle and Mass examine resolution- orographic precipitation (1999) dependence Microphysical schemes influence results

40. OBS PRECIP IN PACIFIC NORTHWEST FLOOD EVENT (1996) from Colle and Mass (1999; MWR) Pronounced orographic effects

41. 4 km MM5 run does well at crest but underestimates lee precipitation

42. Horizontal resolution affects precipitation patterns near mountain due to resolution of mountain wave effects. Model QPF performance in lee of mountain fluctuates - low bias is best in coarsest run, but heaviest precipitation just to lee of crest occurs with highest resolution 1.33 4 12 36

43. Although precipitation forecasts generally improved as resolution was refined from 36 to 4km, little additional improvement occurred with 1.3 km resolution (Colle & Mass)

44. Model QPF in relation to resolution of topography

45. Microphysical schemes may have significant influences at high resolution. Colle and Mass (1999; MWR) found that lee- side precipitation was too small in high-res MM5 simulations, partly because snow fallspeeds were too large.

46. Best results may not occur with most sophisticated microphysical scheme

47. Microphysical scheme differences affect QPF in different areas

48. Mesoscale initialization may be poor and affect QPF Stensrud and Fritsch (1994) have shown the impacts of improved cold pool initialization

49. Typical initialization deficiencies Low-level jet characteristics Cloud boundaries Fronts and drylines Convective outflows Surface characteristics

50. Stensrud and Fritsch 1994 MWR: Initialization of NE KS mesoscale boundary has important impact on QPF MM4 -25KM

51. How do we verify QPF? Bias scores (how many grid points have X amount of rain compared to observations) Threat Scores (area correct/(area forecast+ area observed - area correct)) Probability of Detection

52. PRIMARY VERIFICATION TOOLS TODAY USED BY NCEP FOR QPF ARE: BIAS: Number of grid points having simulated rain of X amount divided by number of observed points with X amount EQUITABLE THREAT SCORE: Ratio of correct forecasts (hits) to total forecasts + observations - hits (with correction for chance hits)

53. BIAS B=F/O Can vary from 0 to >> 1 Bias > 1 means the model is generous with areal coverage of precipitation Bias < 1 means the model doesn’t generate enough areas with precipitation Many operational models have B>1 for small precipitation amounts, and B<1 for large amounts

54. ETS ETS=(H-C)/[F+O-(H+C)] 0<ETS<1 Similar to a Threat Score but takes into account that even “chance” forecasts will be correct some of the time (Schaefer 1990; Gandin and Murphy 1992)

55. 1995-1997 ETS AT NCEP (Mesinger 1998).01.10.25.50.75 1.00 1.50 2.00 29ETA 48ETA MRF NGM

56. How valuable are these verification methods? Model A covers your state with 1inch of rain Model B simply produces 5 inches in the one county to your east A lone supercell drops 5 inches on your county Which model had the better forecast?

57. What are our Bias and ETS? For measurable precip (or any category less than 1 inch): Assume one grid point per county with 100 counties in state Bias in model A: 100/1 = 100.0 Bias in model B: 1/1 = 1.00 ETS in A: 1/(100+1-1) = 0.01 ETS in B: 0/(1+1-0)= 0.0

58. Objective scores may not agree with your answer! Improved mesoscale QPF verification may involve a phase shift of the simulated precipitation field. Kalnay and others (1999) are studying such an approach

59. Concluding Thoughts QPF is probably the most difficult aspect of NWP - the hardest one to envision being solved in 25 years If convective parameterizations are used, behavior of these schemes exerts powerful impact (primary differences between different models are probably related to the Cu scheme) Thus, forecasters can benefit by understanding the specifics of how the schemes behave

60. Concluding Thoughts (Cont.) At very high resolutions, microphysics will likewise complicate the picture Forecasters need to be aware of small-scale boundaries of importance, which will most likely be poorly depicted in initialization New methods of evaluating what is a “good” QPF will be needed