1. Mark T. Stoelinga University of Washington Thanks to: Steve Koch, NOAA/ESRL/GSD Brad Ferrier, NCEP Verification and Calibration of Simulated Reflectivity Products During DWFE

2. Hurricane WRF (Chen 2006, WRF Workshop)

3. 2006 NOAA/SPC Spring Program

4. 2005 DTC Winter Forecast Experiment (DWFE) (Koch et al. 2005) WRF-ARW SRWRF-ARW 3-h Precip Obs Composite Reflectivity WRF-ARW 700-hPa winds/RH

5. Variational Data Assimilation: Variational Data Assimilation: What is the best “forward operator” to use as a bridge between observed radar reflectivity and the model microphysics? Forecaster Testimonials “….(we) liked the 4 km BAMEX model run and DON’T want it to go away. The reflectivity forecasts were really very helpful, and almost uncanny.’’ - NWS Forecaster after BAMEX field study “Love the reflectivity product!” - NWS Forecaster after DWFE However,… “Before any meaning can be ascribed to the Reflectivity Product for the purpose of interpreting mesoscale model forecasts, it is important to understand how it is determined.” -Koch et al. (2005)

6. Study Goals Using archived forecast model runs and observed reflectivity from DWFE, examine Simulated Reflectivity (SR) from two different perspectives: 1.Use statistics and direct examination to see where and why different SR products resemble or differ from observed reflectivity. 2.Consider the question: If it can be shown that there is a systematic error in a particular SR product, such that the SR product consistently produces too much or too little of a given reflectivity value, can the SR product be “calibrated” to more closely match the observed radar reflectivity?

7. Data Sources Archived Gridded Forecast Model Output from DWFE Archived Observed and Simulated Composite Reflectivity Imagery 3-D Gridded Observed Reflectivity from the National Mosaic and Multi-Sensor Quantitative Precipitation Estimation (NMQ) Thanks to DTC Thanks to NSSL

8. Stratiform Area Convective/Stratiform Area 13 February 2005 Cyclonic Storm System

9. Observed NMM consistent ARW generic ARW consistent Stratiform Area: Composite Reflectivity

10. Observed NMM consistent ARW generic ARW consistent Stratiform Area: CFADs (Yuter and Houze 1995)

11. 0100020003000400050006000 -2 0 2 4 6 8 Stratiform Area: Frequency Distribution of Height of Maximum Reflectivity Height above Freezing Level (km) Number of Occurrences ARW generic NMM consistent ARW consistent Observed

12. Differences in ARW Reflectivity Products Real-time ARW post-processor used a “generic” SR that assumes a constant intercept parameter for the snow size distribution. “Consistent” ARW SR product uses T-dependent intercept, consistent with WSM5 microphysics in used in ARW. -50-45-40-35-30-25-20-15-10-50 10 6 7 8 9 00.511.522.533.544.55 10 -4 10 -2 10 0 2 4 6 8 N 0 (m -4 )N (m -4 ) Snow particle size distributions for same mixing ratio q s =0.1 g kg -1 Particle size (mm)Temperature (ºC)

13. Differences in ARW Reflectivity Products Real-time ARW post-processor used a “generic” SR that does not account for the change in dielectric factor for wet snow (“brightband”) “Consistent” ARW SR product uses the liquid-water dielectric factor for snow that is at T ≥ 0 ºC. → Increases reflectivity by ~7 dBZ in the melting layer

14. Differences in ARW Reflectivity Products ARW genericARW generic + var. N 0S (b) – (a) (a)(b)

15. Differences in ARW Reflectivity Products (b) – (a) (a)(b) ARW generic + var. N 0S ARW generic + var. N 0S + wet snow ( = ARW consistent)

16. Differences between NMM and ARW Reflectivity Products ARW genericNMM consistent (a)(b)

17. Stratiform Area: Composite Reflectivity Statistics Observed NMM consistent ARW generic ARW consistent

18. Stratiform Area: Composite Reflectivity Frequency Distributions -20-1001020304050 10 0 1 2 3 4 ARW generic NMM consistent ARW consistent Observed Number of grid boxes Reflectivity (dBZ)

19. Calibration of Composite Simulated Reflectivity Consider the question: If it can be shown that there is a systematic error in a particular SR product, such that the SR product consistently produces too much or too little of a given reflectivity value, can the SR product be “calibrated” to more closely match the observed radar reflectivity? How would we do this? Use the bias? No. SR is too high in some places, too low in others. Use correlation/linear regression? No. Forecast and observed precipitation are not spatially well-correlated. (Ebert and McBride 2000) How about matching the frequency distribution?

20. Calibration of Composite Simulated Reflectivity -20-1001020304050 10 0 1 2 3 4 ARW generic NMM consistent ARW consistent Observed Number of grid boxes Reflectivity (dBZ)

21. We seek a “calibration function” Z new = h(Z m ), such that where Z m is the composite SR, and f(Z) and g(Z) are the frequency distributions of the simulated and observed composite reflectivity, respectively. Calibration of Composite Simulated Reflectivity

22. While h(Zm) is difficult to extract mathematically, there is a practical and simple way to arrive at it: 1.Start with a set of SR values that will be used to obtain the calibration equation (e.g., all the grid values of composite SR in a single plot) 2.Rank all the values in order from lowest to highest value. 3.Do the same for the corresponding observed reflectivity set. It is important that the same number of points is used for both. 4.Align the two ranked sets (simulated and observed). The full set of pairs of reflectivity values provide the precise calibration function needed to transform the SR plot into one that has the exact same frequency distribution as the corresponding observed reflectivity plot.

23. Calibration of Composite Simulated Reflectivity Observed NMM consistent ARW generic ARW consistent

24. -20-10010203040506070 -20 -10 0 10 20 30 40 50 60 70 1-to-1 Calibrated Reflectivity (dBZ) Simulated Reflectivity (dBZ) Calibration Curves for Stratiform Area ARW generic NMM consistent ARW consistent

25. Uncalibrated Composite Simulated Reflectivity Observed NMM consistent ARW generic ARW consistent

26. Calibrated Composite Simulated Reflectivity (a)(b) (c)(d) Observed NMM consistent ARW generic ARW consistent

27. Stratiform Area 13 February 2005 Cyclonic Storm System Convective/Stratiform Area

28. Convective/Stratiform Area: Composite Reflectivity Observed NMM consistentARW generic ARW consistent

29. Convective/Stratiform Area CFADs Low observed frequency of 20-30dBZ echoes aloft (compared to all models) Observed NMM consistent ARW generic ARW consistent

30. 0 -2 0 2 4 6 8 Convective/Stratiform Area: Frequency Distribution of Height of Maximum Reflectivity Height above Freezing Level (km) Number of Occurrencess 50010001500 ARW generic NMM consistent ARW consistent Observed

31. Convective/Stratiform Area: Composite Reflectivity Frequency Distributions -20-1001020304050 10 0 1 2 3 4 ARW generic NMM consistent ARW consistent Observed Number of grid boxes Reflectivity (dBZ)

32. -20-10010203040506070 -20 -10 0 10 20 30 40 50 60 70 1-to-1 Calibrated Reflectivity (dBZ) Simulated Reflectivity (dBZ) Calibration Curves for Convective/Stratiform Area ARW generic NMM consistent ARW consistent

33. Uncalibrated Composite Simulated Reflectivity

34. Calibrated Composite Simulated Reflectivity

35. 4-Week Study of Calibration of Composite Simulated Reflectivity What about the mean behavior of the SR products over many different types and intensities of precipitation? 4-week study: 28 February – 24 March 2005 (sub-period of DWFE) Daily forecasts and observations of composite reflectivity at 18, 21, and 00 UTC (18, 21, and 21-h model forecasts) Area covering CONUS from Rocky Mountains eastward Used archived imagery – only 5 dBZ resolution (width of color bands)

36. 4-Week Study of Calibration of Composite Simulated Reflectivity

37. Frequency Distribution 010203040506070 -2 0 1 2 3 4 5 10 ARW generic NMM consistent ObservedNumber of pixels Reflectivity (dBZ)

38. 4-Week Study of Calibration of Composite Simulated Reflectivity -20-10010203040506070 -20 -10 0 10 20 30 40 50 60 70 1-to-1 Calibrated Reflectivity (dBZ) Simulated Reflectivity (dBZ) WRF-ARW (constant N 0 ) WRF-ARW WRF-NMM Calibration Curves

39. Caveats of SR Calibration 1.Calibration of SR will not significantly improve correlation of SR and observed reflectivity. 2.Calibration can only partially compensate for flaws in model microphysics or SR algorithm. 3.Calibration functions should be based on sufficiently large data sets such that they are not influenced by a small number of bad forecasts, i.e., they should reflect the mean behavior of the model. 4.Calibration functions are dependent on many factors, including: - observational data quality - method of “cartesianizing” the observed reflectivity - precipitation type - geographic location and time of year - model resolution, physics, and forecast hour

40. Merits of SR Calibration 1.Calibration can remove systematic under or overprediction of various reflectivity ranges and improve the “look” of SR products. 2.The process of determining the frequency distribution of SR vs. observed reflectivity, and deriving calibration functions, leads to insights into general flaws in model microphysics and SR algorithms. 3.Calibration functions may provide a more reasonable “forward operator” for assimilating observed reflectivity data into models than the straight D 6 function that is used. 4.There is potential to enhance the calibration functions, by training them on more limited spatio-temporal windows, or by seeking dependencies on particular types of frequency distributions.

41. Recommendations 1.Model microphysics should be formulated not only to optimize QPF, but also to produce reasonable hydrometeor fields and size distributions that affect the model reflectivity. 2.To the extent possible, SR algorithms should be precisely consistent with all assumptions in the associated model microphysical scheme. 3.Ideally, SR should be calculated within the model as it runs, to take advantage of the increasingly complex and dynamic size distributions calculated by the schemes. 4.Real-time or operational SR products should be statistically examined (using CFADs and other frequency distribution tests) to understand how they behave relative to observations. 5.Real-time or operational SR products should be calibrated with observed reflectivity using the methods described herein. 6.Calibration functions should be used in forward operators for assimilating reflectivity data into models.

42. Finis