1. Bias Adjusted Precipitation Scores Fedor Mesinger NOAA/Environmental Modeling Center and Earth System Science Interdisciplinary Center (ESSIC), Univ. Maryland, College Park, MD VX-Intercompare Meeting Boulder, 20 February 2007

2. Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?

3. “The higher the value, the better the model skill is for the particular threshold” (a recent MWR paper)

4. Example: Three models, ETS, Bias, 12 months, “Western Nest” Is the green model loosing to red because of a bias penalty?

5. What can one do ?

6. BIAS NORMALIZED PRECIPITATION SCORES Fedor Mesinger 1 and Keith Brill 2 1 NCEP/EMC and UCAR, Camp Springs, MD 2 NCEP/HPC, Camp Springs, MD J12.6 17th Prob. Stat. Atmos. Sci.; 20th WAF/16th NWP ( Seattle AMS, Jan. ‘04)

7. Two methods of the adjustment for bias (“Normalized” not the best idea) 1.dHdF method: Assume incremental change in hits per incremental change in bias is proportional to the “unhit” area, O-H Objective : obtain ETS adjusted to unit bias, to show the model’s accuracy in placing precipitation ( The idea of the adjustment to unit bias to arrive at placement accuracy: Shuman 1980, NOAA/NWS Office Note) 2. Odds Ratio method: different objective

8. Forecast, Hits, and Observed (F, H, O) area, or number of model grid boxes:

9. dHdF method, assumption: can be solved; a function H (F) obtained that satisfies the three requirements:

10. Number of hits H -> 0 for F -> 0; The function H(F) satisfies the known value of H for the model’s F, the pair denoted by F b, H b, and, H(F) -> O as F increases

12. West Eta GFS NMM Bias adjusted eq. threats

13. A downside: if H b is close to F b, or to O, it can happen that dH/dF > 1 for F -> 0 Physically unrealistic ! Reasonableness requirement:

14. “dHdM” method: Assume as F is increased by dF, ratio of the infinitesimal increase in H, dH, and that in false alarms dM=dF-dH, is proportional to the yet unhit area:

15. One obtains ( Lambertw, or ProductLog in Mathematica, is the inverse function of )

16. H (F) now satisfies the additional requirement: dH/dF never > 1

17. H(F)H(F) H = OH = O H = FH = F Fb, HbFb, Hb dHdF method

18. H(F)H(F) H = OH = O H = FH = F Fb, HbFb, Hb dHdM method

19. Results for the two “focus cases”, dHdM method (Acknowledgements: John Halley Gotway, data; Du š an Jovi ć, code and plots)

20. 5/13 Case dHdM wrf2caps wrf4ncar wrf4ncep

21. 6/01 Case dHdM wrf2caps wrf4ncar wrf4ncep

22. Impact, in relative terms, for the two cases is small, because the biases of the three models are so similar !

23. One more case, for good measure:

24. 5/25 Case dHdM wrf2caps wrf4ncar wrf4ncep

25. Comment: Scores would have generally been higher had the verification been done on grid squares greater than ~4 km This would have amounted to a poor-person’s version of “fuzzy” methods !