Bias Adjusted Precipitation Scores Fedor Mesinger NOAA/Environmental Modeling Center and Earth System Science Interdisciplinary Center (ESSIC), Univ. Maryland, College Park, MD VX-Intercompare Meeting Boulder, 20 February 2007

Most popular “traditional statistics”: ETS, Bias Problem: what does the ETS tell us ?

“The higher the value, the better the model skill is for the particular threshold” (a recent MWR paper)

Example: Three models, ETS, Bias, 12 months, “Western Nest” Is the green model loosing to red because of a bias penalty?

What can one do ?

BIAS NORMALIZED PRECIPITATION SCORES Fedor Mesinger 1 and Keith Brill 2 1 NCEP/EMC and UCAR, Camp Springs, MD 2 NCEP/HPC, Camp Springs, MD J th Prob. Stat. Atmos. Sci.; 20th WAF/16th NWP ( Seattle AMS, Jan. ‘04)

Two methods of the adjustment for bias (“Normalized” not the best idea) 1.dHdF method: Assume incremental change in hits per incremental change in bias is proportional to the “unhit” area, O-H Objective : obtain ETS adjusted to unit bias, to show the model’s accuracy in placing precipitation ( The idea of the adjustment to unit bias to arrive at placement accuracy: Shuman 1980, NOAA/NWS Office Note) 2. Odds Ratio method: different objective

Forecast, Hits, and Observed (F, H, O) area, or number of model grid boxes:

dHdF method, assumption: can be solved; a function H (F) obtained that satisfies the three requirements:

Number of hits H -> 0 for F -> 0; The function H(F) satisfies the known value of H for the model’s F, the pair denoted by F b, H b, and, H(F) -> O as F increases

West Eta GFS NMM Bias adjusted eq. threats

A downside: if H b is close to F b, or to O, it can happen that dH/dF > 1 for F -> 0 Physically unrealistic ! Reasonableness requirement:

“dHdM” method: Assume as F is increased by dF, ratio of the infinitesimal increase in H, dH, and that in false alarms dM=dF-dH, is proportional to the yet unhit area:

One obtains ( Lambertw, or ProductLog in Mathematica, is the inverse function of )

H (F) now satisfies the additional requirement: dH/dF never > 1

H(F)H(F) H = OH = O H = FH = F Fb, HbFb, Hb dHdF method

H(F)H(F) H = OH = O H = FH = F Fb, HbFb, Hb dHdM method

Results for the two “focus cases”, dHdM method (Acknowledgements: John Halley Gotway, data; Du š an Jovi ć, code and plots)

5/13 Case dHdM wrf2caps wrf4ncar wrf4ncep

6/01 Case dHdM wrf2caps wrf4ncar wrf4ncep

Impact, in relative terms, for the two cases is small, because the biases of the three models are so similar !

One more case, for good measure:

5/25 Case dHdM wrf2caps wrf4ncar wrf4ncep

Comment: Scores would have generally been higher had the verification been done on grid squares greater than ~4 km This would have amounted to a poor-person’s version of “fuzzy” methods !