Furthermore… References Katz, R.W. and A.H. Murphy (eds), 1997: Economic Value of Weather and Climate Forecasts. Cambridge University Press, Cambridge. Jolliffe, I.T., and D.B. Stephenson, 2003: Forecast Verification. A Practitioner's Guide in Atmospheric Science. Wiley and Sons Ltd, 240 pp. Click here to see the Table of Contents. von Storch, H. and F.W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, Cambridge. Wilks, D.S., 2006: Statistical Methods in the Atmospheric Sciences. An Introduction. Academic Press, San Diego, 467 pp.

Challenges in Spatial Forecasts Forecasts 1-4 have POD=0; FAR=1; CSI=0 Fifth forecast has POD>0, FAR 1 OF OF OFO F F O

Verifying Spatial Forecasts

Verifying Extreme Forecasts What are extremes? –Events large in magnitude –Events rare in occurrence Traditional skill scores become unstable as the probability of the event becomes increasingly small. Extreme value statistics. Different distributions are more appropriate for use with extreme events.

Confidence Intervals Skill scores are statistics and as such it is reasonable to ask for confidence intervals. Some techniques for estimating confidence intervals –Parametric assumptions and inference –Bootstrapping (re-sampling original data) –Use of holdout data in creating model Difficulties include –Dependent observations –Huge numbers of observation – everything is significant –Small number of data