Intensity-scale verification technique B. Casati, G. Ross and D.B. Stephenson (2004) “A New intensity-scale verification approach for the verification of spatial precipitation forecasts”, Meteorol Appl, vol 11, 141-154 pp Evaluate the forecast skill as a function of the precipitation intensity and the spatial scale of the error NOTE: scale = single band spatial filter  features of different scales  feedback on different physical processes and model parameterizations In the neighborhood based (fuzzy) verification, the scale is the neighborhood size (low band pass filter): as the scale increases the exact positioning requirements are more and more relaxed

Nimrod case study: intense storm displaced Gridded data, square domain with dimension 2n It can be applied to any meteorological field … however, it was specifically designed for spatial precipitation forecasts …

Intensity: threshold to obtain binary images (categorical approach) Binary Analysis Binary Error Image u=1mm/h 1 -1 Binary Forecast

Scale: wavelet decomposition of the binary images Scale l=8 (640 km) Scale l=1 (5 km) mean (1280 km) Scale l=6 (160 km) Scale l=7 (320 km) Scale l=5 (80 km) Scale l=4 (40 km) Scale l=3 (20 km) Scale l=2 (10 km) 1 -1

Intensity-scale skill score For each threshold and scale component: skill score associated to the MSE of binary images ( = HSS) Skill versus random chance, equally partitioned across the scales 1 -1 -2 -3 -4 SSu,l

Links with categorical verification Binary Analysis Binary Rec.Forecast Overlapping X > u X < u Y > u Hits a False Alarms b a+b Y < u Misses c Correct Rejections d c+d a+c b+d a+b+c+d=n

Strenghts Categorical approach  robust and resistant Wavelets  cope with spatially discontinuous fields characterized by the presence of few sparse non-zero features  suitable for spatial precipitation forecasts Weaknesses need gridded data on a square domain with dimension 2n : work in progress …

Intensity-scale verification technique summary evaluate the skill as function of precipitation intensity and spatial scale of the error it is capable of isolating specific IS errors (e.g. displaced storm  negative minimum in skill for 160 km scale) in general small scales have negative skill (small scale displacements) and large scales have positive skill bridges categorical approaches (joint distribution) and scale verification approaches (physical properties) is suitable for spatial precipitation forecasts: wavelets (discontinuities and features) + categorical approach (robust and resistant)

Spring 2005, 13 May case

Intensity-Scale Skill Score

Future work Confidence intervals, p-values Under-sampling Energy and bias on each scale Random chance can be partitioned across the scales in proportion of magnitude and number of events characterizing the scale … THANK YOU !

Intense storm displaced Intensity-scale verification technique Casati et al. (2004), Met App, vol. 11 The intensity-scale verification approach measures the skill as function of precipitation intensity and spatial scale of the error intensity: threshold  binary images (categorical approach) scale: 2D Wavelets decomposition of binary images For each threshold and scale: skill score associated to the MSE of binary images = Heidke Skill Score Skill threshold (mm/h) 1 -1 -2 -3 -4 640 320 160 80 40 20 10 5 scale (km) 0 1/16 ¼ ½ 1 2 4 8 16 32 Intense storm displaced threshold = 1mm/h Casati et al. introduces an intensity-scale verification technique which measure the skill a a function of the precipitation intensity and on the spatial scale of the error. Forecast and observation fields are transformed into binary images by thresholding for different precipitation rates. A 2D wavelet decomposition of the error image, obtained as the difference between forecast and observation fields, is used to separate different scale components. A skill score, based on the MSE of binary images and related to the HSS, is then evaluated for each scale component and precipitation threshold. Note that the technique links categorical approaches with scale-verification approaches. Note that the technique uses a categorical approach (robust and resistant, therefore suitable for precipitation). Moreover uses wavelets, which cope well with discontinuous on and off fields such as precipitation. Figure example is an intense storm badly advected so that it is displaced of almost entirely its length. The displacement is well detect by a minimum negative skill corresponding at the 160 km scale, for intensities of 0.5 to 4 mm/h.