1. Quantifying efficiency of homogenisation methods Dr. Peter Domonkos dopeter@t-online.hu COST HOME ES0601

2. Measuring efficiency our expectations Gaining the real climatic trends, Gaining the real trends and fluctuations, Identifying large inhomogeneity-shifts one-by-one, Identifying as many shifts as we can

3. Measuring efficiency general practice Usually the rate of correct detection is examined (Ducré- Robitaille, Mestre, Menne and Williams, etc.) Menne and Williams (2005) apply the hit rate (or power, = H), false detection rate (F), false alarm rate (FAR), bias of detection frequency (B), and the improvement in skill compared to random forecasts (HSS).

4. Measuring efficiency general practice

5. Measuring efficiency this presentation Arbitrary, but reasonable choices 1 = standard deviation of estimated noise Factual shift: Shift with M  M 0 magnitude between two adjacent 3 year long periods. M 0 = 2 or M 0 = 3 here. Right detection: A shift with M  1.5 for M 0 = 2 (M  2 for M 0 = 3) is detected with maximum 1 year lapse. False detection: A shift with M  1.5 for M 0 = 2 (M  2 for M 0 = 3) is detected at year j, but there is no shift of the same direction than the detected one with M > 0 within the (j-2,j+2) period.

6. Measuring efficiency this presentation Let the number of the time series be m, the total of the factual shifts is k, the number of right detections is D R, that of false detections is D F, then

7. Measuring efficiency this presentation Reliability of trends!? Let the mean bias of trend slopes, caused by inhomogeneities is t 0 before the homogenisation, and t after the homogenisation. Then the improvement in trend reliability is indicated by General (combined) efficiency (Domonkos, 5th Seminar, 2006)

8. Properties of time series Five versions of simulated datasets are examined here. Each dataset has 10,000, one hundred year long time series. The scale of the properties is wide from a single inhomogeneity per time series to the inclusion of very complex inhomogeneity-structures „Hungarian standard” (Domonkos, 5th Seminar, 2006). (1) 1 shift with M = 3; (2) 1 shift with M = 3 and 4 shifts with M = 1.5; (3) and (4) Shifts with 1/ decade frequency, exponential distribution of M above 1, and uniform distribution of M below 1. (3) M max <2; (4) M max <3; (5) Hungarian standard

9. Distribution of difference (percentage) between the detected inhomogeneity-properties of simulated and real climatic time series for HU STANDARD. k : simple, wk : weighted with sample size

10. Homogenisation methods 15 objective homogenisation methods: 2-2 versions of Bayes-test [Bay, Ba1], Buishand-test [Bu1, Bu2], SNHT [SNH, SNT] and t-test [tt1, tt2]; Caussinus-Mestre test [C-M], Easterling-Peterson test [E-P], Mann-Kendall test [M-K], MASH [MAS], Multiple Linear Regression [MLR], Pettitt-test [Pet] and Wilcoxon Rank Sum test [WRS].

11. Method parameterisation With original parameterisations the chance of detecting at least 1 inhomogeneity is ~5% in pure white noise. Minimum length of subperiods for calculating own statistical properties: usually 5 years, but in C-M and MAS 1 year, and in E-P 3 years. Outliers are prefiltered; Concerning multiple inhomogeneities the semihierarchic algorithm of Moberg and Alexandersson (1997) is included in Bay, Ba1, Bu1, Bu2, M-K, MLR, Pet, SNH, SNT and WRS. In a few experiments optimised parameterisation is applied (its use is indicated).

12. Red = C-M Blue = MASH Green = E-P Black = t- test (tt1) Brown = SNHT for shifts Lila = MLR

13. Identification A, 1 shift (M=3)

14. Identification A, 1 shift (M=3) + 4 small shifts

15. Identif. A of M  3, Exp. M<6

16. Identif.A of M  2, Hu standard

17. Identif.A of M  3, Hu standard

18. Identif.B of M  2, Exp. M<2

19. Identif.B of M  3, Exp. M<2

20. Absence of large shifts number of kinds: 7, best: tt1, C-M, Bay

21. Trends, 1 shift (M=3) filled columns = optimised parameters

22. Trends, 1 shift + 4 small shifts

23. Trends, Exp. M<2

24. Trends, Exp. M<6

25. Trends, Hu standard

26. Identification A, 1 shift

27. Identif.A, 1 shift + 4 small shifts

28. Identif.B of M  2, Exp. M<2

29. Identif.B of M  3, Exp. M<2

30. Identif.A of M  2, Hu standard

31. Identif.A of M  3, Hu standard

32. Identif.A of M  3, Exp. M<6

33. Discussion Identification of M>3 shifts is best with MASH, but its reproduction of climatic trends is not among the best results. This drawback of MASH can be reduced with parameter-optimisation. Many results with C-M are on the top, except for cases of very low rate of large inhomogeneities. If the evaluations of shorter than 3-year sections are excluded, and detection results with M 3 exceeds the performance of MASH.

34. Conclusions The efficiency-order of homogenisation methods strongly depends on the properties of time series, the purposes/priorities of the homogenisation, and on the way of the efficiency evaluation. Direct methods for identifying multiple inhomogeneities (C-M and MASH) usually perform better, than the other methods. When the avoidance of false detection has enhanced importance t-test and E- P methods are also competitive. Parameter-optimisation may yield improved results.

35. Thank you for your attention! COST HOME ES0601