Correction of spurious trends in climate series caused by inhomogeneities Ralf Lindau

Dipdoc Seminar – 12. November 2015 Break detection Consider the differences of one station compared to a neighbor reference from a surrounding network of stations. The dominating natural variance is cancelled out, because it is very similar at both stations. Breaks become visible by abrupt changes in the station-reference time series. Internal variance (Noise) within the subperiods External variance (Signal) between the means of different subperiods Break criterion: Maximum external (explained) variance

Break-aware idea Breaks are only detected, but not corrected. Calculate the mean trend over all homogeneous subperiods (omitting the known breakpoints). This trend should reflect the true trend. Dipdoc Seminar – 12. November 2015 Is not promising, because:

Correction is important Dipdoc Seminar – 12. November 2015

Station or network trend? From my point of view, the network-mean (regionally averaged) trend is per se more interesting. Moreover, trend corrections for individual stations are easy to derive, if the network-mean trend is known. Therefore, we concentrate on corrections of the network-mean trend. Dipdoc Seminar – 12. November 2015

Network-mean trend error Dipdoc Seminar – 12. November 2015 Observed = True + Spurious station trend Observed trend anomaly = Spurious station trend against the network-mean– network-mean trend error trend

Two ways of correction 1.Break-by-break method: Consider an individual break of one candidate station. Compare the two homogeneous subperiods before and after this break with homogeneous subperiods of suited neighbor stations, which preferably long overlap periods with the candidate break. 2.ANOVA method: Minimize the variance of the entire network. Discussed in the following, because it is better defined. Dipdoc Seminar – 16. June 2014

ANOVA correction scheme (1/3) Observation b (station, year) Climate signal c (year) Inhomogeneity a (station, year) Noise  station, year) Minimize the squared difference between theory and observations. Derivation with respect to c(j) leads to m equations, one for each c(j). Dipdoc Seminar – 12. November 2015

ANOVA correction scheme (2/3) Analogously, we get h equations, one for each of the homogeneous sub-periods. Altogether we have m+h equations for m+h unknowns. Thus a (m+h) x (m+h) matrix has to be solved. E.g. 115x115-matrix for 100 years and 15 subperiods. We can insert the m climate equations for each year into the h equations for each subperiod, so that only a hxh-matrix has to solved. Dipdoc Seminar – 12. November 2015

ANOVA correction scheme (3/3) n = 5 stations with data from 100 years. h = 15 homogeneous sub-periods in total. Consider sub-period no. 8: It has an overlap of 32 years with a 2, of 13 years with a 4, of 19 years with a 5, etc. The overlap periods constitutes the non- diagonal matrix elements. The diagonal is given by (n-1) length(a i ) Dipdoc Seminar – 12. November 2015

Simulated data Test the performance of the ANOVA correction scheme with simulated data. The simulated data consist of three superimposed signals: 1.The climate signal, which identical for all stations of a network. 2.Noise, which mimics the difference between the stations, e.g. due to weather. 3.Inhomogeneities inserted at random timings and with random strengths. Dipdoc Seminar – 12. November 2015

Test under perfect conditions 1000 networks 10 stations with 5 breaks each No mean trend error No noise Known break positions Result: Perfect skill. - We made no programming errors. - The method works perfectly. Dipdoc Seminar – 19. November 2015 Inserted yearly station inhomogeneity Detected yearly station inhomogeneity

Signal / noise = 1 Still: No mean trend error Perfectly known break positions Equal break and noise variance: SNR = 1 Result: As expected no longer perfect, but r = Dipdoc Seminar – 19. November 2015 Inserted yearly station inhomogeneity Detected yearly station inhomogeneity

Network-mean trends From individual inhomogeneities to network-mean trends. Both regressions are calculated. That taking the x-axis data as independent is in all three cases equal to the 1-to-1 line. What does this mean? Dipdoc Seminar – 19. November 2015 Inhomogeneities Station trends Network trends r = r = r =

What does it mean? Dipdoc Seminar – 12. November 2015

Remaining trend error It is convenient to display not the detected (y), but the remaining (y-x) trend error. As shown the inserted and the remaining quantities are uncorrelated. The remaining errors are smaller, but comparable in size. This is valid for SNR = 1 Dipdoc Seminar – 12. November 2015 Inserted network-mean trend error Remaining network-mean trend error  x 2 =  y 2 = 0.141

Preliminary conclusion I Dipdoc Seminar – 12. November 2015

Improvement ratio (1/2) Ratio q depends on the SNR. Upper panel: Doubled noise (SNR = ½ ) leads to doubled remaining trend error. The inserted trend error is unchanged. Doubled q Lower panel: Doubled signal (SNR = 2) leads to doubled inserted trend error. The remaining trend error is unchanged. Halved q Inserted and remaining errors are independent. The inserted error is determined by the break (signal) variance. The remaining error is determined by the noise variance. Dipdoc Seminar – 12. November 2015 Inserted  x 2 =  y 2 = Remaining Inserted Remaining SNR = ½  x 2 =  y 2 = SNR = 2 q = 1.46 (0.73) q = (0.73)

Improvement ratio (2/2) Further parameters (besides SNR) that may also affect q are: break and station number. It shows: The ratio does mainly depend on break number and not on station number. For 6 breaks the ratio is about 1. Dipdoc Seminar – 12. November 2015 Break number Station number

Preliminary conclusion II Does the correction act neutrally if no correction is necessary? Yes, depending on SNR, for SNR=1, break number=6, length=100 the data is neither upgraded nor downgraded. But, the obtained homogenized data is mutually dependent. Standard statistical techniques (using data independency) cannot be applied. All variances are underestimated. Dipdoc Seminar – 12. November 2015

And IF there is a trend error? Dipdoc Seminar – 12. November 2015 Year Mean inserted  I

Non-zero mean trend error The scatter is conserved, compared to zero mean trend error. The data cloud is shifted as a whole to the right. The uncertainty remains, but the mean trend error is well corrected. Dipdoc Seminar – 12. November 2015 Inserted network-mean trend error Remaining network-mean trend error x mean = y mean = 0.010

Including break position errors Question: What happens, if the break positions have errors and are not perfectly known (as in reality). Simulation: Scatter the correct positions by adding noise with standard deviation of 2 years. Result: Only 80% of the trend error is corrected, 20% remains. Dipdoc Seminar – 12. November 2015 Inserted network-mean trend error Remaining network-mean trend error x mean = y mean = 0.161

Conclusion If the original data contains no trend error (if the inhomogeneities have by chance no overall effect) the trend error for individual networks is (under realistic conditions) not improved. However, mutually dependent data results from homogenization. This is a disadvantage. Mean trend errors (due to inhomogeneities) are corrected perfectly, if the break positions are known perfectly. For realistic position errors (  = 2 years) the trend error is only partly corrected, 20% remains. Dipdoc Seminar – 12. November 2015