On the multiple breakpoint problem and the number of significant breaks in homogenisation of climate records Separation of true from spurious breaks Ralf Lindau & Victor Venema University of Bonn

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Internal and External Variance Consider the differences of one station compared to a neighbour or a reference. Breaks are defined by abrupt changes in the station-reference time series. Internal variance within the subperiods External variance between the means of different subperiods Criterion: Maximum external variance attained by a minimum number of breaks

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Decomposition of Variance n total number of years N subperiods n i years within a subperiod The sum of external and internal variance is constant.

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 First Question How do random data behave? Needed as stop criterion for the number of significant breaks.

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Random Time Series with stddev = 1 Segment averages x i scatter randomly mean : 0 stddev:1/ Because any deviation from zero can be seen as inaccuracy due to the limited number of members.

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012  2 -distribution The external variance is equal to the mean square sum of a random standard normal distributed variable. Weighted measure for the variability of the subperiods‘ means

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 From  2 to  distribution n = 21 years k = 7 breaks As the total variance is normalized to 1, a kind of normalized chi 2 -distribution is expected: This is the  -distribution. data   The exceeding probability P gives the best (maximum) solution for v Incomplete Beta Function 7 breaks in 21 years

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Added variance per break Incomplete  -function: Transformation to dv/dk: mean 90% 95%

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 The extisting algorithm Prodige Original formulation of Caussinus and Mestre for the penalty term in Prodige Translation into terms used by us. Normalisation by k* = k / (n -1) Derivation to get the minimum In Prodige it is postulated that the relative gain of external variance is a constant for given n.

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Shorter length, less certainty n = 21 yearsn = 101 years Exceeding probability 1/128 1/64 1/32 1/16 1/8 1/4

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Second Question How do true breaks behave?

True Breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 

Identical Behaviour True breaks behave identical to random data. But the abscissa-scale is now: k / n k instead of k / n. Compared to random time series the external variance grows faster by the factor n / n k 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 data theory n k = 19 true breaks within n = 100 years time series Assumed / True Break Number k / n k

Break vs Scatter Regime Simulated data with 19 breaks interfered by scatter The internal variance decrease as a function of break number. In the break regime the variance decrease faster by the factor: 15 breaks are detectable, depending on signal to noise ratio. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Time series length Number of true breaks

12th EMS Annual Meeting, Lodz, Poland – 13. September 2012 Conclusions The analysis of random data shows that the external variance is  -distributed, which leads to a new formulation for the penalty term. True breaks are also  -distributed. Their external variance increases faster by a factor of n/n k compared to random scatter.