1 Detection of discontinuities using an approach based on regression models and application to benchmark temperature by Lucie Vincent Climate Research Branch, Science and Technologies Branch, Environment Canada Presentation to the COST meeting in Tarragona, Spain March 9-11, 2009

2 Outline  Methodology - identification of changepoints in annual mean temperature - adjustment of monthly and daily values  Testing methodology using simulated values - homogenous series, single step, random number of steps  Identification of biases in Canadian climate data - bias in relative humidity due to change in instruments - bias in radiosonde temperature due to introduction of correction factor - bias in daily minimum temperature due to a change in observing time  Application to benchmark temperature datasets - monthly mean minimum temperature at Groix  Discontinuities in precipitation due to joining station observations

3 Methodology

4 Let y the candidate series and x the reference series Model 1: to identify an homogeneous series y = a 1 + c 1 x 1 + e Model 2: to identify a trend y = a 2 + b 2 i + c 2 x 1 + e i = 1, …, n Model 3: to identify a step y = a 3 + b 3 I + c 3 x 1 + e i = 1, …, n I = 0 for i = 5, …, p-1 I = 1 for i = p, …, n-5 Difference between candidate and reference Model 4: to identify a step w trends bef & aft y = a 4 + b 4 iI 1 + a 5 I 2 + b 5 iI 2 + c 4 x 1 + e i = 1, …, n I 1 = 1 and I 2 = 0 for i = 5,…, p-1 I 1 = 0 and I 2 = 1 for i = p, …, n-5 Identification of changepoint in annual mean temperature

5 Durbin-Watson test: to determine if candidate series is homogeneous (autocorrelation) e i = ρe i-1 + μ i H 0 : ρ = 0 versus H a : ρ > 0 D = Σ(e i – e i-1 ) 2 / Σe i 2 if D > d u => H 0 ; if D H a if d l ≤ D ≤ d u test is inconclusive F test: to determine if the introduction of additional variables improve the fit Model 1 and Model 3 are compared H 0 : b 3 = 0 versus H a : b 3 ≠ 0 F* = [(SSE1–SSE3)/(df1–f3)] / SSE3/df3 if F* > F(1-α; 1, n-3) reject H 0 Statistical tests If there is a significant changepoint, divide series into two segments and re-test each segment separately

6 Example Annual mean of daily maximum temperature of Pointe-au-Père / Mont-Joli Trend of 1.8°C / 84 years Model 1 Model 3

7 Example Difference between candidate and reference Step of 1.1°C in 1943 Annual mean of daily maximum temperature Trend of 1.8°C / 84 years Adjusted series Trend of 0.1°C / 84 years Step of 1.1°C in 1943

8 Remarks  First changepoint is not always associated to a “real” change - use an hierarchical procedure to find all changepoints until. convergence of the position of each changepoint. each segment is homogeneous. each segment is too short  Reference series can contain inhomogeneities - a step in the neighbour series can affect the candidate series - a network bias is difficult to detect - preferable to confirm the changepoint with metadata

9 Application to the 12 monthly series for changepoint p identified in annual mean temperature Difference between candidate and reference January July December Adjustment of monthly temperature If a i show seasonality => apply a i If a i randomly distributed => apply annual adjustment Monthly Adjustments (a i i=1,12)

10 Example Annual mean of daily maximum temperature of Pointe-au-Père / Mont-Joli Step of 1.1°C in 1943 Monthly adjustments Before 1943 After 1943 Instruments on the roof

11 Adjustment of daily temperature Linear interpolation between midmonth target values objectively chosen so that the average of the daily adjustments over a given month is equal to the monthly adjustment: T = A -1 M where M are monthly adjustments and A = Regression model 3 applied to individual daily series for changepoint p: y = a 3 + b 3 I + c 3 x 1 + e i = 1, …, n I = 0 for i = 5, …, p-1 I = 1 for i = p, …, n-5

12 Testing the methodology using simulated values

13 Simulation of annual mean temperatures  Homogeneous Series (series with no steps) Random numbers ~ N(0,1) with AR(1)= homogeneous series of 100 values (years)  Series with one step Step of magnitude 0.25, 0.50, 0.75, …, 2.00 σ Position 5, 10, 15, 20, 35, series with a single step  Series with a random number of steps Step of magnitude ∂ = 0.5 to 2.0 σ; ∂ ~ N(0,1) Position ∆t = exp(0.05), ∆t ≥ series with a random number of steps (0 to 7 steps)  Reference series Reference series cross-correlated with candidate series with correlation factor ~ 0.8 and re-standardized

14 Identification of homogeneous series SNHTTPR MLRWSR Position and magnitude of steps falsely detected when the procedure is applied to 1000 homogeneous series Position and magnitude of steps falsely detected when the procedure is applied to 1000 homogeneous series

15 Identification of a single step Percentage of steps identified when one step is introduced in the candidate series SNHTTPR WSRMLR

16 Identification of a random number of steps Percentage of steps detected versus number of steps introduced

17 Identification of biases in Canadian climate data

18 Bias in relative humidity due to a change instruments 75 Canadian climate stations Example: Kuujjuaq Québec, , dewcel introduced in 1978 Winter SummerFall Spring Step -8.0% Step -3.3%Step -2.8% Step -7.1% Original values Adjusted values Many missing very cold values before the introduction of the dewcel

19 AnnualWinterSummer Temperature anomalies mean for Canada 5 pressure levels observations at 12 UTC Bias in radiosonde temperature due to the introduction of a radiation correction 25 Canadian stations During : - semi-automated system implemented - switch to Vaisala instruments - introduction of radiation correction

20 Bias in daily minimum temperature due to a change in observing time 120 Canadian climate stations On July 1, 1961, the climatological day was redefined to end at 06 UTC Prior to that, it ended at 12 UTC for max temp and 00 UTC for min temp The redefinition of the climatological day has created a cold bias in daily min temp Decreasing step identified in 1961; a filled triangle indicate a significant step at 5% level

21 Bias in daily minimum temperature due to a change in observing time Hourly temperatures at Kapuskasing from July 18 to 23, UTC 00 UTC 06 UTC

22 Application to benchmark temperature datasets

23 Temperature surrogate group 1  Calculate monthly anomalies - departures from the reference period  Produce the long series - sequence of monthly values for 100 years (1200 values)  Produce a reference series for each station - average of the remaining stations  Search for all potential changepoints - the first changepoint is not necessary a real one  When all changepoints identified, determine magnitude of each step

24 Position and magnitude (°C) of each step identified by the regression approach Temperature surrogate group 1

25 Monthly anomalies at GroixAdjusted monthly anomalies at Groix Temperature surrogate group 1

26 Discontinuities in precipitation due to joining station observations

27 Does joining precipitation records create any artificial steps?

28 Methodology Let T i and N i be the monthly total rain (or snow) at the tested site and neighbour respectively for year i: Ratios:. if T i > 0 and N i > 0 then q i =T i /N i. if T i = 0 and N i = 0 then q i = 1. if T i or N i = 0 (or missing) then q i = missing Outliers:. q i < q 0.25 – (3*(q q 0.25 )) q 0.25 and q 0.75 are 25 th and 75 th percentiles. q i > q (3*(q q 0.25 )) outliers q i = missing Standardized ratios:. z i = (q i – Q) / s q Q is average of q i, s q is standard deviation Apply t-test on {z i } to determine if the difference in the means before and after the joining date is different from zero at the significance level 5%:. 30 years before and after joining date. minimum of 5 years on each side of the joining date Adjustments:. A i = q ai / q bi q bi & q ai are ratio means before & after joining date

29 Example Digby Airport and Bear River joined in 1957 Monthly, annual and long series (LS) adjustments by each neighbour. The number in bold indicates that the adjustment correspond to a step significant at the 5% level.

30 Example Digby Airport and Bear River joined in 1957 Monthly, annual and long series (LS) adjustments obtained using the neighbours (purple) and overlapping data (green) Rain Snow

31 Box plots of the differences between the adjustments from neighbours and adjustments from overlapping data obtained from the 60 stations. The circle, box and whiskers indicate the median, 10 th and 90 th percentiles, and minimum and maximum values. Rain Snow Comparing adjustments from neighbours and overlap

32 Thank you! Merci! Gracias!