1. Managing Time Series and Estimating Missing Values
Outlier Detection and the Estimation of Missing Values Martin Charlton and Paul Harris National Centre for Geocomputation National University of Ireland Maynooth Maynooth, Co Kildare, IRELAND
ESPON 2013 Programme Workshop
Managing Time Series and Estimating Missing Values
6 May 2010
Luxembourg

2. Outline Time Series ESPON DB data issues Detecting exceptional values
Estimation of missing values
Case study

3. 1: Time Series

4. What is a time series?
A variable which is measured sequentially in time at fixed sampling intervals is known as a time series
The behaviour of such series can be modelled
The main features of time series are trend and (sometimes) seasonal variation
Observations which are close together in time tend to be correlated

5. Air Passengers
A time plot of the number of air passengers per month between January 1949 and December 1960 in the USA reveals a rising trend
There is also a seasonal pattern of travel within each year. More people travel in the summer than the winter.

6. Aggregating the series annually reveals the rising trend, and the boxplot shows that more people travel in the summer months.

7. Forecasting: 1 There are many modelling and forecasting techniques.
Here we use the Holt Winters procedure to model the series behaviour…
The fit is quite promising

8. Forecasting: 2
And if the growth of the US air traffic during the first 4 years of the 1960s follows the pattern of the previous 12…
the forecast is for some 800 million passengers by 1965

9. Models There are a wide variety of different models, including
Basic stochastic models (like Holt Winters)
Stationary models (AR, MA, ARMA)
Non-stationary models (ARIMA, ARCH)
Spectral analysis (based on the Fourier transform)
Multivariate models (two or more series are involved)

10. 2: ESPON DB Data Issues

11. Some typical data… household income
We might normalise these by the population to reach a comparable ‘per capita’ figure
The NUTS2 regions in Austria are the Länder – here we have short time series concerning disposable income of private households from 1995 to Each series has only 13 elements

12. Short series…
We should be aware that there is an interaction between the amount of data available and what can be done with it
Paas, Kusk, Schlitte and Võrk’s 2007 analysis of income convergence in selected countries of the EU using NUTS3 data had this to say:

13. George Box, 1976, Science and Statistics
Models include not just the analytical tools that others might use, but those which we use to examine the data for outliers and estimating values
‘Wrong’ for Box includes models that fail to encapsulate the process under investigation

14. ESPON Tigers
Long time series tend to be for large areal units, such as countries, or major administrative regions – the MAUP may well also be a tiger
Smaller regions…
shorter series
incomplete series
a long time period between elements (decennial censuses) in the case of very small units

15. 3: Detecting Exceptional Values

16. Exceptional values Two types: Identification methods
Logical errors (e.g. negative unemployment rate)
Statistical outlier (e.g. unusually high unemployment rate)
Identification methods
Logical errors: mechanical (& statistical) techniques
Statistical outliers: statistical techniques

17. Types of outliers

18. Our approach There is no single ‘best’ detection technique, so…
Apply a selection of outlier detection methods, which are simple and robust
Flag an observation if it is a likely outlier according to each technique
Build up the weight of evidence for the likelihood of an value being statistically exceptional
Suggest what type of outlier it is likely to be
aspatial, spatial, temporal, relationship, a mixture
Consult an expert of the data to decide on the appropriate cause of action

19. Issues Temporal outliers Modifiable Areal Unit Problem MAUP
The time series are often too short to apply a ‘standard’ technique reliably
So... Parallel time series are treated as additional variables (there will be a high positive correlation between series from different years)
Then... Apply an aspatial/spatial/relationship detection technique
That is... We add the spatial component which is then treated either implicitly or explicitly
Modifiable Areal Unit Problem MAUP
Identify exceptional values at the finest spatial resolution

20. Weight of evidence
If we apply a range of techniques, then we can build up the weight of evidence for the likelihood of an observation being exceptional
Observations which are exceptional on most or all of the tests are those which we would select for further investigation
Here’s an example showing three observations…

21. Identification technique
Identification type
Obsn. 1
Obsn. 2
Obsn. 3
1. Boxplot
Aspatial & univariate
Yes
2. Bagplot
Aspatial & bivariate
Relationship
3. Residuals from locally weighted mean & Hawkins test statistic
Spatial & univariate
4. Residuals from multiple linear regression*
(requires modelling decisions)
Aspatial & multivariate
Linear relationships
5. Residuals from locally weighted regression*
Nonlinear relationships
6. Residuals from geographically weighted regression* (requires modelling decisions)
Spatial & multivariate
7. Basic & robust principal component analysis* (model-decision free)
8. Locally weighted principal component analysis* (model-decision free)
9. Geographically weighted principal component analysis* (model-decision free)
* Can have a spatial, univariate form if the coordinate data are used as variables

22. 4: Estimating Missing Data

23. Data estimation techniques
There is an enormous range of possibilities
Choice depends on
Data type, size, dimensionality, and properties
Objective – prediction or prediction uncertainty accuracy
Model complexity
We can estimate missing values using...
Averaging
Regression (with or without autocorrelation, global and local)
Inverse distance weighting
Regression Kriging
Co-Kriging
Bayesian Markov Chain Monte Carlo methods

24. 5: Case study Identifying NUTS regions with exceptional time-series values

25. Unemployment at NUTS
A dataset for NUTS23 regions was obtained from UMS-RIATE
For each year there are counts of
Economically active population
Unemployed, economically active population
Shapefile created from NUTS2/NUTS3 shapefiles in Mapkit
Analysis undertaken in R

26. Eight ‘unemployment rate’ variables for 2000 to 2007
Rate = [Unemployed/Economically active]
790 x 8 observations at NUTS 2/3 level
Some island data removed

27. Data post-processing Logical input errors
Original data checked
There appear to be none, appear to be a few exceptional values
Assessing outlier detection methods
320 values randomly picked (~5% of the data)
These are in 271 regions
Values doubled and then randomly redistributed among the 320 positions in the data
These observations are assumed to be outlying in some way (but we cannot guarantee this)

28. Effect of outliers?
Merely looking at some maps doesn’t help in easily identifying the regions with exceptional values

29. Interseries correlations
Those plots about the main diagonal are highly correlated.
The effect of the randomly introduced values is clearer on the more distant plots (these are also ‘distant’ in time)

30. Detection Techniques for comparison
Simple time-series approach (TS) – outlined in FIR: we have used a simplified version
Principal Components Analysis (PCA)
GWPrincipal Components Analysis (GWPCA)
The PCA based methods allow us to consider more than simply pairs of time series simultaneously

31. We’ll compare the various methods

32. Time Series method (TS)
For each of the 790 regions, index TS is calculated at each of 8 time observations (using the 8-observation data set):
TS = [observation – mean]2/[variance]
Assuming Gaussian errors, a time observation is taken as outlying if TS > 3.84 (95% level)
In this study, we simply find outliers according to boxplot statistics
An indicator variable is then set at any region for which at least one time observation is outlying

33. Principal Components Analysis (PCA)
Principal Components Analysis is a technique which transforms m correlated variables into m new variables which are have a correlation of zero
All of the variance in the original m variables is retained during the transformation
Values of the new variables are known as scores – we can use these for identifying exceptional values

34. Geographically Weighted PCA
PCA is a global transformation but it ignores the spatial arrangement of the NUTS regions
With GWPCA we obtain local transformations by applying geographical weighting – this gives us a set of components for each NUTS region
We can use the scores from these local transformations to identify exceptional values

35. PCA for the unemployment series
The series are highly correlated, so the first component accounts for the majority of the variance

36. Using PCA and GWPCA
Examine the residual component data (those with small variances)
Use boxplot statistics to define outlying values
In this case, a significant result indicates one or more outlying time observations in a NUTS region
GWPCA will also indicate a spatial ‘outlyingness’ in the data

37. The various techniques are compared on the next slides

38. (a) TS method compared with PCA
The TS method appears to be less discriminating than the global PCA method

39. (b) TS compared with GWPCA
The GWPCA method would appear to be very discriminating in identifying potentially exceptional regions

40. (c) PCA compared with GWPCA
The global PCA is slightly less discriminating than the GW PCA

41. Results for the 271 randomised sites
Sites not identified as outlying – 21.4%
Outlying by at least one method – 78.6%
Outlying by one method only – 55.3%
Outlying by two methods – 18.8%
Outlying by all three methods – 4.8%

42. Identification by method:
TS (75.6%)
PCA (22.5%)
GWPCA (8.8%)
False positives at 519 un-affected sites:
TS (29.5%)
PCA (2.3%)
GWPCA (1.3%)
These results endorse the “weight of evidence” approach to the identification of exceptional values…

43. Acknowledgements
We are disappointed that Eyjafjallajökull decided to send some ash to Ireland
We are deeply grateful to Claude for presenting this work – some of it is not easy
We also acknowledge statistical advice from Professor Chris Brunsdon, Professor of Geographic Information at the University of Leicester

44. Thank You!