Andrew Smith Denoising UK house prices 14th April 2010

2 Regression on a graph Denoising UK house prices Discrete spatial processes Longitudinal data analysis Scatterplot smoothing Image analysis

3 Outline UK house price data The need for regression Graph structure Regression on a graph Penalised regression Results and uses of algorithm

4 Video available at

5 Regression Seek a (simpler) pattern that explains observed data Model: Data=Signal+Noise Price year,town =True value year,town +σ year Z year,town

6 No covariate values Euclidean distance inappropriate Missing values Challenges for existing methods

7 Regression on a graph Consider observations Price year,town to be taken at the vertices of a graph Edges of the graph give an idea of closeness

8 Consider observations Price year,town to be taken at the vertices of a graph Edges of the graph (neighbouring towns) give an idea of closeness Regression on a graph

9 Penalised regression on a graph Need to penalise distance from data Penalty term: Sum (over all vertices) of squares of residuals

10 Penalised regression on a graph Need to penalise roughness Penalty term: Sum (over all edges) of absolute differences

11 Penalised regression on a graph Minimise: Distance from data + λ Roughness Computationally intensive, so use new algorithm

12 Video available at

13 Regional analysis The estimate identifies regions of constant value These regions change in a similar way through time

14 Regional analysis The estimate identifies regions of constant value These regions change in a similar way through time They are not the same as government office regions

15 Summary Detecting a smooth national trend in noisy UK house price data Use regression on a graph Penalise distance from data (vertices) and roughness (edges) arxiv.org/abs/ (Kovac & Smith 2009) All house price data courtesy of Halifax