SADNA – Ad Auction lecture #3 Time Series Yishay Mansour Mariano Schain

Pg-dvd Pg-null null-dvd null-null

Bounded Window Parameter W Take the average of the last W samples – Smooth the samples – Larger W – smoother outcome – Smaller W – better following recent trends Computationally – Requires keeping last W samples – Simple update: S t+1  S t - y t-W /W + y t /W

Exponential Weight Moving average Parameter 0 < α < 1 Formula: S t+1  αy t + (1- α)S t Effect of α: – Smaller α: larger weight to history – Large α: short reaction to trend – Effective window size: 1/ α Why “exponential”: S t+1 = α ∑ (1- α) j y t-j

Discussion Setting up the parameters: – α = depends on the stability of the data Can be found by minimizing objective function R t = y t – S t Minimize MSE = min α ∑ R t 2 – why MSE ? Note S t+1 = S t + α R t – S 1 = undefined (need to initialize somehow) Problems: – Trend

Double Exponential Smoothing Handles trend Parameters: α, γ Formula: S t+1  αy t + (1- α)(S t + b t ) equivalent: S t+1  S t + α(y t - (S t + b t )) b t+1  γ (S t+1 - S t ) + (1- γ) b t Motivation – S t tracks the smoothed point – b t tracks the smoothed slope Forecasting: F t+1 = S t + b t

Double Exponential Smoothing Simple Example: – y t = t – Then: b t = 1 and S t =t-1 Residual Analysis – R t = y t – S t – Again S t+1 = S t + α R t Why not: b t+1  γ (y t – y t-1 ) + (1- γ) b t

Triple Exponential Smoothing Handles seasonality of cycle L – (Holt-Winters) Parameters: α, β, γ Formula: S t+1  αy t /I t + (1- α)(S t + b t ) b t+1  γ (S t+1 - S t ) + (1- γ) b t I t+1  β y t / S t + (1- β)I t-L F t+1  (S t + b t ) I t-L

Triple Exponential Smoothing Average and trend as before I t should be cyclic – cycle size L (how to find it?) – measures that ratio of current value to average. Illustrative: I=2 I=1 I=0.5 I=1 I=2

Linear Regression: Basics Simple model to fit the data Basic example a 1 X 1 + … + a p X p Goal: minimize square error (MSE) ∑ t (a 1 X 1 t + … + a p X p t – Y t ) 2 – The constants are the X’s and Y’s – We are solving for the coefficients: a’s

Linear Regression What counts as linear ?! – a 0 + a 1 x – a 0 + a 1 x + a 2 x 2 + a 3 x 3 – a 1 x 1 + a 2 x 2 + a 3 x 1 x 2 + a 4 x a 5 x 2 2 – a 1 log x 1 + a 2 exp(x 2 ) – a 0 + a 1 a 0 x 1 – a + x/a

Linear Regression: Computation Simple case: ax + b Minimize ∑ t (a x t + b – y t ) 2 Need to solve for a and b ∑ t 2 (a x t + b – y t ) x t = 0 ∑ t 2 (a x t + b – y t ) =0 The equations: a*avg(x 2 ) +b*avg(x) = avg (xy) a * avg(x) + b = avg(y) The Solution

Non-Linear Regression We need to optimize the MSE Now the derivatives are not linear – in the a’s Need a more complicate solver There are software out there that do it – non-linear fitting procedures

Autoregressive Model (AR) A model of the current value – given previous observed values Model – X t = c 0 + a 1 X t-1 + a 2 X t-2 + … + a p X t-p + ε t usually ∑ a i < 1 c 0 = (1- ∑ a i ) E[X] – Need to solve for the coefficients – Simple linear regression

Moving Average Model (MA) A model of the current value – given previous unobserved residuals Model: – X t = μ + ε t - b 1 ε t-1 - b 2 ε t-2 - … - b q ε t-q This is a linear regression in the residuals ε’s PROBLEM: we do not observe the residuals directly – non-linear fitting procedures

ARIMA(p,q) Combines – an autoregressive model (AR) p values back – a moving average (MA) q values back Model: X t = c 0 + a 1 X t-1 + a 2 X t-2 + … + a p X t-p + ε t - b 1 ε t-1 - b 2 ε t-2 - … - b q ε t-q

Detecting change When is there a shift Consider the Residuals – R t = y t – F t Stable residuals  No change Much higher residuals  maybe change In the simulation: detect a burst

How can you use this Understand the concepts Understand the alternatives There is enough software you can use – you will need to select the model – specify the input – understand what the output means If you use software, remember: – document which software you use – document where you use it