Data Analytics CMIS Short Course part II Extra Material: Forecasting Sam Buttrey December 2015

Some Notes On Time Series Time Series data are collected over time and show time dependency We could include time as a covariate in a regular model, but usually its effect is non- linear (or periodic) A big branch of statistics handles data from time series Spatial data is different, though lots of varies in both both time and space

Two Approaches 1.Time domain: A process evolving over time: X 1, X 2, …, X t, … –Usually numeric rather than categorical –Usually assumed to be sampled regularly Many models assume stationarity: joint dist’n of X’s is unchanged over time and mixing: long-term dependence decays If that’s not so, we might transform our original series into a stationary one

Frequency Example 2.Frequency domain: characterize vibration signature (spectrum) –E.g. rotating machinery Find changes in spectrum that might indicate change in machinery health Identifying spectrum is tricky: adjacent frequencies similar… Represent data by Fourier series (sines, cosines) and select a subset –Shades of principal componenets

Exponential Smoothing Often our goal is to predict the next observation, or the next few, as accurately as possible Consider data with no trend, periodicity A natural predictor is a moving average, perhaps with weights decreasing as you go backwards in time: Predict X t+1 by a 1 X t + a 2 X t–1 + a 3 X t–2 … where the a’s add up to 1

Exponential Smoothing

Smoothing plan #1

Implementation Example: Nile data (class ts ) HoltWinters(Nile, beta=FALSE, gamma=FALSE) ses () {forecast} What do predictions look like? –If no new observations are available, we predict in a (horizontal) line with constant y

Modeling Trend Example: uspop Exponential smoothing always lags behind a series with trend or periodicity In “double exponential smoothing,” there are two smoothers:  for level L as before,  for trend T Set L 1 = X 1, T 1 = X 2 – X 1

Double Expo Smoothing

Periodicity For periodicity, keep a set of seasonal effects with one common smoother  E.g., after observing a December, smooth on (This obs – Est’d Dec. effect) Use that to update level and trend Then predict January with Lt+1 + Tt+1 + Est’d January effect… …And then go back and update your estimate of the December effect for next year – triple exponential smoothing

Holt Winters Seasonal Method Triple exponential smoothing is also called the “Holt-Winters” method Example: retail sales –Lots of data is periodic (esp. monthly, quarterly), so this approach is widely used –How to get SE’s for prediction? –So far, no formal modelling or inference!

Prediction Intervals Forecast library Hyndman describes fifteen sorts of expo. smoothing model –Trend: none, additive, additive damped, multiplicative, multiplicative damped –Seasonal: none, additive, multiplicative For some, given iid N(0,  2 ) noise, we can write down prediction intervals For others, simulate, say, 1000 paths and use that range…BUT

Prediction Intervals I, personally, am always wary of prediction intervals for time series They assume the model is more-or-less correct now, and that it will continue to be correct into the future And why should it be? Stuff happens –“Black swans”: Unpredictable, high impact, rationalized after the fact Ex.: hw() on the first 200 entries of “retail”

Final Thoughts Time series models fill an important niche, but lots of issues regarding real-life problems remain It’s important to include outside (“exogenous”) information Extensions: multivariate outcomes, categorical outcomes, irregular sampling, exogenous regressors… Predict at your own risk