Stochastic models - time series. Random process. an infinite collection of consistent distributions probabilities exist Random function. a family of random variables, e.g. {Y(t), t in Z}

Specified if given F(y 1,...,y n ;t 1,...,t n ) = Prob{Y(t 1 )  y 1,...,Y(t n )  y n } that are symmetric F(  y;  t) = F(y;t),  a permutation compatible F(y 1,...,y m, ,...,  ;t 1,...,t m,t m+1,...,t n } = F(y 1,...,y m ;t 1,...,t m )

Finite dimensional distributions First-order F(y;t) = Prob{Y(t)  t} Second-order F(y 1,y 2 ;t 1,t 2 ) = Prob{Y(t 1 )  y 1 and Y(t 2 )  y 2 } and so on

Other methods i) Y(t;  ),  : random variable ii) urn model iii) probability on function space iv) analytic formula Y(t) =  cos( t +  )  : fixed  : uniform on (- ,  ]

There may be densities The Y(t) may be discrete, angles, proportions,... Kolmogorov extension theorem. To specify a stochastic process give the distribution of any finite subset {Y(  1 ),...,Y(  n )} in a consistent way,  in A

Moment functions. Mean function c Y (t) = E{Y(t)} =  y dF(y;t) =  y f(y;t) dy if continuous =  y j f(y j ; t) if discrete E{  1 Y 1 (t) +  2 Y 2 (t)} =  1 c 1 (t) +  2 c 2 (t) vector-valued case mean level - signal plus noise: S(t) +  (t) S(.): fixed

Second-moments. autocovariance function c YY (s,t) = cov{Y(s),Y(t)} = E{Y(s)Y(t)} - E{Y(s)}E{Y(t)} non-negative definite   j  k c YY (t j, t k )  0 scalars  crosscovariance function c 12 (s,t) = cov{Y 1 (s),Y 2 (t)}

Stationarity. Joint distributions, {Y(t+u 1 ),...,Y(t+u k-1 ),Y(t)}, do not depend on t for k=1,2,... Often reasonable in practice - for some time stretches Replaces "identically distributed"

mean E{Y(t)} = c Y for t in Z autocovariance function cov{Y(t+u),Y(t)} = c YY (u) t,u in Z u: lag = E{Y(t+u)Y(t)} if mean 0 autocorrelation function  (u) = corr{Y(t+u),Y(t)}, |  (u)|  1 crosscovariance function cov{X(t+u),Y(t)} = c XY (u)

joint density Prob{x < Y(t+u) < x+dx and y < Y(t) < y+ dy} = f(x,y|u) dxdy

Some useful models Chatfield notation Purely random / white noise often mean 0 Building block

Random walk not stationary

(*)

Moving average, MA(q) From (*) stationary

MA(1)  0 =1  1 = -.7

Backward shift operator Linear process. Need convergence condition

autoregressive process, AR(p) first-order, AR(1) Markov Linear process For convergence/stationarity *

a.c.f. From (*) p.a.c.f. corr{Y(t),Y(t-m)|Y(t-1),...,Y(t-m+1)} linearly = 0 for m  p when Y is AR(p)

In general case, Useful for prediction

ARMA(p,q)

ARIMA(p,d,q).

Some series and acf’s

Yule-Walker equations for AR(p). Correlate, with X t-k, each side of

Cumulants. multilinear functional 0 if some subset of variantes independent of rest 0 of order > 2 for normal normal is determined by its moments