1. The process has correlation sequence Correlation and Spectral Measure where, the adjoint of is defined by The process has spectral measure where

2. The process is a moving average (MA) if MA and AR Processes has finite support. The process is autoregressive (AR) if there exists a polynomial no zeros in the closed unit disc such that with Then so

3. The MA process has spectral measure MA and AR Processes The AR process has spectral measure whereis a trigonometric roots have modulus > 1. polynomial sois a nonnegative trig. poly. whereis a polynomial whose Lemma (L. Fejer & F. Riesz) Every nonnegative trigonometric polynomial has the form whereis a polynomial whose root moduli < 1. Proof pages 117-118. Corollary Every nonneg. trig. poly. can be the spec. meas. of a MA process. Every reciprocal pos. trig. poly. can be the spec. meas. of a AR pr.

4. Assume that we are given correlation coefficients Prediction and wish to determine so as to minimize A standard fact about least squares estimation is that this quantity is minimized iff These equations, in matrix form, are the

5. Normal Equations Notice that the matrix is Hermitian, Toeplitz and positive semi-definite. If it is positive definite then the solution is unique. If it is singular then Theorem 3.1.2 implies that it has a solution with

6. For the AR process Prediction for AR Process if we choose then Therefore the prediction coefficients are uniquely determined and yield a residual error that equals the innovation processIn fact since the best predictor for as a linear combination of is

7. Empirical Prediction In many (perhaps most) prediction applications we do not have a statistical model for a random processbut instead we have sample values of a ‘time series’and for some positive integerwe want to computethat minimizes or equivalently (prove this) solves the system These equations can be expressed in the form

8. Empirical Normal Equations where Remark It is only approximately Toeplitz Question Show this matrix is positive semidefinite

9. Improved Emp. Normal Equations where hence in the trig. poly. is the coefficient of Show this Toeplitz matrix is pos. semidefinite.

10. Law of Large Numbers numbers that This argument can be made precise using Give an informal argument using the law of large ergodic theory which we will not pursue in this course.

11. Homework Project in MATLAB generate a sequence one million of an white noise process that has a real 1. Using the normal random number generator gaussian distribution. 2. Generate samples of an AR process using: samples 3. Compute the spectral measure for the AR process and use to compute a closed form for the 4. Compute the empirical correlation sequence correlation sequence of the AR process. coefficients and use them to estimate the AR coefficients.