The Spectral Representation of Stationary Time Series

Stationary time series satisfy the properties: 1.Constant mean (E(x t ) =  ) 2.Constant variance (Var(x t ) =  2 ) 3.Correlation between two observations (x t, x t + h ) dependent only on the distance h. These properties ensure the periodic nature of a stationary time series

and X 1, X 1, …, X k and Y 1, Y 2, …, Y k are independent independent random variables with where 1, 2, … k are k values in (0,  ) Recall is a stationary Time series

With this time series and We can give it a non-zero mean, , by adding  to the equation

We now try to extend this example to a wider class of time series which turns out to be the complete set of weakly stationary time series. In this case the collection of frequencies may even vary over a continuous range of frequencies [0,  ].

The Riemann integral The Riemann-Stiltjes integral If F is continuous with derivative f then: If F is is a step function with jumps p i at x i then:

First, we are going to develop the concept of integration with respect to a stochastic process. Let {U( ):  [0,  ]} denote a stochastic process with mean 0 and independent increments; that is E{[U( 2 ) - U( 1 )][U( 4 ) - U( 3 )]} = 0 for 0 ≤ 1 < 2 ≤ 3 < 4 ≤ . and E[U( ) ] = 0 for 0 ≤ ≤ .

In addition let G( ) =E[U 2 ( ) ] for 0 ≤ ≤  and assume G(0) = 0. It is easy to show that G( ) is monotonically non decreasing. i.e. G( 1 ) ≤ G( 2 ) for 1 < 2.

Now let us define: analogous to the Riemann-Stieltjes integral

Let 0 = 0 < 1 < 2 <... < n =  be any partition of the interval. Let. Let i  denote any value in the interval [ i-1, i ] Consider: Suppose that and there exists a random variable V such that *

Then V is denoted by:

Properties:

The Spectral Representation of Stationary Time Series

Let {X( ):  [0,  ]} and {Y( ): l  [0,  ]} denote a uncorrelated stochastic process with mean 0 and independent increments. Also let F( ) =E[X 2 ( ) ] =E[Y 2 ( ) ] for 0 ≤ ≤  and F(0) = 0. Now define the time series {x t : t  T}as follows:

Then

Also

Thus the time series {x t : t  T} defined as follows: is a stationary time series with: F( ) is called the spectral distribution function: If f( ) = F ˊ ( ) is called then is called the spectral density function:

Note The spectral distribution function, F( ), and spectral density function, f( ) describe how the variance of x t is distributed over the frequencies in the interval [0,  ]

The autocovariance function,  (h), can be computed from the spectral density function, f( ), as follows: Also the spectral density function, f( ), can be computed from the autocovariance function,  (h), as follows:

Example: Let {u t : t  T} be identically distributed and uncorrelated with mean zero (a white noise series). Thus and

Graph:

Example: Suppose X 1, X 1, …, X k and Y 1, Y 2, …, Y k are independent independent random variables with Let 1, 2, … k denote k values in (0,  ) Then

If we define {X( ):  [0,  ]} and {Y( ):  [0,  ]} Note: X( ) and Y( ) are “random” step functions and F( ) is a step function.

Another important comment In the case when F( ) is continuous then

in this case Sometimes the spectral density function, f( ), is extended to the interval [- ,  ] and is assumed symmetric about 0 (i.e. f s ( ) = f s (- ) = f ( )/2 ) It can be shown that

Hence From now on we will use the symmetric spectral density function and let it be denoted by, f( ).

Linear Filters

Let {x t : t  T} be any time series and suppose that the time series {y t : t  T} is constructed as follows:: The time series {y t : t  T} is said to be constructed from {x t : t  T} by means of a Linear Filter. input x t output y t Linear Filter a s

Let  x (h) denote the autocovariance function of {x t : t  T} and  y (h) the autocovariance function of {y t : t  T}. Assume also that E[x t ] = E[y t ] = 0. Then::

Hence where of the linear filter

Note: hence

Spectral density function Moving Average Time series of order q, MA(q) Let  0 =1,  1,  2, …  q denote q + 1 numbers. Let {u t |t  T} denote a white noise time series with variance  2. Let {x t |t  T} denote a MA(q) time series with  = 0. Note: {x t |t  T} is obtained from {u t |t  T} by a linear filter.

Now Hence

Example: q = 1

Example: q = 2

Spectral density function for MA(1) Series

Spectral density function Autoregressive Time series of order p, AR(p) Let  1,  2, …  p denote p + 1 numbers. Let {u t |t  T} denote a white noise time series with variance  2. Let {x t |t  T} denote a AR(p) time series with  = 0. Note: {u t |t  T} is obtained from {x t |t  T} by a linear filter.

Now Hence

Example: p = 1

Example: p = 2

Example : Sunspot Numbers ( )

Autocorrelation function and partial autocorrelation function

Spectral density Estimate

Assuming an AR(2) model

A linear discrete time series Moving Average time series of infinite order

Let  0 =1,  1,  2, … denote an infinite sequence of numbers. Let {u t |t  T} denote a white noise time series with variance  2. –independent –mean 0, variance  2. Let {x t |t  T} be defined by the equation. Then {x t |t  T} is called a Linear discrete time series. Comment: A linear discrete time series is a Moving average time series of infinite order

The AR(1) Time series Let {x t |t  T} be defined by the equation. Then

where and An alternative approach using the back shift operator, B. The equation: can be written

Now since The equation: has the equivalent form:

The time series {x t |t  T} can be written as a linear discrete time series where and For the general AR(p) time series: [  (B)] -1 can be found by carrying out the multiplication

can be written: where Thus the AR(p) time series: Hence This called the Random Shock form of the series

can be written: where Thus the AR(p) time series: Hence This called the Random Shock form of the series

An ARMA(p,q) time series {x t |t  T} satisfies the equation: where and The Random Shock form of an ARMA(p,q) time series:

Again the time series {x t |t  T} can be written as a linear discrete time series namely where  (B) =[  (B)] -1 [  (B)] can be found by carrying out the multiplication

Thus an ARMA(p,q) time series can be written: where

The inverted form of a stationary time series Autoregressive time series of infinite order

An ARMA(p,q) time series {x t |t  T} satisfies the equation: where and Suppose that This will be true if the roots of the polynomial all exceed 1 in absolute value. The time series {x t |t  T} in this case is called invertible.

Then where or

Thus an ARMA(p,q) time series can be written: where This is called the inverted form of the time series. This expresses the time series an autoregressive time series of infinite order.