Estimation of the spectral density function

The spectral density function, f( ) The spectral density function, f(x), is a symmetric function defined on the interval [- ,  ] satisfying and The spectral density function, f(x), can be calculated from the autocovariance function and vice versa.

Some complex number results: Use

Expectations of Linear and Quadratic forms of a weakly stationary Time Series

Expectations, Variances and Covariances of Linear forms

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then and where andS r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0.

Proof

Also since Q.E.D.

TheoremLet {x t :t  T} be a weakly stationary time series. Let and

Expectations, Variances and Covariances of Linear forms Summary

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then and where andS r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0.

TheoremLet {x t :t  T} be a weakly stationary time series. Let and Then where and

Then where and Also S r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0.

Expectations, Variances and Covariances of Quadratic forms

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then

and

and S r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0,  (h,r,s) = the fourth order cumulant = E[(x t -  )(x t+h -  )(x t+r -  )(x t+s -  )] - [  (h)  (r-s)+  (r)  (h-s)+  (s)  (h-r)] Note  (h,r,s) = 0 if {x t :t  T}is Normal.

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then

and where

Examples The sample mean

and Thus

Also

and where

Thus Compare with

Basic Property of the Fejer kernel: If g() is a continuous function then : Thus

The sample autocovariance function The sample autocovariance function is defined by:

where or if  is known

where or if  is known

TheoremAssume  is known and the time series is normal, then: E(C x (h))=  (h),

and

Proof Assume  is known and the the time series is normal, then: and

where

since

hence

Thus

and Finally

Where

Thus

Expectations, Variances and Covariances of Linear forms Summary

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then and where andS r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0.

TheoremLet {x t :t  T} be a weakly stationary time series. Let and Then where and

Expectations, Variances and Covariances of Quadratic forms

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then

and

and S r = {1,2,..., T-r}, if r ≥ 0, S r = {1- r, 2 - r,..., T} if r ≤ 0,  (h,r,s) = the fourth order cumulant = E[(x t -  )(x t+h -  )(x t+r -  )(x t+s -  )] - [  (h)  (r-s)+  (r)  (h-s)+  (s)  (h-r)] Note  (h,r,s) = 0 if {x t :t  T}is Normal.

TheoremLet {x t :t  T} be a weakly stationary time series. Let Then

Estimation of the spectral density function

The Discrete Fourier Transform

Let x 1,x 2,x 3,...x T denote T observations on a univariate one-dimensional time series with zero mean (If the series has non-zero mean one uses in place of x t ). Also assume that T = 2m +1 is odd. Then

where with k = 2  k/T and k = 0, 1, 2,..., m.

The Discrete Fourier transform: k = 0, 1,2,...,m.

Note:

Since

Thus

Summary: The Discrete Fourier transform k = 0, 1,2,...,m.

Theorem with k  k/T) E[X k ] = 0 with k  k/T) and h  h/T)

where

Proof Note Thus

where

Thus Also

with  =2  (k/T)+ with  =2  (h/T)+

Thus and

Defn: The Periodogram: k = 0,1,2,..., m with k = 2  k/T and k = 0, 1, 2,..., m.

Periodogram for the sunspot data

note:

Theorem

In addition: If k ≠ 0 If k ≠ h

Proof Note Let

Recall Basic Property of the Fejer kernel: If g() is a continuous function then : The remainder of the proof is similar

Consistent Estimation of the Spectral Density function f( )

Smoothed Periodogram Estimators

Defn: The Periodogram: k = 0,1,2,..., m

Properties: If k ≠ 0 If k ≠ h

Spectral density Estimator

Properties: If k ≠ 0 The second properties states that: is not a consistent estimator of f( ):

Periodogram Spectral density Estimator Properties: If k ≠ 0 The second property states that: is not a consistent estimator of f( ): Asymptotically unbiased

Examples of using packages SPSS, Statistica

Example 1 – Sunspot data

Using SPSS Open the Data

Select Graphs- > Time Series - > Spectral

The following window appears Select the variable

Select the Window Choose the periodogram and/or spectral density Choose whether to plot by frequency or period

Periodogram Spectral density Estimator Properties: If k ≠ 0 The second property states that: is not a consistent estimator of f( ): Asymptotically unbiased

Smoothed Estimators of the spectral density

The Daniell Estimator

Properties

Now let T  ∞, d  ∞ such that d/T  0. Then we obtain asymptotically unbiased and consistent estimators, that is

Choosing the Daniell option in SPSS

k = 5

k = 9

k = 5

Other smoothed estimators

More generally consider the Smoothed Periodogram and where

Theorem (Asymptotic behaviour of Smoothed periodogram Estimators ) and Let where {u t } are independent random variables with mean 0 and variance  2 with Let d T be an increasing sequence such that

and Then Proof (See Fuller Page 292)

Weighted Covariance Estimators Recall that where

The Weighted Covariance Estimator where {w m (h): h = 0, ±1,±2,...} are a sequence of weights such that: i) 0 ≤ w m (h) ≤ w m (0) = 1 ii) w m (-h) = w m (h) iii) w m (h) = 0 for |h| > m

The Spectral Window for this estimator is defined by: i) W m ( ) = W m (- ) ii) Properties :

also (Using a Reimann-Sum Approximation) = the Smoothed Periodogram Estimator Note:

1. Asymptotic behaviour for large T 2. 3.

1. Bartlett Examples w m (h) = w(h/m) Note:

2. Parzen w(x) = 1 -2 a + 2a cos(  x) 3. Blackman-Tukey with a = 0.23 (Hamming), a = 0.25 (Hanning)

DaniellTukey Parzen Bartlett

1. Approximate Distribution and Consistency 2. 3.

1. Note: If W m ( ) is concentrated in a "peak" about = 0 and f( ) is nearly constant over its width, then 2. and

Confidence Limits in Spectral Density Estimation

1. Satterthwaites Approximation: 2. where c and r are chosen so that

Thus = The equivalent df (EDF)

and Now Thus

Then a [1-  100 % confidence interval for f( ) is: Confidence Limits for The Spectral Density function f( ) : Let  and  denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.

Estimation of the spectral density function Summary

The spectral density function, f( ) The spectral density function, f(x), is a symmetric function defined on the interval [- ,  ] satisfying and

Periodogram Spectral density Estimator Properties: If k ≠ 0 The second property states that: is not a consistent estimator of f( ): Asymptotically unbiased

Smoothed Estimators of the spectral density

Smoothed Periodogram Estimators and where The Daniell Estimator

The Weighted Covariance Estimator where {w m (h): h = 0, ±1,±2,...} are a sequence of weights such that: i) 0 ≤ w m (h) ≤ w m (0) = 1 ii) w m (-h) = w m (h) iii) w m (h) = 0 for |h| > m

1. Bartlett Choices for w m (h) = w(h/m) 2. Parzen w(x) = 1 -2 a + 2a cos(  x) 3. Blackman-Tukey with a = 0.23 (Hamming), a = 0.25 (Hanning)

The Spectral Window for this estimator is defined by: i) W m ( ) = W m (- ) ii) Properties :

also (Using a Reimann-Sum Approximation) = the Smoothed Periodogram Estimator Note:

1. Approximate Distribution and Consistency 2. 3.

1. Note: If W m ( ) is concentrated in a "peak" about = 0 and f( ) is nearly constant over its width, then 2. and

Then a [1-  100 % confidence interval for f( ) is: Confidence Limits for The Spectral Density function f( ) : Let  and  denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.

and Now Thus

Then a [1-  100 % confidence interval for f( ) is: Confidence Limits for The Spectral Density function f( ) : Let  and  denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.

and