1. Estimation of the spectral density function

2. The spectral density function, f(l)
The spectral density function, f(x), is a symmetric function defined on the interval [-p,p] satisfying
and
The spectral density function, f(x), can be calculated from the autocovariance function and vice versa.

3. Some complex number results:
Use

5. Expectations of Linear and Quadratic forms of a weakly stationary Time Series

6. Expectations, Variances and Covariances of Linear forms

7. Theorem Let {xt:t  T} be a weakly stationary time series.
Then
and
where
and Sr = {1,2, ..., T-r}, if r ≥ 0, Sr = {1- r, 2 - r, ..., T} if r ≤ 0.

8. Proof

9. Also since
Q.E.D.

10. Theorem Let {xt:t  T} be a weakly stationary time series.
and

11. Expectations, Variances and Covariances of Linear forms Summary

12. Theorem Let {xt:t  T} be a weakly stationary time series.
Then
and
where
and Sr = {1,2, ..., T-r}, if r ≥ 0, Sr = {1- r, 2 - r, ..., T} if r ≤ 0.

13. Theorem Let {xt:t  T} be a weakly stationary time series.
Let and
Then
where and

14. Then
where and
Also Sr = {1,2, ..., T-r}, if r ≥ 0, Sr = {1- r, 2 - r, ..., T} if r ≤ 0.

15. Expectations, Variances and Covariances of Quadratic forms

16. Theorem Let {xt:t  T} be a weakly stationary time series.
Then

17. and

19. and Sr = {1,2, ..., T-r}, if r ≥ 0,
Sr = {1- r, 2 - r, ..., T} if r ≤ 0,
k(h,r,s) = the fourth order cumulant
= E[(xt - m)(xt+h - m)(xt+r - m)(xt+s - m)]
- [s(h)s(r-s)+s(r)s(h-s)+s(s)s(h-r)]
Note k(h,r,s) = 0 if {xt:t  T}is Normal.

20. Theorem Let {xt:t  T} be a weakly stationary time series.
Then

22. where
and

23. Examples
The sample mean

24. Thus
and

25. Also

26. and
where

27. Thus
Compare with

28. Basic Property of the Fejer kernel:
If g(•) is a continuous function then :
Thus

32. The sample autocovariance function
The sample autocovariance function is defined by:

33. or if m is known
where

34. or if m is known
where

35. Theorem Assume m is known and the time series is normal, then:
E(Cx(h))= s(h),

36. and

37. Proof
Assume m is known and the the time series is normal, then:
and

40. and

41. where

42. since

43. hence

44. Thus

45. and
Finally

46. Where

47. Thus

48. Expectations, Variances and Covariances of Linear forms Summary

49. Theorem Let {xt:t  T} be a weakly stationary time series.
Then
and
where
and Sr = {1,2, ..., T-r}, if r ≥ 0, Sr = {1- r, 2 - r, ..., T} if r ≤ 0.

50. Theorem Let {xt:t  T} be a weakly stationary time series.
Let and
Then
where and

51. Expectations, Variances and Covariances of Quadratic forms

52. Theorem Let {xt:t  T} be a weakly stationary time series.
Then

53. and

55. and Sr = {1,2, ..., T-r}, if r ≥ 0,
Sr = {1- r, 2 - r, ..., T} if r ≤ 0,
k(h,r,s) = the fourth order cumulant
= E[(xt - m)(xt+h - m)(xt+r - m)(xt+s - m)]
- [s(h)s(r-s)+s(r)s(h-s)+s(s)s(h-r)]
Note k(h,r,s) = 0 if {xt:t  T}is Normal.

56. Theorem Let {xt:t  T} be a weakly stationary time series.
Then

57. Estimation of the spectral density function

58. The Discrete Fourier Transform

59. Let x1,x2,x3, ...xT denote T observations on a univariate one-dimensional time series with zero mean (If the series has non-zero mean one uses in place of xt).
Also assume that T = 2m +1 is odd.
Then

60. where
with lk = 2pk/T and k = 0, 1, 2, ... , m.

61. The Discrete Fourier transform:
k = 0, 1,2, ... ,m.

62. Note:

63. Since

64. Thus

65. Summary: The Discrete Fourier transform
k = 0, 1,2, ... ,m.

66. Theorem
E[Xk] = 0
with lk= 2p(k/T)
with lk= 2p(k/T) and lh= 2p(h/T)

67. where

68. Proof
Note
Thus

70. Thus
where

71. Thus
Also

72. with q =2p(k/T)+l
with f =2p(h/T)+l

73. Thus
and

74. Defn: The Periodogram:
k = 0,1,2, ..., m
with lk = 2pk/T and k = 0, 1, 2, ... , m.

75. Periodogram for the sunspot data

76. note:

78. Theorem

80. In addition:
If lk ≠ 0
If lk ≠ lh

81. Proof
Note
Let

84. Recall
Basic Property of the Fejer kernel:
If g(•) is a continuous function then :
The remainder of the proof is similar

85. Consistent Estimation of the Spectral Density function f(l)

86. Smoothed Periodogram Estimators

87. Defn: The Periodogram:
k = 0,1,2, ..., m

88. Properties:
If lk ≠ 0
If lk ≠ lh

89. Spectral density Estimator

90. Properties: If lk ≠ 0 The second properties states that:
is not a consistent estimator of f(l):

91. Periodogram Spectral density Estimator
Properties:
Asymptotically unbiased
If lk ≠ 0
The second property states that:
is not a consistent estimator of f(l):

92. Examples of using R

93. Example 1 – Sunspot data

94. Open the Data
> sunData<-read.table("C:/Users/bill/Desktop/Sunspot.txt",header=TRUE)
Set the vector y to the data “no”
> y<-sunData[,"no"]
Draw the raw periodogram
Two commands achieve this
> spectrum(y,method="pgram") or
> spec.pgram(y, taper=0, log=“yes")

95. > spectrum(y,method="pgram") yields

96. > spec.pgram(y, taper=0, log="no") yields

97. > spec.pgram(y, taper=0, log=“yes") yields

98. Drawing the smoothed periodogram using Daniel window
This is achieved using the command
> spec.pgram(y, spans= 9, taper=0, log=“yes")

99. If one does not want the log-scale on y axis then use the command
> spec.pgram(y, spans= 9, taper=0, log=“no")

100. If one want to use the Daniel window on two passes.
Use the command.
> spec.pgram(y, spans= c(9,9) , taper=0, log=“yes")

101. Periodogram Spectral density Estimator
Properties:
Asymptotically unbiased
If lk ≠ 0
The second property states that:
is not a consistent estimator of f(l):

102. Smoothed Estimators of the spectral density

103. The Daniell Estimator

104. Properties 1. 2. 3.

105. Now let T  ∞, d  ∞ such that d/T  0
Now let T  ∞, d  ∞ such that d/T  0. Then we obtain asymptotically unbiased and consistent estimators, that is

106. Choosing the Daniell option in SPSS

107. k = 5

108. k = 5

109. k = 9

110. k = 5

111. Other smoothed estimators

112. More generally consider the Smoothed Periodogram
where
and

113. Theorem (Asymptotic behaviour of Smoothed periodogram Estimators )
Let
where {ut} are independent random variables with mean 0 and variance s2 with
Let dT be an increasing sequence such that
and

114. Then
and
Proof (See Fuller Page 292)

115. Weighted Covariance Estimators
Note
where

116. Proof

118. The Weighted Covariance Estimator
where {wm(h): h = 0, ±1,±2, ...} are a sequence of weights such that:
i) 0 ≤ wm(h) ≤ wm(0) = 1
ii) wm(-h) = wm(h)
iii) wm(h) = 0 for |h| > m

119. The Spectral Window for this estimator is defined by:
Properties :
i) Wm(l) = Wm(-l)
ii)

120. also (Using a Reimann-Sum Approximation)
Note:
also (Using a Reimann-Sum Approximation)
= the Smoothed Periodogram Estimator

121. Asymptotic behaviour for large T1. 2. 3.

122. Examples wm(h) = w(h/m)
1. Bartlett
Note:

123. 2. Parzen
3. Blackman-Tukey
w(x) = 1 -2 a + 2a cos(px)
with a = 0.23 (Hamming) , a = 0.25 (Hanning)

124. Daniell
Tukey
Parzen
Bartlett

125. Approximate Distribution and Consistency1. 2. 3.

126. Note: If Wm(l) is concentrated in a "peak" about l = 0 and f(l) is nearly constant over its width, then 1.
and 2.

127. Confidence Limits in Spectral Density Estimation

128. Satterthwaites Approximation:
where c and r are chosen so that 1. 2.

129. Thus
= The equivalent df (EDF)

130. Now
and
Thus
and

131. Confidence Limits for The Spectral Density function f(l):
Let and denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.
Then a [1- a]100 % confidence interval for f(l) is:

132. Estimation of the spectral density function Summary

133. The spectral density function, f(l)
The spectral density function, f(x), is a symmetric function defined on the interval [-p,p] satisfying
and

134. Periodogram Spectral density Estimator
Properties:
Asymptotically unbiased
If lk ≠ 0
The second property states that:
is not a consistent estimator of f(l):

136. Smoothed Estimators of the spectral density

137. Smoothed Periodogram Estimators
where
and
The Daniell Estimator

138. The Weighted Covariance Estimator
where {wm(h): h = 0, ±1,±2, ...} are a sequence of weights such that:
i) 0 ≤ wm(h) ≤ wm(0) = 1
ii) wm(-h) = wm(h)
iii) wm(h) = 0 for |h| > m

139. Choices for wm(h) = w(h/m)
1. Bartlett
2. Parzen
3. Blackman-Tukey
w(x) = 1 -2 a + 2a cos(px)
with a = 0.23 (Hamming) , a = 0.25 (Hanning)

140. The Spectral Window for this estimator is defined by:
Properties :
i) Wm(l) = Wm(-l)
ii)

141. also (Using a Reimann-Sum Approximation)
Note:
also (Using a Reimann-Sum Approximation)
= the Smoothed Periodogram Estimator

142. Approximate Distribution and Consistency1. 2. 3.

143. Note: If Wm(l) is concentrated in a "peak" about l = 0 and f(l) is nearly constant over its width, then 1.
and 2.

144. Confidence Limits for The Spectral Density function f(l):
Let and denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.
Then a [1- a]100 % confidence interval for f(l) is:

145. Now
and
Thus
and

146. Confidence Limits for The Spectral Density function f(l):
Let and denote the upper and lower critical values for the Chi-square distribution with r d.f. i.e.
Then a [1- a]100 % confidence interval for f(l) is:

147. and