Correlation and spectral analysis Objective: –investigation of correlation structure of time series –identification of major harmonic components in time series Tools: –auto-covariance and auto-correlation function –cross-covariance and cross-correlation function –variance spectrum and spectral density function

Autocovariance and autocorrelation function Autocovariance function of series Y(t): Covariance at lag 0 = variance: To ensure comparibility scaling of autocovariance by variance: For a random series:

Estimation of autocovariance and autocorrelation function –Estimation of autocovariance: –Estimation of autocorrelation function: –Note that estimators are biased (division by n rather than by (n-k-1): gives smaller mean square error –95% confidence limits for zero correlation:

Example autocorrelation function for monthly rainfall station PATAS ( )

Example PATAS continued (1)

Example PATAS continued (2) Periodicity with T = 12 months due to fixed occurrence of monsoon rainfall

Autocorrelogram of daily rainfall at PATAS Lag = 1 month

Autocorrelation function of harmonic it follows: –covariance of harmonic remains a harmonic of the same frequency f, so it preserves frequency information –information on phase shift vanishes –amplitude of covariance is equal to variance of Y(t)

Y(t) = harmonic series

Autocovariance and autocorrelation of harmonic

Cross-covariance and cross-correlation function Cross-covariance: X(t) Y(t+k) Lag = k X Y Time t

Cross-covariance and cross-correlation function Definition cross-covariance function: Definition of cross-correlation function: Note: –for lag 0:  XY (0) < 1 unless perfect correlation –maximum may occur at lag  0 –for two water level stations along river, maximum correlation will be equal to k = distance / celerity –hence upstream station X(t) will give maximum correlation with Y(t+k) at downstream site –or downstream site Y(t) will give maximum correlation with X(t-k) at upstream site

Estimation of cross-covariance and cross- correlation function Estimation of cross-covariance: Estimation of cross-correlation: Note that estimators are biased since n rather than (n-k-1) is used in divider, but estimator provides smaller mean square error

Variance spectrum and spectral density function Plot of variance of harmonics versus their frequencies is called power or variance spectrum –If S p (f) is the ordinate of continuous spectrum then the variance contributed by all frequencies in the frequency interval f, f+df is given by S p (f).df. Hence: –Hydrological processes can be considered to be frequency limited, hence harmonics with f > f c do not significantly contribute to variance of the process. Then: –f c = Nyquist or cut-off frequency

Spectrum (2) Scaling of variance spectrum by variance (to make them comparable) gives the spectral density function: spectral density function is Fourier transform of auto- correlation function

Estimation of spectrum Replace  YY (k) by r YY (k) for k=0,1,2,…,M M = maximum lag for which acf is estimated M is to be carefully selected To reduce sampling variance in estimate a smoothing function is applied: the spectral density at f k is estimated as weighted average of density at f k-1,f k,f k+1 Smoothing function is spectral window Spectral window has to be carefully designed Appropriate window is Tukey window: In frequency domain it implies f k is weighted average according to: 1/4f k-1,1/2f k,1/4f k+1 Bandwidth Number of degrees of freedom

Estimation of spectrum (2) Spectral estimator s(f) for S d (f) becomes: To be estimated at: According to Jenkins and Watts number of frequency points should be 2 to 3 times (M+1) (1-  )100% confidence limits: f c = 1/(2  t)

Confidence limits for white noise (s(f) =2) Variance reduces with decreasing M; for n > 25 variance reduces only slowly

Estimation of spectrum (3) To reduce sampling variance, M should be taken small, say M = 10 to 15 % of N (= series length) However, small M leads to large bandwidth B Large B gives smoothing over large frequency range E.g. if one expects significant harmonics with periods 16 and 24 hours in hourly series: –frequency difference is 1/16 - 1/24 = 1/48 –hence: B < 1/48 –so: M > 4x48/3 = 64 –by choosing M = 10% of N, then N > 640 data points or about one month Since it is not known in advance which harmonics are significant, estimation is to be repeated for different M White noise:  YY (k) = 0 for k > 0 it follows since 0  f  1/2, for white noise s(f)  2

Example of spectrum of monthly rainfall data for station PATAS

Example of spectrum of monthly rainfall data for station PATAS (2)

Example of spectrum of monthly rainfall data for station PATAS (3) Variance is maximum at f = = 1/12