REGRESSION (CONTINUED) LECTURE 4 REGRESSION (CONTINUED) Analysis of Variance; Standard Errors & Confidence Intervals; Prediction Intervals; Examination of Residuals Supplementary Readings: Wilks, chapters 6,9; Bevington, P.R., Robinson, D.K., Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, 1992.

What should we require of them? Recall from last time… Define: We call these residuals What should we require of them?

What should we require of them? Recall from last time… GAUSSIAN What should we require of them?

Analysis of Variance (“ANOVA”)? Recall from last time… Analysis of Variance (“ANOVA”)? 2(n=5) Gaussian data

Analysis of Variance (“ANOVA”) is guaranteed by linear regression procedure Why “n-2”?

Analysis of Variance (“ANOVA”) Define:

Analysis of Variance (“ANOVA”) 1 and n-2 degrees of freedom Define: 1 and n-2 degrees of freedom

Analysis of Variance (“ANOVA”) 1 and n-2 degrees of freedom Source df SS MS F-test Total n-1 SST Regression 1 SSR MSR=SSR MSR/MSE Residual n-2 SSE MSE=se2 1 and n-2 degrees of freedom

Analysis of Variance (“ANOVA”) for Simple Linear Regression Source df SS MS F-test Total n-1 SST Regression 1 SSR MSR=SSR MSR/MSE Residual n-2 SSE MSE=se2 We’ll discuss ANOVA further in the next lecture (“multivariate regression”)

‘Goodness of Fit’

‘Goodness of Fit’ For simple linear regression

‘Goodness of Fit’ Outside the “support” of the regression, in general,

‘Goodness of Fit’ Outside the “support” of the regression, in general,

‘Goodness of Fit’ Reliability Bias

Analysis of Variance (“ANOVA”) Under Gaussian assumptions, the estimates from linear regression of the parameter a and b represent unbiased estimates of means of a Gaussian distribution Where the standard errors in the regression parameters are:

Confidence Intervals The estimated regression slope ‘b’ is likely to be within some range of the true ‘b’

Confidence Intervals This naturally defines a t test for the presence of a trend:

Prediction Intervals MSE in a predicted value or, (‘Prediction Error’) is larger than the nominal MSE, increasing as the predictand value departs from the mean Note that sy approaches se as the ‘training’ sample becomes large

Linear Correlation ‘r’ suffers from sampling error both in the regression slope and the estimates of variance…

Linear Correlation ‘r’ suffers from sampling error both in the regression slope and the estimates of variance…

Linear Correlation Coefficient

Examining Residuals Heteroscedasticity A trend in residual variance violates the assumption of Gaussian residuals…

Examining Residuals Heteroscedasticity Often a simple transformation of the original data will yield more closely Gaussian residuals…

Examining Residuals Leverage Points can still be a problem!

Examining Residuals Autocorrelation Durbin-Watson Statistic

Examining Residuals Autocorrelation Suppose we have the simple (‘first order autoregressive’) model Then we can still use all of the results based on Gaussian statistics, but with the modified sample size: For example:

Examining Residuals Autocorrelation Suppose we have the simple (‘first order autoregressive’) model Then we can still use all of the results based on Gaussian statistics, but with the modified sample size: Different for tests of variance

Examining Residuals Autocorrelation Suppose we have the simple (‘first order autoregressive’) model Then we can still use all of the results based on Gaussian statistics, but with the modified sample size: Different again for correlations

We can remove the serial correlation through Examining Residuals Suppose we have the simple (‘first order autoregressive’) model We can remove the serial correlation through