1. Multiple Linear Regression.
Concept and uses.
Model and assumptions.
Intrinsically linear models.
Model development and validation.
Problem areas.
Non-normality.
Heterogeneous variance.
Correlated errors.
Influential points and outliers.
Model inadequacies.
Collinearity.
Errors in X variables.
September 19, 2018
AGR206

2. Concept & Uses. Did you know?
REGRESSION
ANOVA
Did you know?
Description restricted to data set. Did biomass increase with pH in the sample?
Prediction of Y. How much biomass we expect to find in certain soil conditions?
Extrapolation for new conditions: can we predict biomass in other estuaries?
Estimation and understanding. How much does biomass change per unit change in pH and controlling for other factors?
Control of process: requires causality. Can we create sites with certain biomass by changing the pH?
September 19, 2018
AGR206

3. Body fat example in JMP.
Three variables (X1, X3, X3) were measured to predict body fat % (Y) in people.
Random sample of people.
Y was measured by an expensive and very accurate method (assume it reveals true %fat).
X1: thickness of triceps skinfold
X2: thigh circumference
X3: midarm circumference.
Bodyfat.jmp
September 19, 2018
AGR206

4. Ho’s or values “of interest”
Does thickness of triceps skinfold contribute significantly to predict fat content?
What is the CI for fat content for a person whose X’s have been measured?
Do I have more or less fat than last summer?
Do I have more fat than recommended?
September 19, 2018
AGR206

5. Model and Assumptions.
Linear, additive model to relate Y to p independent variables.
Note: here, p is number of variables, but some authors use p for number of parameters, which is one more than variables due to the intercept.
Yi=b0+ b1 Xi1+…+ bp Xip+ei
where ei are normal and independent random variables with common variance s2.
In matrix notation the model and solution are exactly the same as for SLR: Y= Xb + e b=(X’X)-1(X’Y)
All equations from SLR apply without change.
September 19, 2018
AGR206

6. Linear models Linear, and intrinsically linear models.
Linearity refers to the parameters. The model can involve any function of X’s for as long as they do not have parameters that have to be adjusted.
A linear model does not always produce a hyperplane.
Yi=b0+ b1 f1(Xi1)+…+ bp fp(Xi1)+ei
Polynomial regression.
Is a special case where the functions are powers of X.
September 19, 2018
AGR206

7. Matrix Equations
September 19, 2018
AGR206

8. Extra Sum of Squares Effects of order of entry on SS.
The 4 types of SS.
Partial correlation.
September 19, 2018
AGR206

9. Extra Sum of Squares:body fat
September 19, 2018
AGR206

10. Response plane and error
The response surface in more than 3D is a hyperplane. X1 X2 Yi
E{Yi} Y
September 19, 2018
AGR206

11. Model development What variables to include. Depends on objective:
descriptive -> no need to reduce number of variables.
Prediction and estimation of Yhat: OK to reduce for economical use.
Estimation of b and understanding: sensitive to deletions; may bias MSE and b. No real solution other than getting more data from better experiment. (Sorry!)
September 19, 2018
AGR206

12. Variable Selection Effects of elimination of variables:
MSE is positively biased unless true b for variables eliminated is 0.
bhat and Yhat are biased unless previous condition or variables eliminated are orthogonal to those retained.
Variance of estimated parameters and predictions is usually lower.
There are conditions for which MSE for reduced model (including variance and bias2) is smaller.
September 19, 2018
AGR206

13. Criteria for variable selection
R2 - Coefficient of determination.
R2 = SSReg/SSTotal
MSE or MSRes - Mean squared residuals.
if all X’s in it estimates s2.
R2adj - Adjusted R2.
R2adj = 1-MSE/MSTo = =1-[(n-1)/(n-p)] (SSE/SSTo)
Mallow’s Cp
Cp=[SSRes/MSEFull] + 2 p- n (p=number of parameters)
September 19, 2018
AGR206

14. Example
September 19, 2018
AGR206

15. Checking assumptions.
Note that although we have many X’s, errors are still in a single dimension.
Residual analysis is performed as for SLR, sometimes repeated over different X’s.
Normality. Use proc univ normal option. Transform.
Homogeneity of variance. Plot error vs. each X. Transform. Weighted least squares.
Independence of errors.
Adequacy of model. Plots errors. LOF.
Influence and outliers. Use influence option in proc reg.
Collinearity. Use collinoint option of proc reg.
September 19, 2018
AGR206

16. code for PROC REG data s00.spart2; set s00.spartina;
colin=2*ph+0.5*acid+sal+rannor(23);
run;
proc reg data=s00.spart2;
model bmss= colin h2s sal eh7 ph acid
p k ca mg na mn zn cu nh4 /
r influence vif collinoint stb partial;
run;
model colin=ph sal acid;
September 19, 2018
AGR206

17. Spartina ANOVA output Model: MODEL1 Dependent Variable: BMSS
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Prob>F
Model
Error
C Total
Root MSE R-square
Dep Mean Adj R-sq
C.V
September 19, 2018
AGR206

18. Parameters and VIF September 19, 2018 AGR206 Parameter Estimates
Parameter Standard T for H0: Standardized
Variable DF Estimate Error Parameter=0 Prob > |T| Estimate
INTERCEP
COLIN
H2S
SAL EH PH
ACID P K CA MG NA MN ZN CU NH
Variance
Inflation
September 19, 2018
AGR206