Overview of our study of the multiple linear regression model Regression models with more than one slope parameter

Is brain and body size predictive of intelligence? Sample of n = 38 college students Response (y): intelligence based on PIQ (performance) scores from the (revised) Wechsler Adult Intelligence Scale. Potential predictor (x 1 ): Brain size based on MRI scans (given as count/10,000). Potential predictor (x 2 ): Height in inches. Potential predictor (x 3 ): Weight in pounds. Example 1

Scatter matrix plot Example 1

Scatter matrix plot Example 1

Scatter matrix plot Illustrates the marginal relationships between each pair of variables without regard to the other variables. The challenge is how the response y relates to all three predictors simultaneously.

A multiple linear regression model with three quantitative predictors where … y i is intelligence (PIQ) of student i x i1 is brain size (MRI) of student i x i2 is height (Height) of student i x i3 is weight (Weight) of student i Example 1 and … the independent error terms  i follow a normal distribution with mean 0 and equal variance  2.

Some research questions Which predictors – brain size, height, or weight – explain some variation in PIQ? What is the effect of brain size on PIQ, after taking into account height and weight? What is the PIQ of an individual with a given brain size, height, and weight? Example 1

The regression equation is PIQ = Brain Height Weight Predictor Coef SE Coef T P Constant Brain Height Weight S = R-Sq = 29.5% R-Sq(adj) = 23.3% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS Brain Height Weight 1 0.0

Baby bird breathing habits in burrows? Experiment with n = 120 nestling bank swallows Response (y): % increase in “minute ventilation”, Vent, i.e., total volume of air breathed per minute Potential predictor (x 1 ): percentage of oxygen, O2, in the air the baby birds breathe Potential predictor (x 2 ): percentage of carbon dioxide, CO2, in the air the baby birds breathe Example 2

Scatter matrix plot Example 2

Three-dimensional scatter plot Example 2

A first order model with two quantitative predictors where … y i is percentage of minute ventilation x i1 is percentage of oxygen x i2 is percentage of carbon dioxide and … the independent error terms  i follow a normal distribution with mean 0 and equal variance  2. Example 2

Some research questions Is oxygen related to minute ventilation, after taking into account carbon dioxide? Is carbon dioxide related to minute ventilation, after taking into account oxygen? What is the mean minute ventilation of all nestling bank swallows whose breathing air is comprised of 15% oxygen and 5% carbon dioxide? Example 2

The regression equation is Vent = O CO2 Predictor Coef SE Coef T P Constant O CO S = R-Sq = 26.8% R-Sq(adj) = 25.6% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS O CO

Is baby’s birth weight related to smoking during pregnancy? Sample of n = 32 births Response (y): birth weight in grams of baby Potential predictor (x 1 ): smoking status of mother (yes or no) Potential predictor (x 2 ): length of gestation in weeks Example 3

Scatter matrix plot Example 3

A first order model with one binary predictor where … y i is birth weight of baby i x i1 is length of gestation of baby i x i2 = 1, if mother smokes and x i2 = 0, if not and … the independent error terms  i follow a normal distribution with mean 0 and equal variance  2. Example 3

Estimated first order model with one binary predictor The regression equation is Weight = Gest Smoking Example 3

Some research questions Is baby’s birth weight related to smoking during pregnancy? How is birth weight related to gestation, after taking into account smoking status? Example 3

The regression equation is Weight = Gest Smoking Predictor Coef SE Coef T P Constant Gest Smoking S = R-Sq = 89.6% R-Sq(adj) = 88.9% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS Gest Smoking

Compare three treatments (A, B, C) for severe depression Random sample of n = 36 severely depressed individuals. y = measure of treatment effectiveness x 1 = age (in years) x 2 = 1 if patient received A and 0, if not x 3 = 1 if patient received B and 0, if not Example 4

Compare three treatments (A, B, C) for severe depression Example 4

A second order model with one quantitative predictor, a three-group qualitative variable, and interactions where … y i is treatment effectiveness for patient i x i1 is age of patient i x i2 = 1, if treatment A and x i2 = 0, if not x i3 = 1, if treatment B and x i3 = 0, if not Example 4

The estimated regression function Example 4 Regression equation is y = age x x agex agex3

Potential research questions Does the effectiveness of the treatment depend on age? Is one treatment superior to the other treatment for all ages? What is the effect of age on the effectiveness of the treatment? Example 4

Regression equation is y = age x x agex agex3 Predictor Coef SE Coef T P Constant age x x agex agex S = R-Sq = 91.4% R-Sq(adj) = 90.0% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS age x x agex agex Example 4

How is the length of a bluegill fish related to its age? In 1981, n = 78 bluegills randomly sampled from Lake Mary in Minnesota. y = length (in mm) x 1 = age (in years) Example 5

Scatter plot Example 5

A second order polynomial model with one quantitative predictor where … y i is length of bluegill (fish) i (in mm) x i is age of bluegill (fish) i (in years) and … the independent error terms  i follow a normal distribution with mean 0 and equal variance  2. Example 5

Estimated regression function Example 5

Potential research questions How is the length of a bluegill fish related to its age? What is the length of a randomly selected five-year-old bluegill fish? Example 5

The regression equation is length = c_age c_agesq Predictor Coef SE Coef T P Constant c_age c_agesq S = R-Sq = 80.1% R-Sq(adj) = 79.6% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Predicted Values for New Observations New Fit SE Fit 95.0% CI 95.0% PI (160.39, ) (143.49, ) Values of Predictors for New Observations New c_age c_agesq Example 5

The good news! Everything you learned about the simple linear regression model extends, with at most minor modification, to the multiple linear regression model: –same assumptions, same model checking –(adjusted) R 2 –t-tests and t-intervals for one slope –prediction (confidence) intervals for (mean) response

New things we need to learn! The above research scenarios (models) and a few more The “general linear test” which helps to answer many research questions F-tests for more than one slope Interactions between two or more predictor variables Identifying influential data points

New things we need to learn! Detection of (“variance inflation factors”) correlated predictors (“multicollinearity”) and the limitations they cause Selection of variables from a large set of variables for inclusion in a model (“stepwise regression and “best subsets regression”)