Regression Models w/ 2 Quant Variables Sources of data for this model Variations of this model Main effects version of the model –Interpreting the regression.

Slides:



Advertisements
Similar presentations
Bivariate &/vs. Multivariate
Advertisements

Canonical Correlation simple correlation -- y 1 = x 1 multiple correlation -- y 1 = x 1 x 2 x 3 canonical correlation -- y 1 y 2 y 3 = x 1 x 2 x 3 The.
Getting More out of Multiple Regression Darren Campbell, PhD.
Lecture 10 F-tests in MLR (continued) Coefficients of Determination BMTRY 701 Biostatistical Methods II.
Automated Regression Modeling Descriptive vs. Predictive Regression Models Four common automated modeling procedures Forward Modeling Backward Modeling.
Research Hypotheses and Multiple Regression: 2 Comparing model performance across populations Comparing model performance across criteria.
ANCOVA Workings of ANOVA & ANCOVA ANCOVA, Semi-Partial correlations, statistical control Using model plotting to think about ANCOVA & Statistical control.
Analytic Comparisons & Trend Analyses Analytic Comparisons –Simple comparisons –Complex comparisons –Trend Analyses Errors & Confusions when interpreting.
Some Terms Y =  o +  1 X Regression of Y on X Regress Y on X X called independent variable or predictor variable or covariate or factor Which factors.
Linear Discriminant Function LDF & MANOVA LDF & Multiple Regression Geometric example of LDF & multivariate power Evaluating & reporting LDF results 3.
Regression Models w/ k-group & Quant Variables Sources of data for this model Variations of this model Main effects version of the model –Interpreting.
Multiple Group X² Designs & Follow-up Analyses X² for multiple condition designs Pairwise comparisons & RH Testing Alpha inflation Effect sizes for k-group.
Introduction to the General Linear Model (GLM) l 1 quantitative variable & 1 2-group variable l 1a  main effects model with no interaction l 1b  interaction.
ANCOVA Workings of ANOVA & ANCOVA ANCOVA, Semi-Partial correlations, statistical control Using model plotting to think about ANCOVA & Statistical control.
Introduction to Path Analysis Reminders about some important regression stuff Boxes & Arrows – multivariate models beyond MR Ways to “think about” path.
Multiple Regression Models Advantages of multiple regression Important preliminary analyses Parts of a multiple regression model & interpretation Differences.
Plots, Correlations, and Regression Getting a feel for the data using plots, then analyzing the data with correlations and linear regression.
Multiple Regression Models: Some Details & Surprises Review of raw & standardized models Differences between r, b & β Bivariate & Multivariate patterns.
Statistics for the Social Sciences Psychology 340 Fall 2006 Putting it all together.
1 Psych 5510/6510 Chapter Eight--Multiple Regression: Models with Multiple Continuous Predictors Part 2: Testing the Addition of One Parameter at a Time.
Analyses of K-Group Designs : Analytic Comparisons & Trend Analyses Analytic Comparisons –Simple comparisons –Complex comparisons –Trend Analyses Errors.
Introduction the General Linear Model (GLM) l what “model,” “linear” & “general” mean l bivariate, univariate & multivariate GLModels l kinds of variables.
Simple Regression correlation vs. prediction research prediction and relationship strength interpreting regression formulas –quantitative vs. binary predictor.
Lecture 24: Thurs. Dec. 4 Extra sum of squares F-tests (10.3) R-squared statistic (10.4.1) Residual plots (11.2) Influential observations (11.3,
Linear Regression and Correlation Analysis
MANOVA LDF & MANOVA Geometric example of MANVOA & multivariate power MANOVA dimensionality Follow-up analyses if k > 2 Factorial MANOVA.
Biol 500: basic statistics
General Linear Models -- #1 things to remember b weight interpretations 1 quantitative predictor 1 quantitative predictor & non-linear component 1 2-group.
Bivariate & Multivariate Regression correlation vs. prediction research prediction and relationship strength interpreting regression formulas process of.
Elaboration Elaboration extends our knowledge about an association to see if it continues or changes under different situations, that is, when you introduce.
Simple Correlation RH: correlation questions/hypotheses and related statistical analyses –simple correlation –differences between correlations in a group.
Introduction the General Linear Model (GLM) l 2 quantitative variables l 3a  main effects model with no interaction l 3b  interaction model l 2 categorical.
An Introduction to Classification Classification vs. Prediction Classification & ANOVA Classification Cutoffs, Errors, etc. Multivariate Classification.
Intro to Parametric Statistics, Assumptions & Degrees of Freedom Some terms we will need Normal Distributions Degrees of freedom Z-values of individual.
Correlation and Regression Analysis
CORRELATIO NAL RESEARCH METHOD. The researcher wanted to determine if there is a significant relationship between the nursing personnel characteristics.
Multiple Linear Regression A method for analyzing the effects of several predictor variables concurrently. - Simultaneously - Stepwise Minimizing the squared.
Statistical hypothesis testing – Inferential statistics II. Testing for associations.
Checking Regression Model Assumptions NBA 2013/14 Player Heights and Weights.
Regression with 2 IVs Generalization of Regression from 1 to 2 Independent Variables.
Correlation and Regression SCATTER DIAGRAM The simplest method to assess relationship between two quantitative variables is to draw a scatter diagram.
Correlational Research Chapter Fifteen Bring Schraw et al.
Coding Multiple Category Variables for Inclusion in Multiple Regression More kinds of predictors for our multiple regression models Some review of interpreting.
ANOVA and Linear Regression ScWk 242 – Week 13 Slides.
Regression Models w/ 2 Categorical Variables Sources of data for this model Variations of this model Definition and advantage of factorial research designs.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 13 Multiple Regression Section 13.3 Using Multiple Regression to Make Inferences.
Multiple Regression BPS chapter 28 © 2006 W.H. Freeman and Company.
Advanced Meta-Analyses Heterogeneity Analyses Fixed & Random Efffects models Single-variable Fixed Effects Model – “Q” Wilson Macro for Fixed Efffects.
© Copyright McGraw-Hill Correlation and Regression CHAPTER 10.
Regression Models w/ 2-group & Quant Variables Sources of data for this model Variations of this model Main effects version of the model –Interpreting.
Plotting Linear Main Effects Models Interpreting 1 st order terms w/ & w/o interactions Coding & centering… gotta? oughta? Plotting single-predictor models.
Analysis of Covariance Combines linear regression and ANOVA Can be used to compare g treatments, after controlling for quantitative factor believed to.
Simple Linear Regression (OLS). Types of Correlation Positive correlationNegative correlationNo correlation.
ANCOVA. What is Analysis of Covariance? When you think of Ancova, you should think of sequential regression, because really that’s all it is Covariate(s)
28. Multiple regression The Practice of Statistics in the Life Sciences Second Edition.
1 Psych 5510/6510 Chapter 14 Repeated Measures ANOVA: Models with Nonindependent Errors Part 1 (Crossed Designs) Spring, 2009.
Multivariate Analysis: Analysis of Variance
4 basic analytical tasks in statistics: 1)Comparing scores across groups  look for differences in means 2)Cross-tabulating categoric variables  look.
ANCOVA Workings of ANOVA & ANCOVA ANCOVA, partial correlations & multiple regression Using model plotting to think about ANCOVA & Statistical control Homogeneity.
General Linear Model What is the General Linear Model?? A bit of history & our goals Kinds of variables & effects It’s all about the model (not the data)!!
Week of March 23 Partial correlations Semipartial correlations
Biostatistics Regression and Correlation Methods Class #10 April 4, 2000.
Linear Discriminant Function Classification vs. Prediction Classification & ANOVA Classification Cutoffs, Errors, etc. Multivariate Classification & LDF.
29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA1 MRC Cognition and Brain Sciences Unit Graduate.
Appendix I A Refresher on some Statistical Terms and Tests.
Stats Methods at IC Lecture 3: Regression.
Plotting Non-linear & Complex Main Effects Models
Introduction to the General Linear Model (GLM)
Quantitative Methods Simple Regression.
Regression and Categorical Predictors
Presentation transcript:

Regression Models w/ 2 Quant Variables Sources of data for this model Variations of this model Main effects version of the model –Interpreting the regression weight –Plotting and interpreting the model Interaction version of the model –Composing the interaction term –Testing the interaction term = testing homogeneity of regression slope assumption –Interpreting the regression weight –Plotting and interpreting the model Plotting more complex models

As always, “the model doesn’t care where the data come from”. Those data might be … 2 measured quant variable (e.g., age & experience) 2 manipulated quant variables (e.g., 4, 16, 32, 64 practices & % receiving feedback 0, 25, 50, 75, 100) a measured quant variable (e.g., age) and a manipulated quant variable (e.g., 4, 16, 32, 64 practices ) Like nearly every model in the ANOVA/regression/GLM family – this model was developed for and originally applied to experimental designs with the intent of causal interpretability !!! As always, causal interpretability is a function of design (i.e., assignment, manipulation & control procedures) – not statistical model or the constructs involved !!!

There are two important variations of this model 1.Main effects model Terms for both quant variables No interaction – assumes regression slope homgeneity b-weights for the 2 quant variables each represent main effect of that variable 2. Interaction model Terms for both quant variables Term for interaction - does not assume reg slp homogen !! b-weights for 2 quant variables each represent the simple effect of that variable when the other variable = 0 b-weight for the interaction term represented how the simple effect of one variable changes with changes in the value of the other variable (e.g., the extent and direction of the interaction)

a  regression constant expected value of Y if all predictors = 0 Models with 2 centered quantitative predictors y’ = b 1 X 1 + b 2 X 2 + a This is called a main effects model  there are no interaction terms. b 1  regression weight for centered quant predictor X 1 expected direction and extent of change in Y for a 1-unit increase in X 1 after controlling for the other variable(s) in the model main effect of X 1 Slope of Y-X 1 regression line for all values of X 2 b 2  regression weight for centered quant predictor X 2 expected direction and extent of change in Y for a 1-unit increase in X 2 after controlling for the other variable(s) in the model main effect of X 2 Height difference of Y-X 1 regression line for different values of X 2 By the way – we can plot this model with either X 1 or X 2 on the x-axis and plot that Y-X 2 regression line for different values of X 1 – our choice

Models with 2 centered quantitative predictors y’ = b 1 X 1 + b 2 X 2 + a This is called a main effects model  there are no interaction terms. Same idea … we want to plot these models so that we can see how both of the predictors are related to the criterion Different approach … when the second predictor was binary or had k-categories, we plotted the Y-X regression line for each Z group now, however, we don’t have any groups – both the X 1 & X 2 variables are centered quantitative variables what we’ll do is to plot the Y-X 1 regression line for different X 2 values We’ll plot 3 lines the most common approach is to plot the Y-X 1 regression line for… the mean of X 1 +1 std above the mean of X 1 -1 std below the mean of X 1

y’ = b 1 X + b 2 *0 + a y’ = b 1 X + a y’ = b 1 X + b 2 *std + a y’ = b 1 X + ( b 2 *std + a) To plot the model we need to get separate regression formulas for each chosen value of X 2. Start with the multiple regression model.. For X 2 = 0 (the mean of centered Z) Substitute the 0 in for X 2 Simplify the formula For X 2 = +1 std Substitute the std value in for X 2 Simplify the formula Model  y’ = b 1 X + b 2 X 1 + a slope height y’ = b 1 X + -b 2 *std + a y’ = b 1 X + (-b 2 *std + a) For X 2 = -1 std Substitute the std value in for X 2 Simplify the formula slope height slope

y’ = b 1 X 1cen + b 2 X 2cen + a a b1b1 b2b2 X 2 =0 +1 std X 2 -1 std X 2 b2b2 X 2cen = X 2 – X 2mean a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 -lines X 1cen = X 1 – X 1mean Plotting & Interpreting Models with 2 centered quantitative predictors This is called a main effects model  no interaction  the regression lines are parallel. 0 slp = +1std slp = -1std slp No interaction  X 1cen

a b 1 = 0 -b 2 Plotting & Interpreting Models with 2 centered quantitative predictors This is called a main effects model  no interaction  the regression lines are parallel. y’ = b 1 X 1cen + b 2 X 2cen + a X 2cen = X 2 – X 2mean X 1cen = X 1 – X 1mean a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 -lines 0 slp = +1std slp = -1std slp No interaction X 2 =0 +1 std X 2 -1 std X  X 1cen

aa -b 2 -b  X 1cen Plotting & Interpreting Models with 2 centered quantitative predictors This is called a main effects model  no interaction  the regression lines are parallel. y’ = b 1 X 1cen + b 2 X 2cen + a X 2cen = X 2 – X 2mean X 1cen = X 1 – X 1mean a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 -lines 0 slp = +1std slp = -1std slp No interaction X 2 =0 +1 std X 2 -1 std X 2

In SPSS you have to compute the interaction term – as the product of the 2 centered quantitative variable So, if you have exp_cen centered at its mean and age_cen centered at its mean, you would compute the interaction as… compute exp_age_int = exp_cen * age_cen. Models with Interactions As in Factorial ANOVA, an interaction term in multiple regression is a “non-additive combination” there are two kinds of combinations – additive & multiplicative main effects are “additive combinations” an interaction is a “multiplicative combination”

Testing the interaction/regression homogeneity assumption… There are two equivalent ways of testing the significance of the interaction term: 1.The t-test of the interaction term will tell whether or not b=0 2.A nested model comparison, using the R 2 Δ F-test to compare the main effect model (2 centered quant variables) with the full model (also including the interaction product term) These are equivalent because t 2 = F, both with the same df & p. Retaining H0: means that the interaction term does not contribute to the model, after controlling for the main effects which can also be called regression homogeneity.

Interpreting the interaction regression weight If the interaction contributes, we need to know how to interpret the regression weight for the interaction term. We are used to regression weight interpretations that read like, “The direction and extent of the expected change in Y for a 1-unit change in X, holding all the other variables in the model constant at 0.” Remember that an interaction in a regression model is about how the slope between the criterion and one predictor is different for different values of another predictor. So, the interaction regression weight interpretation changes just a bit… An interaction regression weight tells the direction and extent of change in the slope of one Y-X regression line for each 1-unit increase in the other X, holding all the other variables in the model constant at 0. Notice that in interaction is about regression slope differences, not correlation differences – you already know how to compare corrs

Interpreting the interaction regression weight, cont. Like interactions in ANOVA, interactions in multiple regression tell how the relationship between the criterion and one variable changes for different values of the other variable – i.e., how the simple effects differ. Just as with ANOVA, we can pick either variable as the simple effect, and see how the simple effect of that variable is different for different values of the other variable. The difference is that in this model, both variables are quantitative variables (X 1 & X 2 ) So, we can describe the interaction in 2 different ways – both from the same interaction regression weight! how does Y-X 1 regression line slope differ as X 2 increases by 1 how does Y-X 2 regression line slope differ as X 1 increases by 1

Interpreting the interaction regression weight, cont. Example: perf’ = 8.2*#pract + 4.5*Age + 4.0Pr_Age We can describe the interaction regression weight 2 ways: 1.The expected direction and extent of change in the Y-X 1 regression slope for each 1-unit increase in X 2, holding… The slope of the performance-practice regression line increases by 4 with each 1-unit increase in age. 2. The expected direction and extent of change in the Y-X 2 regression slope for each 1-unit increase in X 1, holding… The slope of the performance-age regression line increases by 4 with each 1-unit increase in practice.

Interpreting the interaction regression weight, cont. perf’ = 8.2*#pract + 4.5*Age + 4.0Pr_AGE The slope of the performance-practice regression line for those with feedback (coded 1) has a slope 4 more than the slope of the regression line for those without feedback (coded 0). Be sure to notice that it says “more” -- it doesn’t say whether all are positive, negative or some of each !!! Both of the plots below show a + interaction regression weight. Mean +1 std -1 std Mean +1 std -1 std

Models with 2 centered quantitative predictors & their Interaction y’ = b 1 X 1 + b 2 X 2 + b 3 X 1 X 2 + a Same idea … we want to plot these models so that we can see how both of the predictors are related to the criterion Different approach … when the second predictor was binary or had k-categories, we plotted the Y-X regression line for each Z group now, however, we don’t have any groups – both the X 1 & X 2 variables are centered quantitative variables what we’ll do is to plot the Y-X 1 regression line for different X 2 values We’ll plot 3 lines the most common approach is to plot the Y-X 1 regression line for… the mean of X 1 +1 std above the mean of X 1 -1 std below the mean of X 1

To plot the model we need to get separate regression formulas for each chosen value of X 2. Start with the multiple regression model.. y’ = b 1 X 1 + b 2 X 2 + b 3 X 1 X 2 + a Model  Gather all “Xs” together Factor out “X” y’ = (b 1 + b 3 X 2 )X 1 + (b 2 X 2 + a) slope height We will apply this reconfigured model with three different values of X 2, to get three Y-X regression lines to plot

y’ = (b 1 + b 3 0)X 1 + (b a) y’ = b 1 X 1 + a y’ = (b 1 + b 3 std)X 1 + (b 2 std + a) To plot the model we need to get separate regression formulas for each chosen value of X 2. Start with the multiple regression model.. For X 2 = 0 (the mean of centered X 2 ) Substitute the 0 for X 2 Simplify the formula For X 2 = +1 std Substitute the std for X 2 Simplify the formula slope height For X 2 = -1 std Substitute the -std for X 2 Simplify the formula slope y’ = (b 1 + b 3 X 2 )X 1 + (b 2 X 2 + a) height y’ = (b 1 + b 3 *-std)X 1 + (b 2 *-std + a) slope height

a b1b1 -b 2 b2b  X 1cen X 1 X 2 = X 1cen * X 2cen b3b3 -b 3 Plotting & Interpreting Models with 2 centered quantitative predictors & their Interaction X 2cen = X 2 – X 2mean X 1cen = X 1 – X 1mean y’ = b 1 X 1 + b 2 X 2 + b 3 X 1 X 2 + a a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 lines at X 1 = 0 b 3 = htdifs among X 2 lines X 2 =0 +1 std X 2 -1 std X 2

a b1b1 b 2 =  X 1cen -b 3 b3b3 X 2 =0 +1 std X 2 -1 std X 2 a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 lines at X 1 = 0 b 3 = htdifs among X 2 lines X 1 X 2 = X 1cen * X 2cen Plotting & Interpreting Models with 2 centered quantitative predictors & their Interaction X 2cen = X 2 – X 2mean X 1cen = X 1 – X 1mean y’ = b 1 X 1 + b 2 X 2 + b 3 X 1 X 2 + a

a b2b2 -b 2 b1b  X 1cen b3b3 -b 3 a = ht of X 2mean line b 1 = slp of X 2mean line b 2 = htdifs among X 2 lines at X 1 = 0 b 3 = htdifs among X 2 lines X 1 X 2 = X 1cen * X 2cen Plotting & Interpreting Models with 2 centered quantitative predictors & their Interaction X 2cen = X 2 – X 2mean X 1cen = X 1 – X 1mean y’ = b 1 X 1 + b 2 X 2 + b 3 X 1 X 2 + a X 2 =0 +1 std X 2 -1 std X 2

So, what do the significance tests from this model tell us and what do they not tell us about the model we have plotted? We know whether or not the slope of the Y-X 1 regression line = 0 for the mean of X 2 (t-test of the X 1 weight). We know whether or not the slope of the Y-X 1 regression weight changes with the value of X 2 (t-test of the interaction term weight). But, there is no t-test to tell us if the slope of the Y-X 1 regression line = 0 for any other value of X 2. We know whether or not the height of the Y-X 1 regression line is different for different values of X 2 when X 1 = 0 (t-test of the X 2 weight). But, there is no test of the Y-X 1 regression line height difference at any other value of X 2.