Introduction the General Linear Model (GLM) l what “model,” “linear” & “general” mean l bivariate, univariate & multivariate GLModels l kinds of variables l some common models

“General Linear Model” “Model” means that we are usually interested in predicting or “modeling” the values of one variable (criteria) from the values of one or more others (the predictors) “Linear” means that the variables will be “linearly transformed (* & /) and “linearly combined” (+ & -) to produce the model’s estimates “General” means that the model intends to provide a way to model & test RHs: about any combination of criterion and predictor variables (i.e., any model), and to test RHs: about comparisons among models

Regression “vs.” GLM The “constant” is often represented differently in GLM than in multiple regression … Single predictor models  single predictor regression y’ = bx + a  single predictor GLM y’ = b 0 + b 1 x 1 Multiple predictor models  multiple predictor regression y’ = b 1 x 1 b 2 x 2 + a  multiple predictor GLM y’ = b 0 + b 1 x 1 + b 2 x 2

Common kinds of GLModels Bivariate  one criterion & one predictor  simple regression y’ = b 0 + b 1 x Univariate  one criterion & multiple predictors  multiple regression in all its forms y’ = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 Multivariate  multiple criterion & multiple predictors  canonical regression in all its forms b 0 + b 1 y 1 + b 2 y 2 = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3

Common kinds of variables Quantitative variables Raw variable Centered variables X – mean Mean  0 simplifies math of more complicated models Re-centered variables X  a more meaningful value Change “start” or “stop” values E.g., “aging & intellectual decline” Mathematical “trick” to get the desired model/weights selecting which group or value will be represented in model’s bs

Common kinds of variables Quadratic quantitative variables X 2 – actually represents combination of linear + quadratic X cen 2 – represents the “pure” quadratic term Model with X 2 will have ≈ R 2 as model with X cen + X cen 2 A model with a quadratic term should always include the linear term for that variable

Common kinds of variables 2-group variables Unit coding (usually 1-2) Dummy Coding “control” or “comparison” group coded 0 “treatment” or “target” group coded 1 Effect Coding “control” or “comparison” group coded -1 “treatment” or “target” group coded 1

Common kinds of variables k-group variables Raw coding (usually 1-2-3, etc.) Dummy Coding “control” or “comparison” group coded 0 “treatment” or “target” groups coded 1 on one variable & 0 on all others the full set of codes must be included in the model Effect Coding “control” or “comparison” group coded -1 “treatment” or “target” group coded 1 on one variable & 0 on all others the full set of codes must be included in the model

Common kinds of variables K-groups variables, cont. Comparison coding Combining simple and complex analytical comparison codes to represent specific, hypothesis driven, group comparisons E.g., Say you have 4 groups and RH: that… Group 1 has higher scores that the average scores of groups 2-4 the codes would be gp1 = 3 gp2 = -1 gp3 = -1 gp4 = -1 Groups 2 & 3 have higher average scores than do 1 & 4 the codes would be gp1 = -1 gp2 = 1 gp3 = 1 gp4 = -1 Group 2 has higher scores than the average scores of groups 3-4 the codes would be gp1 = 0 gp2 = 2 gp3 = -1 gp4 = -1 Usually havea set of k-1 codes

Common kinds of variables K-groups variables, cont. Polynomial coding If the groups represent a quantitative continuum, you use codes to represent different polynomial functions (linear, quadratic, cubic, etc.) to explore the shape of the relationship between that variable and the criterion E.g., for a 5-group variable, the polynomial codes are … Linear Quadratic Cubic Quartic the full set of codes must be included in the model

Common kinds of variables Ordered-category variables Sometimes you have a quantitative variable that you want to change into a set of ordered categories e.g.  % grade into “A” “B” “C” “D” “F” e.g.  % grade into “Pass” “Fail” e.g.  aptitude test scores into “remedial” “normal” “gifted” Sometimes this is done to help with “ill-behaved distributions” e.g.  frequency variable with mean=1.1, std=8.4, sk=4.2 e.g.  frequency variable with 60% “0” 38% “1” max = 118 Important because  skewed univariate distributions can “create” apparently nonlinear bivariate relationships

Common kinds of variables Ordered-category variables, cont. Once you form the ordered categories (using “IF,” “RECODE” or other transformations), you can enter those variables into the GLM in different ways Using the category values (e.g., 1, 2, 3, etc) *** Centering or re-centering the category values *** Dummy codes of the category values Effect codes of the category values Polynomial codes of the category values*** *** indicates approaches that make assumptions about the interval nature of the variable and/or its normal distribution, with which not everyone agrees!

Common kinds of variables Interactions Interactions represent the “joint effect” or “non-additive combination” of 2 or more predictors as they relate to a criterion (or set of criteria in the multivariate case). They are the “moderation,” “it depends,” “sometimes,” or “maybe” that makes our science and statistical analyses so interesting. Interactions can be formed from the combination of any 2 or more variables of the types just discussed. There are some “guidelines” about forming, including and interpreting interaction terms.

Common kinds of variables Interaction  “Guidelines” When including a 2-way interaction, both related main effects must be included When including a 3-way interaction, all 3 main effects and all 3 2-way interactions must be included When including a non-linear interaction term, the related linear and nonlinear main effects, and linear interaction terms must be included The associated terms can not exceed the df of the variables involved (except for quantitative variables)

Common kinds of GLModels “Linear” Multiple regression models y’ = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 Can include any of the variable types: Quantitative (raw, centered or re-centered) 2- or k-group (with dummy, effect, or comparison coding) Ordered category (coded)

Common kinds of GLModels “Non-Linear” Multiple regression models with quant variables y’ = b 0 + b 1 x 1 + b 2 x b 3 x b 4 x 2 2 Can include any of the variable types: Linear terms should be centered Non-linear terms should be centered then powered Non-linear terms above quadratic should be based on theory Include linear term for all non-linear terms, at least at first

Common kinds of GLModels “2-way Interaction” Multiple regression models y’ = b 0 + b 1 x + b 2 z + b 3 xz Can include any of the variable types: Quantitative variables should be centered 2- or k-group variables should be coded Interaction terms formed as product of main effect terms Must included main effects terms for any interaction variable

Common kinds of GLModels “3-way Interaction” Multiple regression models y’ = b 0 + b 1 x + b 2 z + b 3 v + b 4 xz + b 5 xv + b 6 zv + b 7 xzv Can include any of the variable types: Quantitative variables should be centered 2- or k-group variables should be coded Interaction terms formed as product of main effect terms Must included main effects terms for any interaction variable

Common kinds of GLModels 2-group ANOVA models y’ = b 0 + b 1 x 1 “X” is a dummy or effect coded 2-group variable

Common kinds of GLModels 3-group ANOVA models y’ = b 0 + b 1 x 1 + b 2 x 2 “X 1 ” & “X 2 ” are a dummy or effect codes for a 3-group variable

Common kinds of GLModels 4-group ANOVA models y’ = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 “X 1,” “X 2 ” & “X 3 ” are a dummy or effect codes for a 4-group variable

Common kinds of GLModels 2x2 Factorial ANOVA model y’ = b 0 + b 1 x + b 2 z + b 3 xz “X” is a dummy or effect code of 1 st 2-group variable “Z” is a dummy or effect code of 2 nd 2-group variable “XZ” represents the interaction of “X” and “Z”

Common kinds of GLModels 2x3 Factorial ANOVA model y’ = b 0 + b 1 x 1 + b 2 z 1 + b 3 z 2 + b 4 xz 1 + b 5 xz 2 “X 1 ” is a dummy or effect code of 1 st 2-group variable “Z 1 ” & “Z 2 ” are dummy or effect codes of 2 nd k-group variable “XZ 1 ” & “ZX 2 ” represent the interaction of “X” and “Z”

Common kinds of GLModels 2-group ANCOVA models y’ = b 0 + b 1 x + b 2 z “X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative)

Common kinds of GLModels 2-group ANCOVA models with covariate interaction y’ = b 0 + b 1 x + b 2 z + b 3 xz “X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative) “XZ” represents the interaction of “X” and “Z”

Common kinds of GLModels 3-group ANCOVA models y’ = b 0 + b 1 x 1 + b 2 x 2 + b 3 z “X 1 ” & “X 2 ” are a dummy or effect codes for a 3-group variable “Z” is the covariate (dummy coded or quantitative)

Common kinds of GLModels 3-group ANCOVA model with covariate interaction y’ = b 0 + b 1 x 1 + b 2 x 2 + b 3 z + b 4 xz 1 + b 5 xz 2 “X 1 ” & “X 2 ” are a dummy or effect codes for a 3-group variable “Z” is the covariate (dummy coded or quantitative) “XZ 1 ” & “ZX 2 ” represent the interaction of “X” and “Z”

Common kinds of GLModels 2x2 Factorial ANCOVA model y’ = b 0 + b 1 x + b 2 z + b 3 xz + b 4 v “X” is a dummy or effect code of 1 st 2-group variable “Z” is a dummy or effect code of 2 nd 2-group variable “XZ” represents the interaction of “X” and “Z” “V” represents the covariate

Common kinds of GLModels 2x2 Factorial ANCOVA model with covariate interactions y’ = b 0 + b 1 x + b 2 z + b 3 xz + b 4 v + b 5 xv + b 6 zv + b 7 xzv “X” is a dummy or effect code of 1 st 2-group variable “Z” is a dummy or effect code of 2 nd 2-group variable “XZ” represents the interaction of “X” and “Z” “V” represents the covariate “XV” represents the interaction of “X” and “V” “ZV” represents the interaction of “Z” and “V” “XZV” represents the interaction of “X,” “Z” and “V”