29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA1 MRC Cognition and Brain Sciences Unit Graduate Statistics Course

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA2 6: The General Linear Model The General Linear Model and complex designs including Analysis of Covariance IAN NIMMO-SMITH 29 October 2009

MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA3 The General Linear Model A common framework in which we can formulate a wide range of statistical models and experimental designs.

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA4 The General Linear Model... Encompasses: Simple linear regression Multiple regression T-test Analysis of Variance: Between-subjects ANOVA Analysis of Covariance: Between-subjects ANCOVA In all cases there is a single error term. More complex error structures are needed for Repeated Measures designs. (See Lecture 8.)

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA5 GLM and Simple Linear Regression A simple example 12 subjects were timed performing the Block Design subtest of the Wechsler Intelligence Scale (BD) and on the Embedded Figures Test (EFT).

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA6 Simple Linear Regression: The Data Table

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA7 Scatter plot and fitted line

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA8 Simple Linear Regression as GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA9 Simple Linear Regression as GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA10 Simple Linear Regression as GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA11 Simple Linear Regression as GLM Parameter b 1 = Intercept Parameter b 2 = Slope

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA12 Simple Linear Regression as GLM Etc.

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA13 Simple Linear Regression as GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA14 Simple Linear Regression as GLM Etc. To fit the model we need to estimate the parameters and the errors. Estimates are denoted by a ^ (hat) symbol

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA15 Simple Linear Regression as GLM This is done by choosing and so that the sum of the squares of the estimated errors is as small as possible. This is called the Method of Least Squares. is called the Residual Sum of Squares (RSS)

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA16 Simple Linear Regression as GLM We can express the model using matrix notation. X is called the Design Matrix

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA17 Least Squares and GLM The Least Squares estimates are constructed by projecting the data y onto the parameter space

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA18 ANOVA and GLM ANOVA is a way of representing a sequence or family of GLMs additional sum of squares explained or reduction in Residual Sum of Squares t test to assess the contribution of a single column of the design matrix X or parameter b F test to assess the contribution of a group of (related) columns of X these tests have to be considered in the context of other terms in the GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA19 Simple Regression in SPSS

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA20 Simple Regression: nested GLMs We can think of this as the comparison of two model, one of which is a special case of the other, ‘nested’ or hierarchically related. Model(0) is equivalent to Model(1) with b bd (1) constrained to equal 0.

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA21 A way of picturing the models Grand Mean (Intercept) RSS = BD RSS = Add BD Reduction = df =1 Model(0) Model(1)

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA22 Multiple Regression

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA23 Multiple Regression as GLM

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA24 Multiple Regression in SPSS

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA25 Multiple Regression as a sequence of GLMs in SPSS Use ‘Enter’ and ‘Blocks’ to create a sequence of models for comparison. 1 2

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA26 A sequence of Multiple Regressions in SPSS

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA27 The full set of GLMs Grand Mean (Intercept) RSS = BD RSS = NART RSS = BD+NART RSS = NART Reduction = df =1 BD Reduction = df =1 NART (adj. BD) Reduction = 956. df =1 BD (adj. NART) Reduction = df =1 BD+NART Reduction = df =2

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA28 The two Groups t-test as a GLM Two groups of three subjects were tested in two different conditions. Expected value in Group 1 is Expected value in Group 2 is

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA29 One-way Layout Columns are ‘dummy’ or ‘coded’ variable. Alternative ways of representing the levels of a factor No ‘grand mean’. Each column represents the mean for that level of the factor. Column 1 represents effect of level 1 of the factor’. Column 2 represents effect of level 2 above that of level 1. Column 3 represents effect of level 3 above that of level 2.

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA30 Multi-factor Model Additive (no interaction) Factor A has 3 levels Factor B has 2 levels 3 Subjects per group for each of the 3 possible combinations Column 1: A1 Column 2: A2 Column 3: A3 Column 4: B1 Column 5: B2

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA31 Multi-factor Model Non-additive (interaction) Model now includes parameters to express combined effect of A and of B on top of the separate additive effects. Column 1: A1 Column 2: A2 Column 3: A3 Column 4: B1 Column 5: B2 Column 6: A1B1 Column 7: A1B2 Column 8: A2B1 Column 9: A2B2 Column 10: A3B1 Column 11: A3B2

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA32 Analysis of Covariance (1) Common regression 1 intercept 1 slope

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA33 Analysis of Covariance (2) Parallel regressions 2 intercepts 1 slope

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA34 Analysis of Covariance (3) Non-parallel regressions 2 intercepts 2 slopes

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA35 An ANCOVA Example

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA36 Sequences of GLMs in ANCOVA Grand Mean (One Intercept; Zero Slope) STRATEGY (Two Intercepts; Zero Slope) BD (One Intercept; One Slope) STRATEGY + BD (Two Intercepts; One Slope) STRATEGY + BD + STRATEGY x BD (Two Intercepts; Two Slopes) BD (unadj.) BD (adj. STRATEGY) STRATEGY (adj. BD) STRATEGY (unadj.) Heterogeneity of regression on BD

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA37 Types of Sums of Squares: I SPSS Terminology Type I: Sequential A sequence of models, each nested in the next, is fitted and the Additional Sum of Squares for each step in the sequence can be used to test the explanatory contribution of each additional set of variables. Confusingly also known as ‘hierarchical’

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA38 Types of Sums of Squares: II Type II: Hierarchical This is intended for multifactorial designs and each source of variability (main effect, 2nd-order interaction, 3- way interaction …) is adjusted for the contribution of terms of the same or lower order E.g. in a two factor design: Main effect of A (adjusted from B) Main effect of B (adjusted for A) Interaction of A with B (adjusted for A and B)

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA39 Types of Sums of Squares: III Type III: Unique This is SPSS’s default. It represents the portion of variability that can be explained by each variable over and above that explained simultaneously by all the other variables in the model. Variable 1 (adjusted for variables 2,3,4,…) Variable 2 (adjusted for variables 1,3,4,…) Variable 3 (adjusted for variables 1,2,4,…) etc. Can be misleading when variables are correlated as it may appear that several related variables are dispensible. Also can be meaningless in factorial design AB (adjusted for A and B) is OK A (adjusted for B and AB) is ODD

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA40 Types of Sums of Squares: IV Type IV: (approximate) Ignore this; it relates to an approximate method for handling missing data that was used when doing calculation manually before the dawn of the electronic computer...

29 October 2009 MRC CBU Graduate Statistics Lectures 4: GLM: The General Linear Model - ANOVA & ANCOVA41 Next Week :00 Ian Nimmo-Smith will speak on CATEGORICAL DATA ANALYSIS