DOCTORAL SEMINAR, SPRING SEMESTER 2007 Experimental Design & Analysis Analysis of Covariance; Within- Subject Designs March 13, 2007

Outline Blocking vs. analysis of covariance ANCOVA Within-subject designs  Greenwald

Randomized Block Design Completely randomized designs based on assumption that random assignment diminishes systematic bias Blocks are experimental counterparts of covariates  Identify “nuisance” variable  Randomly assign within blocks

Randomized Block Design Example: Experiment about effect of 4 types of training programs on learning  Completely randomized design: N=60, a=4 levels, randomly assign 15 subjects to each of 4 levels  Randomized block design: assume intelligence could confound results, categorize subjects according to IQ or GPA (high, medium, low), assign 20 to each “block” and randomly assign within each block to one of 4 conditions

Randomized Block Design Code block as a factor  Do not want treatment x block interaction More blocks require more subjects

Analysis of Covariance The main purpose of ANCOVA is statistical control of variability when experimental control can not be used  Statistical method for increasing power of ANOVA by reducing mean square error (within-condition error)  ANCOVA can be used to correct for extraneous variables and rule out rival explanations

Analysis of Covariance ANCOVA is like ANOVA on the residuals of the values of the dependent variable, after removing the influence of the covariate, rather than on the original values themselves  In so far as the measures of the covariate are taken in advance of the experiment and they correlate with the measures of the dependent variable they can be used to reduce experimental error  Based on partial correlation analysis

Analysis of Covariance Examples of covariates  If we have a theory about two different methods of learning and our dependent variable is memory, level of education, intelligence, IQ, experience and age may all be covariates  If we have a theory about the effect of management style on firm profitability, some covariates may be firm size, competition, length of CEO tenure  If we have a theory about the relationship between number of outstanding shares and share price, we may consider market capitalization, earnings per share and IPO year as covariates  If we have a theory about the interactive effects of advertising and sales force training, some covariates to consider may be brand equity of product, size of sales force and past advertising

Analysis of Covariance One way of understanding ANCOVA is in terms of deviations from a common regression line  If the regression lines of the dependent variable vs. covariate have same slopes but different intercepts, the effect of the treatment is consistent  If the regression lines of the dependent variable vs. covariate have different slopes, the effect of the treatment depends on the value of the covariate

Analysis of Covariance ANCOVA is based on the sums of squares and sums of products  It is used to test the same hypothesis as standard ANOVA  The only difference is that each of the sums of squares are adjusted on the basis of the covariate variable  The covariate reduces the amount of experimental error  In calculations, reduce degree of freedom by 1 for each covariate

ANCOVA Assumptions Dependent variable continuous Covariate may be continuous (like regression) or discrete (like ANOVA) Factor has discrete levels Variables are normally distributed Relationship is linear

ANCOVA Multiple covariates possible Should be theoretically motivated How much variance does covariate account for?

Within-Subject Designs Advantages of within-subject designs  Economy  Power In fully crossed designs, random assignment permits the assumption of equivalence (subject comparability). In within- subject designs, subjects are the same, thus removing source of unwanted variability and reducing the error term  Usefulness Allows study of behavioral/attitudinal change and learning

Within-Subjects Designs Within-subjects designs involve applying all treatments to the same individuals  Think about within-subjects designs as a 2-way ANOVA where the columns are the treatments and the rows the individuals, with one observation per cell  This design introduces the idea of individuals as a random factor that crosses or intersects other factors in the design  Within-subjects, or repeated measures design, has advantages and disadvantages Advantage is that each individual serves as his or her own control, thus a source on experimental error resulting from individual difference is controlled for Disadvantage is that it introduces dependencies across the treatments, known as carryover effects

Within-Subject Designs: Economy For a 2x2 fully- crossed factorial  N=40 (different subjects in each cell) For a 2x2 within- subject design  N=10 (same 10 subjects in each cell) a1 a2 b1 b2 n=10 a1 a2 b1 b2 n=10

Within-Subject Designs: Power Consider example in which 1 factor (A) is manipulated  Because factor A is completely crossed with subjects factor S, we denote as AxS Analogous to two-factor design AxB in which variability is SS total = SS A + SS B + SS AB + SS error  Examine claim of greater power SS error = SS total – SS effects In one-factor example: SS error = SS total – SS A - SS S

Within-Subject Designs Consider complete within-subjects 2x2 design (each subject sees a 1 b 1, a 1 b 2, a 2 b 1, a 2 b 2 conditions)  Sources of variability: A, B, S, AxB, AxS, BxS, AxBxS  To test effects: compare mean square of effects (A, B, AxB) with mean square for effects with subjects (MS AxS, MS BxS, MS AxBxS )

Within-Subjects Designs Example  Let’s say that we are interested in the effect of different types of exercise on memory We use two treatments, aerobic exercise and anaerobic exercise  In the aerobic condition, participants run in place for five minutes, after which they take a memory test  In the anaerobic condition they lift weights for five minutes, after which they take a different memory test of equivalent difficulty  In a within-subjects design all participants begin by running in place and taking the test, after which the same group of people lift weights and then take the test  We compare the memory test scores in order to answer the question as to what type of exercise most aids memory