1. PSYC512: Research Methods PSYC512: Research Methods Lecture 12 Brian P. Dyre University of Idaho

2. PSYC512: Research Methods Lecture 12 Outline Within vs. between subjects designs Within vs. between subjects designs Condition ordering Condition ordering Subject assignment Subject assignment More on research design More on research design Using 2 or more groups Using 2 or more groups Multifactor research – using two or more independent variables Multifactor research – using two or more independent variables Experimentation vs. Quasi-experimentation Experimentation vs. Quasi-experimentation

3. PSYC512: Research Methods Features of Experimental Designs Within-subjects design Within-subjects design Most efficient-requires fewest subjects Most efficient-requires fewest subjects Statistically powerful: each subject acts as their own control – eliminates error variance due to subject variables Statistically powerful: each subject acts as their own control – eliminates error variance due to subject variables PROBLEM: Carry-over and ordering effects PROBLEM: Carry-over and ordering effects Between-subjects design Between-subjects design Randomized Groups: if subject variables are not expected to covary with the measure Randomized Groups: if subject variables are not expected to covary with the measure Matched Groups: if subject variables are expected to covary with the measure (some argue randomization is still better) Matched Groups: if subject variables are expected to covary with the measure (some argue randomization is still better)

4. PSYC512: Research Methods Controlling for Carry-over and Order Effects Balance carry-over and order effects across treatments Balance carry-over and order effects across treatments Randomization and Blocked Randomization Randomization and Blocked Randomization Counterbalancing of N treatments Counterbalancing of N treatments Complete: present each subject with a unique order and use every possible order: requires N! orders/Ss Complete: present each subject with a unique order and use every possible order: requires N! orders/Ss Partial (Latin Square): present each subject a unique order carefully chosen from a subset of all possible orders Partial (Latin Square): present each subject a unique order carefully chosen from a subset of all possible orders

5. PSYC512: Research Methods Constructing a Latin Square for N treatments Randomly assign each treatment a number Randomly assign each treatment a number Five treatments (A, B, C, D, E) are assigned (1, 2, 3, 4, 5) Five treatments (A, B, C, D, E) are assigned (1, 2, 3, 4, 5) Determine First Order using: Determine First Order using: 1, 2, N, 3, N-1, 4, N -2… A, B, E, C, D

6. PSYC512: Research Methods Constructing a Latin Square for N treatments Fill in N-1 more orders by incrementing down and “wrapping”. For odd N, also use reverse orders Fill in N-1 more orders by incrementing down and “wrapping”. For odd N, also use reverse orders N = 4 (4 Ss)N = 5 (10 Ss) A B D CA B E C D D C E B A B C A DB C A D EE D A C B C D B AC D B E AA E B D C D A C BD E C A BB A C E D E A D B CC B D A E

7. PSYC512: Research Methods Controlling for Carry-over and Order Effects Minimize carry-over and order effects across treatments Minimize carry-over and order effects across treatments Practice Sessions Practice Sessions Breaks Breaks Make the treatment order a between-subjects independent variable Make the treatment order a between-subjects independent variable Creates a mixed design Creates a mixed design

8. PSYC512: Research Methods Controlling for Subject Variance in Between-subjects Designs Random Assignment Random Assignment Ensures subject characteristics don’t correlate with treatments, if you have enough subjects Ensures subject characteristics don’t correlate with treatments, if you have enough subjects Matched Groups Matched Groups Assess participants on one or more characteristics that might correlate with the DV Assess participants on one or more characteristics that might correlate with the DV Distribute like-participants to groups Distribute like-participants to groups Both methods attempt to equate the groups and treat subject variance as error variance Both methods attempt to equate the groups and treat subject variance as error variance

9. PSYC512: Research Methods Choosing the Number and Type of Independent Variables Single factor (IV) vs. multiple factors (IVs) Single factor (IV) vs. multiple factors (IVs) Single factor designs are simpler but more limited in scope Single factor designs are simpler but more limited in scope Multiple factors allow for examining the synergistic effects of variables (interactions) Multiple factors allow for examining the synergistic effects of variables (interactions) Parametric vs. Non-parametric Designs Parametric vs. Non-parametric Designs Parametric: IV is quantitative (ratio or interval scale) Parametric: IV is quantitative (ratio or interval scale) Non-parametric: IV is qualitative (nominal or ordinal scale) Non-parametric: IV is qualitative (nominal or ordinal scale)

10. PSYC512: Research Methods Factorial Research Designs Used to assess the effects of 2 or more independent variables (factors) on your dependent variable Used to assess the effects of 2 or more independent variables (factors) on your dependent variable Using multiple IVs in one experiment is Using multiple IVs in one experiment is more economical (particularly for within-Ss) more economical (particularly for within-Ss) provides more information provides more information Main effects of each IV (separate effects of each IV) Main effects of each IV (separate effects of each IV) Interaction between the IVs (synergism): effect of one variable changes across the levels of the other variable Interaction between the IVs (synergism): effect of one variable changes across the levels of the other variable

11. PSYC512: Research Methods Factorial Research Designs: Possible Outcomes of a 2 x2 Design DV Factor A A1A2 B1, B2 DV Factor A A1A2 B1, B2 DV Factor A A1A2 B2 No effects Main Effect A Only Main Effect B Only No effects Main Effect A Only Main Effect B Only B1

12. PSYC512: Research Methods DV Factor A A1A2 DV Factor A A1A2 B1 B2 Main Effects for A and B, No Interaction Factorial Research Designs: Possible Outcomes of a 2 x2 Design Main Effects for A and B, Interaction (ordinal) B1 B2 DV Factor A A1A2 B1 B2 DV Factor A A1A2 B1 B2 Main Effect for A only, Interaction (disordinal) Main Effect for B only, Interaction (ordinal) IMPORTANT: always interpret the highest order effect or interaction

13. PSYC512: Research Methods Example Factorial Experiment: Perceiving Heading An experiment on perceiving the heading direction during steering control vs. riding and pointing An experiment on perceiving the heading direction during steering control vs. riding and pointing

14. PSYC512: Research Methods Example Factorial Experiment: Perceiving Heading IV1: observer’s task IV1: observer’s task Steering control: Steer so that you appear to be moving straight ahead Steering control: Steer so that you appear to be moving straight ahead Pointing: Point in the direction towards which you perceive yourself to be moving while “riding” Pointing: Point in the direction towards which you perceive yourself to be moving while “riding” IV2: simulated velocity of observer movement IV2: simulated velocity of observer movement Three levels: 50, 100, and 200 ms -1 Three levels: 50, 100, and 200 ms -1 Dependent Variable (measure): RMS (root-mean- squared) heading error = sqrt(  error /N) Dependent Variable (measure): RMS (root-mean- squared) heading error = sqrt(  error /N)

15. PSYC512: Research Methods Results Significant Effects Task Main Effect Interaction between task and velocity IMPORTANT! ALWAYS INTERPRET HIGHEST ORDER EFFECT

16. PSYC512: Research Methods Specialized Research Designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Quasi-Experimental Designs Quasi-Experimental Designs Pretest-posttest designs Pretest-posttest designs Developmental designs (Longitudinal or cross-sectional) Developmental designs (Longitudinal or cross-sectional)

17. PSYC512: Research Methods Combining Between-Subjects and Within Subjects Designs The Mixed Design The Mixed Design Also known as “split plot” Also known as “split plot” Groups of subjects each receive a unique level of the between Ss variable(s) and all levels of the within Ss variable(s) Groups of subjects each receive a unique level of the between Ss variable(s) and all levels of the within Ss variable(s)

18. PSYC512: Research Methods Mixed Three-factor Design: Adding Presentation Order as a Factor to Account for Differential Carryover Presentation order Variable (between) Group A (control then pointing) Group B (pointing then control) Found 3-way interaction that indicates differential carryover (asymmetrical transfer)

19. PSYC512: Research Methods Combining Between-Subjects and Within Subjects Designs Example of a Mixed Design Example of a Mixed Design Task order manipulated between Ss Task order manipulated between Ss Group A (control then pointing) Group A (control then pointing) Group B (pointing then control) Group B (pointing then control) 3-way interaction indicates differential carryover effect (asymmetrical transfer) 3-way interaction indicates differential carryover effect (asymmetrical transfer)

20. PSYC512: Research Methods Next Time… Quasi-experimental designs Quasi-experimental designs Small-n designs Small-n designs