PSYC512: Research Methods PSYC512: Research Methods Lecture 11 Brian P. Dyre University of Idaho

PSYC512: Research Methods Lecture 11 Outline Causation and Experimentation Causation and Experimentation Experimental research designs Experimental research designs Reactivity Reactivity Within vs. between subjects designs Within vs. between subjects designs Condition ordering Condition ordering Subject assignment Subject assignment More on research design More on research design Using 2 or more groups Using 2 or more groups Multifactor research – using two or more independent variables Multifactor research – using two or more independent variables Experimentation vs. Quasi-experimentation Experimentation vs. Quasi-experimentation

PSYC512: Research Methods Experimental Designs Manipulate one or more independent variables and observe effect on dependent variable Manipulate one or more independent variables and observe effect on dependent variable Possible to achieve strong internal validity if extraneous variables are carefully controlled  causation can be inferred! Possible to achieve strong internal validity if extraneous variables are carefully controlled  causation can be inferred! Extraneous variables (subject and environmental you aren’t interested in) add error variance Extraneous variables (subject and environmental you aren’t interested in) add error variance Differences in treatments might be due to error variance rather than your manipulations! Differences in treatments might be due to error variance rather than your manipulations!

PSYC512: Research Methods Experimental Research: Reactivity Reactivity and the Hawthorne effect Reactivity and the Hawthorne effect Demand characteristics: cues in experiment that allow a subject to determine the experimenter’s purpose, hypotheses, or expectations Demand characteristics: cues in experiment that allow a subject to determine the experimenter’s purpose, hypotheses, or expectations Good subject role: S produces expected effect Good subject role: S produces expected effect Faithful-subject role (neutral) Faithful-subject role (neutral) Negativistic-subject role: S sabotages expected effect Negativistic-subject role: S sabotages expected effect Countering demand characteristics Countering demand characteristics unobtrusive measures or observations done in the “field” unobtrusive measures or observations done in the “field” Deception Deception Withholding information Withholding information

PSYC512: Research Methods Experimental Designs Goal: minimize the amount of error variance and ensure that it doesn’t correlate with your independent variables Goal: minimize the amount of error variance and ensure that it doesn’t correlate with your independent variables How? How? Reduce error variance Reduce error variance Increase effectiveness (variance) of your IV by choosing more extreme treatments Increase effectiveness (variance) of your IV by choosing more extreme treatments Randomize error variance across groups through random assignment of subjects Randomize error variance across groups through random assignment of subjects Use inferential statistics to estimate the effects of error variance Use inferential statistics to estimate the effects of error variance

PSYC512: Research Methods Features of Experimental Designs Subject Assignment: Within-Subjects (repeated measures) vs. Between Subjects Subject Assignment: Within-Subjects (repeated measures) vs. Between Subjects Number of Independent Variables: Single factor (IV) vs. multiple factors (IVs) Number of Independent Variables: Single factor (IV) vs. multiple factors (IVs) Number of Dependent Variables: Single DV vs. multiple DVs (multivariate) Number of Dependent Variables: Single DV vs. multiple DVs (multivariate)

PSYC512: Research Methods Features of Experimental Designs Factors that determine optimal subject assignment Factors that determine optimal subject assignment If at all possible use a within-subjects design If at all possible use a within-subjects design Most efficient-requires fewest subjects Most efficient-requires fewest subjects Statistically powerful: each subject acts as their own control – eliminates error variance due to subject variables Statistically powerful: each subject acts as their own control – eliminates error variance due to subject variables PROBLEM: Carry-over and ordering effects PROBLEM: Carry-over and ordering effects If significant carry-over and ordering effects are expected then use a between-subjects design If significant carry-over and ordering effects are expected then use a between-subjects design Randomized Groups: if subject variables are not expected to covary with the measure Randomized Groups: if subject variables are not expected to covary with the measure Matched Groups: if subject variables are expected to covary with the measure (some argue randomization is still better) Matched Groups: if subject variables are expected to covary with the measure (some argue randomization is still better)

PSYC512: Research Methods Controlling for Carry-over and Order Effects Balance carry-over and order effects across treatments Balance carry-over and order effects across treatments Randomization and Blocked Randomization Randomization and Blocked Randomization Counterbalancing of N treatments Counterbalancing of N treatments Complete: present each subject with a unique order and use every possible order: requires N! orders/Ss Complete: present each subject with a unique order and use every possible order: requires N! orders/Ss Partial (Latin Square): present each subject a unique order carefully chosen from a subset of all possible orders Partial (Latin Square): present each subject a unique order carefully chosen from a subset of all possible orders

PSYC512: Research Methods Constructing a Latin Square for N treatments Randomly assign each treatment a number Randomly assign each treatment a number Five treatments (A, B, C, D, E) are assigned (1, 2, 3, 4, 5) Five treatments (A, B, C, D, E) are assigned (1, 2, 3, 4, 5) Determine First Order using: Determine First Order using: 1, 2, N, 3, N-1, 4, N -2… A, B, E, C, D

PSYC512: Research Methods Constructing a Latin Square for N treatments Fill in N-1 more orders by incrementing down and “wrapping”. For odd N, also use reverse orders Fill in N-1 more orders by incrementing down and “wrapping”. For odd N, also use reverse orders N = 4 (4 Ss)N = 5 (10 Ss) A B D CA B E C D D C E B A B C A DB C A D EE D A C B C D B AC D B E AA E B D C D A C BD E C A BB A C E D E A D B CC B D A E

PSYC512: Research Methods Controlling for Carry-over and Order Effects Minimize carry-over and order effects across treatments Minimize carry-over and order effects across treatments Practice Sessions Practice Sessions Breaks Breaks Make the treatment order a between-subjects independent variable Make the treatment order a between-subjects independent variable Creates a mixed design Creates a mixed design

PSYC512: Research Methods Controlling for Subject Variance in Between-subjects Designs Random Assignment Random Assignment Ensures subject characteristics don’t correlate with treatments, if you have enough subjects Ensures subject characteristics don’t correlate with treatments, if you have enough subjects Matched Groups Matched Groups Assess participants on one or more characteristics that might correlate with the DV Assess participants on one or more characteristics that might correlate with the DV Distribute like-participants to groups Distribute like-participants to groups Subject attrition could be a problem Subject attrition could be a problem Both methods attempt to equate the groups and treat subject variance as error variance Both methods attempt to equate the groups and treat subject variance as error variance

PSYC512: Research Methods Choosing the Number and Type of Independent Variables Single factor (IV) vs. multiple factors (IVs) Single factor (IV) vs. multiple factors (IVs) Single factor designs are simpler but more limited in scope Single factor designs are simpler but more limited in scope Multiple factors allow for examining the synergistic effects of variables (interactions) Multiple factors allow for examining the synergistic effects of variables (interactions) Parametric vs. Non-parametric Designs Parametric vs. Non-parametric Designs Parametric: IV is quantitative (ratio or interval scale) Parametric: IV is quantitative (ratio or interval scale) Non-parametric: IV is qualitative (nominal or ordinal scale) Non-parametric: IV is qualitative (nominal or ordinal scale)

PSYC512: Research Methods Experimental vs. Quasi- Experimental Research Designs Experimental Research: random assignment of subjects to conditions Experimental Research: random assignment of subjects to conditions Quasi-experimental: assignment of subjects to groups based on a “subject variable” or measured attribute Quasi-experimental: assignment of subjects to groups based on a “subject variable” or measured attribute Treats a DV as an IV in the hope of establishing causality Treats a DV as an IV in the hope of establishing causality May be necessary in the context of field studies May be necessary in the context of field studies Examples: age, income, test-performance Examples: age, income, test-performance Problems Problems Confounding variables may covary with subject variable Confounding variables may covary with subject variable Regression to the mean Regression to the mean

PSYC512: Research Methods Next Time… Multi-factor experimentation Multi-factor experimentation