Design and Control in Experimental Research Lawrence R. Gordon Psychology Research Methods I

Remind me again… 4 Why am I taking this class???

Academic Skills in Psychology Majors 4 Kruger & Zechmeister (2001) 4 Interested in seeing the skills that psychology majors develop during college 4 Graduating senior and first-year psychology majors completed a scale measuring 10 skill areas

Academic Skills 4 Written/oral communication 4 Information gathering 4 Group/team work 4 Interpersonal 4 Behavior management 4 Diversity 4 Critical thinking 4 Methods and statistics 4 Ethics and values 4 Technology and computers

Review 4 Independent variable? –What type is it? 4 Dependent variable? 4 What type of research is this? 4 Cause-effect conclusions?

Results 4 Seniors reported significantly more skill in 7 of the 10 academic skill areas than did the first-years 4 No differences in group/team work, behavior management, or diversity 4 Look at the graph...

Mean Scores in Each Skill Area

Academic Skill Gains? 4 How else could you measure skill gains in psychology majors during college from first-year to senior year? 4 P.S. They also compared Psych majors to majors in natural sciences, social sciences, & humanities. Psych majors had greater exposure in five areas & were most similar to natural science majors.

Research Designs 4 Between-subjects vs. Within-subjects 4 Between-subjects –participants participate in only one condition of the independent variable(s) –their results are compared to other groups in other conditions of the independent variable –another group serves as the control 4 EXAMPLE: Memory Experiment?

Why do we use Between-subjects designs? 4 Often, independent variables are subject variables, and it is impossible to be in more than one level 4 With some manipulated IVs, it is not appropriate to participate in additional conditions after having been in one 4 May sometimes reduce session time

Problems with Between-subjects designs 4 Will need more participants 4 Control groups and experimental groups contain different people –To deal with this, create equivalent groups Random assignment (also block) Matching

Research Designs (cont.) 4 Within-subjects –participants serve in all conditions of the independent variable(s) –their results in one condition are compared to their results in other conditions of the independent variable –hence, participants serve as their own control 4 EXAMPLE: Memory Experiment?

Why use Within-subjects designs? 4 Procedure takes big prep time, but testing is short (E.g., S & P) 4 Participants are hard to come by (small population) 4 Need fewer participants

Problems with Within-subjects designs 4 Sequence or Order effects –Progressive effects (practice, fatigue) –To deal with this Once per S: Counterbalancing: complete or partial Two+ per S: Reverse counterbalancing, block randomization –Carryover effects –To deal with this difficult -- test or do btwn-Ss 4 May increase research time

SPECIAL CASE: Cross-sectional vs. Longitudinal 4 Often want to investigate DVs that you expect may change over time 4 Two ways to do this: 1. Measure different groups of people at one time (b/w-Ss, cross-sectional) 2. Measure the same group of people at each time (w/in-Ss, longitudinal) –Example

Treatment of Juvenile Offenders 4 Why do juveniles act out through crime? 4 Borduin, et al. (1995) 4 Effects of multisystemic therapy (MST) on prevention of criminal activity in juvenile offenders 4 Participants were juvenile offenders ages in Missouri 4 Treatment focused on cognitive and systemic factors associated with offending

Treatment of Juvenile Offenders (cont.) 4 Measured –Individual adjustment (pre-post) psychiatric symptomatology behavior problems –Family relations (pre-post) –Peer relations (pre-post) –Criminal activity (4 years post)

Treatment of Juvenile Offenders (cont.) 4 Results for individuals –decrease in psychiatric symptomatology and in behavior problems –positive changes in family relation on both scale and observation –no differences in peer relations

Treatment of Juvenile Offenders (cont.) 4 Results for criminal activity –lower risks for arrest (see graph) –arrested less often –arrested for less serious offenses –less likely to be arrested for violent crimes (e.g. rape, assault and battery) 4 How about a picture? 

MST vs. IT

MST vs. IT (w/ dropouts)

Problems with longitudinal research? Threats to Validity, again... 4 For longitudinal studies 1. History 2. Testing 3. Instrumentation 4. Regression to the mean 5. Attrition 6. Maturation 7. Participant selection

Threats to Validity for All Designs 4 Experimenter bias –Clever Hans example –To deal with this Use strict (even automated) procedure Use double-blind procedures Use multiple experimenters

Threats to Validity for All Designs (cont.) 4 Participant bias (often from demand characteristics) 1. Cooperative participants being a “good subject” 2. Defensive participants 3. Uncooperative participants “ screw you effect”

Threats to Validity for All Designs (cont.) 4 To deal with this Sometimes appropriate to use mild deception Placebo control group where possible Manipulation checks Field research 4 One final study 

Appreciating difficult methods 4 Shotland & Heinfold (1985) 4 Bystander response to arterial bleeding 4 What did they do? 4 How did they convince people that the victim needed help? 4 Did the people believe the scenario?