1. Controls to Reduce Threats to Validity
Graziano and Raulin
Research Methods: Chapter 9
This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

2. Copyright © Allyn & Bacon (2010)
Threats to Validity
Covered in Chapter 8
Validity can be threatened in many ways
Presence of confounding variables
Unrepresentative samples
Inappropriate statistical tests or violations of statistics assumptions
Subject and experimenter effects
All these threats can be controlled
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

3. Copyright © Allyn & Bacon (2010)
Control Procedures
General control procedures (applicable to virtually all research)
Control over subject and experimenter effects
Control through the selection and assignment of participants
Control through experimental design
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

4. General Control Procedures
Preparation of the setting
Free of distractions that might interfere
A natural setting increases external validity
Response Measurement
Use reliable and valid measures
Replication
Demonstrates that findings are consistent and robust
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

5. Copyright © Allyn & Bacon (2010)
Types of Replication
Exact Replication
Repeating a study using identical procedures to the original
Systematic Replication
Using a theoretical or procedural change
Conceptual Replication
Varying the operational definitions of the variables to get new research hypotheses
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

6. Subject and Experimenter Effects
Blind procedures
Best control for expectancy effects
Single-blind: The experimenter does not know what condition the participant is in
Double-blind: Neither the experimenter nor the participant knows what condition the participant is in
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

7. Subject and Experimenter Effects
Automation
Reduces contact between participants and the experimenter
Gives the experimenter less opportunity to affect participants
Using objective measures
Objective measure require less judgment
Provides less opportunity for subtle experimenter biases to affect the data
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

8. Subject and Experimenter Effects
Multiple observers
Reduces bias because it challenges observers to be as precise and objective as possible
Can measure amount of observer agreement (percent agreement or Kappa)
Using deception
Hides purpose of the study from participants
Balanced placebo design is a good example
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

9. Balanced Placebo Design
Separates the pharmacological effects from the expectancy effects of alcohol
A two-factor design
Factor 1 is whether the person drinks alcohol
Factor 2 is whether the person thinks he or she is drinking alcohol
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

10. Balanced Placebo Design
This design crosses the consumption of alcohol with the belief that alcohol is being consumed
People Led to Believe
Drinking Alcohol
Not Drinking Alcohol
Actual Situation
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

11. Participant Selection
Can generalize only if your sample is representative
Populations and samples
General population: all potential participants
Target population: those participants you are interested in
Accessible population: portion of target population that is available to the researcher
Sample: drawn from the accessible population
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

12. Populations and Sampling
This figure shows the relationship between the various populations
General Population
Target Population
Accessible Population
Sample
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

13. Copyright © Allyn & Bacon (2010)
Sampling Procedures
Random sampling
Every participant has an equal chance of being sampled
Stratified random sampling
Random sampling within strata (subgroups)
Ad hoc samples
Random sample from accessible population
Must generalize cautiously
Should describe sample to help define limits of generalization
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

14. Participant Assignment
Assignment Procedures
Free random assignment
Random assignment of participants to groups
Randomize within blocks
Randomly assign in blocks of one participant per condition
Matched random assignment
Random assignment of participants in matched sets to groups
Other matching procedures
e.g., match groups on key characteristics
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

15. Matched Random Assignment
Match on relevant variables
Variables likely to affect the dependent measure
Variables that show the largest variability in the population
Procedures
Match in sets on the relevant variable
Set size is the number of groups in the study
Randomly assign participants from the set, one to each group
Keep track of matching data for the statistical analysis
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

16. Copyright © Allyn & Bacon (2010)
Experimental Design
Main focus of Chapters 10 through 13
Experimental design maximizes validity
Need to also include the other control procedures covered in this chapter
Key elements of experiments
One or more control groups
Random assignment of participants to groups
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

17. Copyright © Allyn & Bacon (2010)
Ethical Principles
Balanced Placebo Design raises several ethical issues
Alcohol is a controlled substance
Intoxication poses risks
Some individuals are at especially high risk (e.g., people with certain medical conditions)
Must address these issues
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

18. Screening Participants
Must assure that participants are legally old enough to consume alcohol
Must screen out people who abuse alcohol and those with no experience with alcohol
Must exclude those with medical problems that might be exacerbated by alcohol
Must inform potential participants that alcohol may be consumed, so those with moral objects can decline
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

19. Copyright © Allyn & Bacon (2010)
Deception
Participants know that, depending on the condition, they may or may not be consuming alcohol
They agree to this in the informed consent
What they don’t know is that they may be deceived about the condition to which they are assigned
Debriefing is required to clear up misconceptions
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

20. Protecting Participants
Must have medical safeguards available in the event of an adverse reaction
Must assure that the participant does not drive intoxicated
Usually by keeping them in the lab until the blood alcohol level has dropped
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)

21. Copyright © Allyn & Bacon (2010)
Summary
Most threats to validity can be minimized with proper use of control procedures
Broad classes of control procedures
General control procedures
Control over subject and experimenter effects
Control through participant selection and assignment
Control through specific experimental design
Copyright © Allyn & Bacon (2010)
Graziano & Raulin (1997)