1. Controls to Reduce Threats to Validity
Graziano and Raulin
Research Methods: Chapter 9
This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.
Copyright © Allyn & Bacon (2007)
Graziano & Raulin (1997)

2. Threats to Validity Covered in Chapter 8
Validity can be threatened in many ways
Presence of confounding variables
Unrepresentative samples
Inappropriate statistical tests or violations of statistics assumptions
Subject and experimenter effects
All these threats can be controlled
Copyright © Allyn & Bacon (2007)

3. Control Procedures
General control procedures (applicable to virtually all research)
Control over subject and experimenter effects
Control through the selection and assignment of participants
Control through experimental design
Copyright © Allyn & Bacon (2007)

4. General Control Procedures
Preparation of the setting
Free of distractions that might interfere
A natural setting increases external validity
Response Measurement
Use reliable and valid measures
Replication
Demonstrates that findings are consistent and robust
Copyright © Allyn & Bacon (2007)

5. Types of Replication Exact Replication Systematic Replication
Repeating a study using identical procedures to the original
Systematic Replication
Using a theoretical or procedural change
Conceptual Replication
Varying the operational definitions of the variables to get new research hypotheses
Copyright © Allyn & Bacon (2007)

6. Subject and Experimenter Effects
Blind procedures
Best control for expectancy effects
Single-blind: The experimenter does not know what condition the participant is in
Double-blind: Neither the experimenter nor the participant knows what condition the participant is in
Copyright © Allyn & Bacon (2007)

7. Subject and Experimenter Effects
Automation
Reduces contact between participants and the experimenter
Gives the experimenter less opportunity to affect participants
Using objective measures
Objective measure require less judgment
Provides less opportunity for subtle experimenter biases to affect the data
Copyright © Allyn & Bacon (2007)

8. Subject and Experimenter Effects
Multiple observers
Reduces bias because it challenges observers to be as objective as possible
Can measure amount of observer agreement (percent agreement or Kappa)
Using deception
Hides purpose of the study from participants
Balanced placebo design is a good example
Copyright © Allyn & Bacon (2007)

9. Balanced Placebo Design
Separates the pharmacological effects from the expectancy effects of a medicine
A two-factor design
Factor 1 is whether the person takes medication
Factor 2 is whether the person thinks he or she is taking medication
Copyright © Allyn & Bacon (2007)

10. Balanced Placebo Design
This design crosses the taking medication with the belief that medication is being used
People Led to Believe
Taking Medication
Not Taking Medication
Actual Situation
Copyright © Allyn & Bacon (2007)

11. Participant Selection
Can generalize only if your sample is representative
Populations and samples
General population: all potential participants
Target population: those participants you are interested in
Accessible population: portion of target population that is available to the researcher
Sample: drawn from the accessible population
Copyright © Allyn & Bacon (2007)

12. Populations and Sampling
This figure shows the relationship between the various populations
General Population
Target Population
Accessible Population
Sample
Copyright © Allyn & Bacon (2007)

13. Sampling Procedures Random sampling Stratified random sampling
Every participant has an equal chance of being sampled
Stratified random sampling
Random sampling within strata (subgroups)
Ad hoc samples
Random sample from accessible population
Must generalize cautiously
Should describe sample to help define limits of generalization
Copyright © Allyn & Bacon (2007)

14. Participant Assignment
Critical to internal validity
Assignment Procedures
Free random assignment
Random assignment of participants to groups
Matched random assignment
Random assignment of participants in matched sets to groups
Other matching procedures
e.g., match groups on key characteristics
Copyright © Allyn & Bacon (2007)

15. Matched Random Assignment
Match on relevant variables
Variables likely to affect the dependent measure
Variables that show the largest variability in the population
Procedures
Match in sets on the relevant variable
Set size is the number of groups in the study
Randomly assign participants from the set, one to each group
Keep track of matching data for the statistical analysis
Copyright © Allyn & Bacon (2007)

16. Experimental Design Main focus of Chapters 10 through 13
Experimental design maximizes validity
Need to also include the other control procedures covered in this chapter
Key elements of experiments
One or more control groups
Random assignment of participants to groups
Copyright © Allyn & Bacon (2007)

17. Summary
Most threats to validity can be minimized with proper use of control procedures
Broad classes of control procedures
General control procedures
Control over subject and experimenter effects
Control through participant selection and assignment
Control through specific experimental design
Copyright © Allyn & Bacon (2007)