Chapter 9 Internal Validity PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Overview Problems with two-group designs Problems with pretest-posttest designs Internal versus external validity PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Two-Group Designs The selection problem Unsuccessful approaches to the selection problem PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Sources of Selection Bias Self-assignment to group produces selection bias Researcher assignment to group produces selection bias Arbitrary assignment to group produces selection bias Choosing groups based on their differences results in having groups that are different PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Matching: A Valiant, but Unsuccessful Strategy for Getting Identical Groups The impossibility of perfectly matching individual participants: Identical participants do not exist The difficulty of matching groups on every variable: There are too many variables Two difficulties with matching groups on every relevant variable Too many relevant variables We don’t know what the relevant variables are Problems with matching on pretest scores* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Problems with Matching on Pretest Scores Selection by maturation interactions: participants growing In different ways Regression PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Conclusions about Two-Group Designs Selection is a big problem Trying to solve the selection problem may result in having new problems: Selection by maturation interactions Regression Even if selection, selection by maturation, and regression aren’t problems, mortality can ruin the internal validity of a study. PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Problems with the Pretest-Posttest Design Even without the treatment, participants may change from pretest to posttest* Even if participants don’t change, their scores may change from pretest to posttest* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Three Reasons Participants May Change Between Pretest and Posttest Maturation: Natural physiological changes within the participant may change the participant’s behavior History: The treatment may not be the only thing in the participant’s environment that changes Testing: Measuring participants changes participants PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

How Measurement changes May Cause Scores to Change Between Pretest and Posttest Instrumentation: Changes in how participants are measured Regression revisited: Extreme pretest scores may lead to less extreme posttest scores Mortality: Changes in how many participants are measured PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Conclusions about Establishing Internal Validity Trying to keep everything except the treatment constant is impossible. Only option is to rule out extraneous variables. As you will see in Chapter 10, one way to automatically rule out extraneous variables is to use random assignment. Without random assignment, have to identify extraneous variables and then try to rule them out. Campbell and Stanley’s 8 threats to validity provides a convenient way to identify extraneous variables. PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

The Relationship between Internal and External Validity Attempts to rule out threats to internal validity may hurt external validity Reducing selection threat at the expense of not studying a heterogeneous group of participants Reducing history threat at expense of not studying participants in a naturalistic setting Internal and external validity are not completely incompatible PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Concluding Remarks Be cautious about accepting cause-effect statements--if the study is not an experiment, the study’s internal validity is probably threatened by at least one of Campbell and Stanley’s 8 threats to validity. Experimental designs, as you will soon see, automatically rule out the 8 threats to internal validity. PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley