1. Threats to Internal Validity
One of the criteria for making causal inference is to rule out alternative explanations.
Threats to internal validity are confounds that serve as plausible alternative explanations for a research finding.
There are eight general classes of confounds referred to as “threats to internal validity”:
(1)History
(2)Maturation
(3)Testing
(4)Instrumentation
(5)Regression
(6)Selection
(7)Subject attrition
(8)Additive effects with selection
Selection + Maturation
Selection + History
Selection + Instrumentation

2. Threats to Internal Validity
History
Events that occur at the same time as treatment (X) which can change participants’ behavior
Does an AIDS awareness/safe sex campaign on campus influence condom sales in campus vending machines?
History threat:
Suppose at Week 4, Magic Johnson announces he is HIV+.
Can you conclude the campaign was effective?

3. Threats to Internal Validity
Maturation
Participants mature over time which may explain changes in participants
Does a new school reading program improve 2nd graders’ reading comprehension?
Maturation threat:
Reading comprehension improves as children naturally mature over the year.
Can you conclude that the reading program was effective?

4. Threats to Internal Validity
Testing
Taking a test generally affects subsequent testing
Does showing students a new problem solving technique influence their ability to solve problems quickly?
Testing threat:
If similar problems are used in the pretest and posttest, faster problem solving may be due to familiarity with the test.
Can we conclude that teaching the new technique improves problem solving?

5. Threats to Internal Validity
Instrumentation
Instruments used to measure participants’ performance may change over time
Observers may become bored, tired, altered expectations
Modification in the testing setup, mechanical, electronics, room location
Operational definition of dependent variable
Suppose a police protection program is implemented to decrease the incidence of rape as recorded in crime reports.
Instrumentation threat:
At the same time the program is implemented, reporting laws are changed such that what constitutes rape is more inclusive.
Can we conclude that the program is effective (or ineffective)?

6. Threats to Internal Validity
Regression (to the mean): if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement
Participants placed into a group because of past performance
For example: have done poorly on writing assignments
Placed into a program to improve writing
Participants sometimes perform very well or very poorly because of chance factors (not luck, intangibles).
Such as mood, items on a test, motivation
chance factors are not likely to be the same in a second testing
scores will not be so extreme, the scores “regress to the mean” so in this they go up because of change of the intangibles

7. Threats to Internal Validity
Regression
Suppose students were selected for an accelerated enrichment program because of their very high scores on a brief test.
Regression threat:
Scores on the first test could be inflated from chance factors so we can expect their scores to regress to the mean during the second testing.
Can we conclude the program is effective?

8. Threats to Internal Validity
Subject Attrition
Participants are lost from the study (attrition)
group equivalence formed at the start of the study may be destroyed
differences between treatment and control groups at the end of the study may be due to differences in those who remained in each group rather than to the effects of treatment.
Suppose an exercise program is offered to employees who would like to lose weight.
At Time 1, 50 participants sign up (M weight = 225 pounds).
At Time 2, 25 participants remain (25 dropped out). Suppose the 25 who persisted in the program weighed, on average, 150 pounds at Time 2.
Did the exercise program help people to lose weight?

9. Threats to Internal Validity
Selection
Differences exist between individuals in treatment and control groups at the start of the study
A community recycling program is evaluated for effectiveness in reducing amount of garbage going into landfill. Individuals who are interested in recycling are encouraged to participate.
The evaluation consists of weighing the garbage of participants in the recycling program and the garbage of those not participating in the new program.
Can we tell if the new recycling program is effective?

10. Threats to Internal Validity
Additive Effects with Selection
When comparing two groups, experimental and control.
Changes over time in one group but not the other.
Selection + History: when one group of participants responds differently to an external event
Section + Maturation: when one group of participants matures differently
Selection + Instrumentation: when one group of participants is measured more sensitively by a test (instrumentation)

11. Additive Effects with Selection
Section + History
Does an AIDS awareness/safe sex campaign at School A influence condom sales in campus vending machines?
History threat: Suppose at Week 4, Magic Johnson announces he is HIV+.
Can you conclude the campaign at School A was effective?
Yes, both groups should have experienced the same history threat equally.

12. Additive Effects with Selection
Section + History
Does an AIDS awareness/safe sex campaign at School A influence condom sales in campus vending machines?
Additive Effect of Selection and History: Suppose at Week 4, the student newspaper at School A reports about students who are HIV+.
Can you conclude the campaign at School A was effective?

13. Threats to Internal Validity
Important Points to Remember:
When there is no comparison group in the study, the following threats to internal validity must be considered:
History
Maturation
Testing
Instrumentation
Regression
Subject attrition
Selection
When a comparison group is added, the following threats to internal validity must be considered:
Additive effects with selection

14. Problems That Even True Experiments May Not Control
Threats to internal validity that true experiments may not eliminate:
Experimenter expectancy effects
Contamination
communication of information about the experiment between groups of participants
Novelty effects
when an innovation such as new work environment is introduced as the treatment
Hawthorne effect: changes in people’s behavior brought about by the interest that “significant others” show in them.
Because of these problems researchers may have difficulty concluding whether a treatment was effective.

15. Problems That Even True Experiments May Not Control
Expectancy Effects: This occurs when an experimenter unintentionally influences the results of an experiment.
Experimenters can make systematic errors in their interpretation of participants’ performance based on their expectations.
Experimenters can make errors in recording data based on their expectations for participants’ performance.
For example, observer bias

16. Problems That Even True Experiments May Not Control
Contamination: This occurs when there is communication about the experiment between groups of participants.
Three possible outcomes of contamination:
resentment: some participants’ performance may worsen because they resent being in a less desirable condition;
rivalry: participants in a less desirable condition may boost their performance so they don’t look bad; and
diffusion of treatments: control participants learn about a treatment and apply it to themselves.

17. Problems That Even True Experiments May Not Control
Novelty Effects: This refers to changes in people’s behavior simply because an innovation (e.g., a treatment) produces excitement, energy, and enthusiasm
A special case of novelty effects is the Hawthorne effect: performance changes when people know “significant others” (e.g., researchers, company bosses) are interested in them or care about their living or work conditions.
Or disruption effects when the change disrupts the workplace

18. Quasi-Experiments
Quasi- (“resembling”) experiments provide an important alternative when true experiments are not possible.
Quasi-experiments lack the degree of control found in true experiments.
Generally do not have random assignment of participants
Researchers must seek additional evidence to eliminate threats to internal validity in a quasi-experiment.
Take into account possible threats to internal validity for their particular study

19. The One-Group Pretest-Posttest Design
This is a “bad experiment” and is sometimes referred to as a “pre-experimental design.”
Not quasi-experimental design
An intact group is selected for a treatment (e.g., a classroom of children, a group of employees).
A pretest measure is used to record participants’ performance before treatment (O1— or “Observation 1”)
The treatment (X) is implemented.
A posttest measure is used to record performance following the treatment (O2).
Observation Treatment -- Observation 2
O1 X O2

20. The One-Group Pretest-Posttest Design
The one-group pretest-posttest design is a bad experiment because none of the threats to internal validity are controlled.
Any change between pretest (O1) and posttest (O2) scores may be due to the treatment (X) or due to:
History (some other event that coincided with treatment),
Testing (the effects of repeated testing),
Maturation (natural changes in participants over time),
or due to the other threats to internal validity

21. Quasi-Experimental Designs
Three different designs
Nonequivalent Control Group Design
Simple Interrupted Time-Series Design
Time Series with Nonequivalent Control Group Design

22. Quasi-Experimental Designs
Nonequivalent Control Group Design:
a group similar to the treatment group serves as a comparison group
researchers obtain pretest and posttest measures for individuals in both groups.
random assignment to groups is not used
pretest scores are used to determine whether the groups are equivalent

23. FIGURE 10.3 Langer and Rodin (1976) used a nonequivalent control group design to study the effect of two different types of responsibility instructions on the behavior of nursing home residents. Because a “true experiment” was not conducted, the researchers examined features of the study to determine if any threats to internal validity were present.

24. Nonequivalent Control Group
Langer and Rodin (1976) Lack of opportunity to make personal decision contributes to psychological debilitation amongst nursing home residents.
Comparison Groups
(1) Treatment group:
communication stressing independent decision making
were also given a small plant as a gift (if they decided to accept it)
living on one floor of the nursing home
(2) Nonequivalent control group:
communication stressing staff responsibility for them
also received a plant as a gift (whether they chose to have one or not)
living on another floor of the nursing home
Questionnaires about control and happiness
One week before treatment
Three weeks after treatment

25. Nonequivalent Control Group
Suppose group differences are observed at a posttest.
Rule out threats to internal validity:
By adding a comparison group, researchers can rule out threats due to history, maturation, testing, instrumentation, and regression.
We assume that these threats happen the same to both groups, therefore, these threats can’t be used to explain posttest differences.