1. Quasi-Experimental Design10
Quasi-Experimental Design

2. 10.1 Foundations of Quasi-Experimental Design
“Quasi” means “sort of”
Quasi-experiments have:
A control group
A treatment (or program) group
Variables
Quasi-experiments do not have:
Random assignment to groups

3. 10-2 The Nonequivalent Groups Design
One of the most frequently used quasi-experimental designs
Looks just like a pretest-posttest design
Lacks random assignment to groups
As a result, the treatment and control groups may be different at the study’s start
Raises a selection threat to internal validity
Figure 10.2 Notation for the Nonequivalent-Groups Design (NEGD).

4. 10.2a Reaching Cause-and-Effect Conclusions with the NEGD
Figure 10.3 This is a flowchart of a quasi-experimental study to examine the effects of providing job resources and recovery opportunities on the health, well-being, and performance of nursing home staff (Spoor, de Jonge, & Hamers, 2010). Flow charts like this are very helpful in communicating study design.

5. 10.2a Plot of Pretest and Posttest Means for Possible Outcome 1
Figure 10.6 Plot of pretest and posttest means for possible outcome.
Here, there are selection threats. Selection threats are any factor other than the program that leads to posttest differences between groups.
The comparison group started with a lower score on the pretest than the program group, so the groups were not equivalent from the start. The comparison group did not change from pretest to posttest, while the program group did—they improved. This could be due to selection-history effects, selection-maturation effects, or selection-regression effects.

6. 10.2a Plot of Pretest and Posttest Means for Possible Outcome 2
Figure 10.7 Plot of pretest and posttest means for possible outcome 2.
Here, both the program and the comparison group improve from pretest to posttest, but there are still differences between the groups from the start of the study. Selection-testing, selection-instrumentation, and selection-mortality are all plausible explanations for this outcome.

7. 10.2a Plot of Pretest and Posttest Means for Possible Outcome 3
Figure 10.8 Plot of pretest and posttest means for possible outcome 3
Here, a selection-regression effect is the most likely explanation for the results. A selection-regression threat is a threat to internal validity that occurs when there are different rates of regression to the mean in the two groups.
The comparison group starts lower than the program group, and does not change throughout the study. The program group starts out very high on the pretest, and there is a sharp drop in the group’s performance, which is likely due to regression toward the mean.

8. 10.2a Plot of Pretest and Posttest Means for Possible Outcome 4
Figure 10.9 Plot of pretest and posttest means for possible outcome 4.
Here, a selection-regression threat is also a likely explanation. The program group starts out considerably lower than the comparison group, and shows a sharp increase in performance. The comparison group does not change. This is likely because the program group is regressing up toward the mean.

9. 10.2a Plot of Pretest and Posttest Means for Possible Outcome 5
Figure Plot of pretest and posttest means for possible outcome 5.
Here, most threats to internal validity are unlikely, and the treatment is likely causing the result.

10. 10.3 The Regression-Discontinuity Design
A pretest- posttest program comparison- group quasi-experimental design in which a cutoff criterion on the preprogram measure is the method of assignment to a group

11. 10.3a The Basic RD Design Notation
C indicates that groups are assigned by means of a cutoff score on the premeasure
An O stands for the administration of a measure to a group.
An X depicts the implementation of a program
Each group is described on a single line
Figure Notation for the Regression-Discontinuity (RD) design.

12. 10.3a Regression Line
A line that describes the relationship between two or more variables
Figure Pre-post distribution for an RD design with no treatment effect.
Figure The RD design with ten-point treatment effect.

13. 10.3b The Role of the Comparison Group in RD Designs
Figure Regression lines for the data shown in Figure
Figure How the RD design got its name

14. 10.3c The Internal Validity of the RD Design
In principle, then, the RD design is as strong in internal validity as its randomized experimental alternatives
In practice, however, the validity of the RD design depends directly on how well you can model the true pre-post relationship, certainly a serious statistical challenge

15. 10.3d Statistical Power and the RD Design
To achieve the same level of statistical accuracy, an RD design needs as much as times the participants as a randomized experiment
Example: if a randomized experiment needs 100 participants to achieve a certain level of power, the RD design might need as many as 275

16. 10.3e Ethics and the RD Design
RD designs tend to be more ethical, because those who need a program or treatment the most can receive it

17. 10.4a The Proxy Pretest Design
A post-only design in which, after the fact, a pretest measure is constructed from preexisting data
Usually done to make up for the fact that the research did not include a true pretest
Figure The Proxy- Pretest design.

18. 10.4b The Separate Pre-Post Samples Design
A design in which the people who receive the pretest are not the same as the people who take the posttest
Figure The Separate Pre-Post Samples design.

19. 10.4c The Double-Pretest Design
A design that includes two waves of measurement prior to the program
Addresses selection-maturation threats
Figure The Double-Pretest design.

20. 10.4d The Switching-Replications Design
A two-group design in two phases defined by three waves of measurement
In the repetition of the treatment, the two groups switch roles:
The original control group in phase 1 becomes the treatment group in phase 2, whereas the original treatment group acts as the control
Figure The Switching-Replications design.

21. 10.4e The Nonequivalent Dependent Variables (NEDV) Design
At first, looks like a weak design
But pattern matching gives researchers a powerful tool for assessing causality
The degree of correspondence between two data items
Figure The NEDV design.
Figure Example of a Pattern-Matching variation of the NEDV design.

22. 10.4f The Regression Point Displacement (RPD) Design
A pre-post quasi-experimental research design where the treatment is given to only one unit in the sample, with all remaining units acting as controls
This design is particularly useful to study the effects of community level interventions
Figure The Regression Point Displacement (RPD) design

23. 10.4f The Regression Point Displacement (RPD) Design
Figure An example of the RPD design
Figure How the RPD design got its name
ANCOVA: An analysis that estimates the difference between the groups on the posttest after adjusting for differences on the pretest
Analyze with ANCOVA

24. Discuss and Debate
Why can quasi-experiments be more ethical than randomized experiments?
What are the strengths and the weaknesses of quasi-experimental designs?
In a quasi-experiment, those participants who may need a treatment or program the most can be included in the experimental group, whereas in a randomized experiment, they cannot.
Quasi-experimental designs can be preferable to randomized experiments for ethical reasons and also for logistical reasons (sometimes, randomization is not possible due to the research topic or accessible population). However, quasi-experiments have internal validity problems that must be addressed in both the design and the analysis of the study. Sometimes, these designs require more participants than a randomized study, in order to increase statistical power.