Quasi-Experimental Designs Slides Prepared by Alison L. O’Malley Passer Chapter 11

Quasi-Experimentation Quasi-experiments resemble experiments, but lack experimental control Generally, lack of random assignment is the key point of distinction between quasi-experiments and “true” experiments (Shadish, Cook, & Campbell, 2002) Quasi-experiments are thus more vulnerable to internal validity threats If quasi-experiments lack experimental control, what good are they?

Quasi-Experimentation: Designs without a control group One-group posttest-only design A treatment occurs and the DV is measured afterward What threats to interval validity are present here?

Quasi-Experimentation: Designs without a control group One-group pretest-posttest design DV measured before and after treatment

Quasi-Experimentation: Designs without a control group Simple interrupted time-series design DV repeatedly measured before and after a treatment History is the primary threat to internal validity

Quasi-Experimentation: Designs with a nonequivalent control group Selection emerges as a major threat to internal validity Selection may interact with other threats (i.e., selection interactions) In such cases, the threat is labeled by replacing the term “selection” with “differential” (e.g., differential attrition, differential testing)

Quasi-Experimentation: Posttest only with nonequivalent control group Participants in one condition exposed to a treatment Participants in the other nonequivalent condition are not exposed to the treatment Outcome measures obtained from both groups Lack of pretests poses difficulties in interpreting results

Quasi-Experimentation: Pretest-posttest with nonequivalent control group Pretreatment and posttreatment scores are obtained for a treatment group and a nonequivalent control group What benefits are added by this research approach?

Quasi-Experimentation: Simple interrupted time-series with nonequivalent control group A series of pre- and posttreatment scores are obtained for a treatment group and a nonequivalent control group

Quasi-Experimentation: Simple interrupted time-series with nonequivalent control group What information can be obtained from examining the pretreatment trend lines?

Switching Replication Designs One group receives a treatment while a nonequivalent group does not receive a treatment but is then exposed to treatment down the road Can be used with both pretest-posttest and time-series designs In the switching replication with treatment removal, the initial treatment group no longer receives the treatment once the control group is switched

Switching Replication with Treatment Removal How might you improve the design of this study?

Program Evaluation Assesses the need for as well as the design, implementation, and effectiveness of a social intervention What is a recent social intervention undertaken in your community?

Program Evaluation Much talk surrounds “evidence-based” programs and public policies How do you know whether a program or policy works?

Program Evaluation: Needs Assessment Needs assessment determines whether there is a need for a social program, and if so, what is required to meet the need Must acquire data from a wide range of sources

Program Evaluation: Program Theory and Design Assessment Rationale for designing a program in a particular way – theoretical and empirical justification

Program Evaluation: Process Evaluation Is program implemented as intended? Also known as program monitoring

Program Evaluation: Outcome Evaluation Likely more comfortable terrain, as this deals with assessing program (treatment) effectiveness If randomized controlled trials aren’t possible, turn to alternative designs

Program Evaluation: Outcome Evaluation Watch out for contamination, which occurs when knowledge, services, or other experiences intended for one group are unintentionally received by another group

Program Evaluation: Efficiency Assessment Cost-benefit analysis of program effectiveness Is the program financially beneficial?

Program Evaluation: Program Diffusion Implementing and maintaining effective programs in other settings or with other groups Dissemination Adoption Implementation Sustainability