chapter 18 Experimental and Quasi- experimental Research

Chapter Outline Sources of invalidity Threats to internal validity Threats to external validity Controlling threats to internal validity Controlling threats to external validity Types of designs

Experimental Research Tries to Establish Cause and Effect Selection of a good theoretical framework Application of appropriate experimental design Use of correct statistical model and analysis Proper selection and control of independent variables Appropriate selection and measurement of dependent variables Correct interpretation of results

Three Criteria for Cause and Effect 1.The cause must precede the effect in time. 2.The cause and effect must be correlated with each other. 3.The correlation between cause and effect cannot be explained by another variable. If the condition is necessary and sufficient to produce the effect, then it is the cause.

Reviewing Important Terms Independent variable (IV) Dependent variable (dv) Categorical variable Control variable Extraneous variable

Distinguishing Between Types of Validity Internal validity: did the treatments (IV) cause the change in the outcome (dv)? External validity: to what populations, settings, or treatments can the outcome be generalized? Is there a trade-off between internal and external validity? Can a series of studies address the trade- off?

Threats to Internal Validity History: events that are not part of treatment Maturation: events due to passage of time Testing: effects of more than one test administration Instrumentation: change in calibration of measurements Statistical regression: selection based on extreme score (continued)

Threats to Internal Validity (continued) Selection biases: nonrandom participant selection Experimental mortality: differential loss of participants Selection–maturation interaction: passage of time influencing groups differently Expectancy: influence of experimenters on participants

Threats to External Validity Reactive or interactive effects of testing: pretest may make participants sensitive to treatment. Interaction of selection biases and treatment: treatment may work only on participants selected on specific characteristic. Reactive effects of experimental arrangements: setting constraints may influence generalizability. Multiple-treatment interference: one treatment may influence the next treatment.

Controlling Threats to Internal Validity Randomization –Real randomization –Matched pairs (not matched groups) –Randomizing treatments or counterbalancing Placebos Blind setups (continued)

Controlling Threats to Internal Validity (continued) Double-blind setups Reactive effects of testing: eliminate pretest. Instrumentation –Calibration and test reliability –Halo effects Experimental mortality: keeping participants

Controlling Threats to External Validity Selecting from larger population –Participants –Treatments –Situations Ecological validity: does the setting capture the essence of the real world?

Types of Designs: Pre-experimental Designs One-shot studies TO One-group pretest-posttest O 1 TO 2 Statistical analysis? Static group comparison TO Statistical analysis? O 2 (continued)

Types of Designs: True Experimental Designs Randomized-groups design RTO 1 Statistical analysis? RO 2 Extending the levels—randomized-groups design RT 1 O 1 Statistical analysis? RT 2 O 2 RO 3 (continued)

Types of Designs: True Experimental Designs (continued)

Types of Designs: True Experimental Designs A factorial design with a categorical factor (B) RA 1 O 1 B 1 RA 2 O 2 RA 3 O 3 Statistical analysis? RA 1 O 4 B 2 RA 2 O 5 RA 3 O 6 (continued)

Types of Designs: True Experimental Designs Pretest-posttest randomized-groups RO 1 TO 2 RO 3 O 4 Statistical analysis? Extending the design on the RM factor RO 1 TO 2 TO 3 Statistical analysis? RO 4 O 5 O 6 (continued)

Types of Designs: True Experimental Designs Extending the pretest-posttest randomized groups design on both factors RO 1 T 1 O 2 T 1 O 3 RO 4 T 2 O 5 T 2 O 6 RO 7 O 8 O 9 Statistical analysis? (continued)

Types of Designs: True Experimental Designs (continued) Solomon four-group design—purpose RO 1 TO 2 RO 3 O 4 RTO 5 RO 6 Statistical analysis (factorial ANOVA) No treatmentTreatment PretestedO 4 O 2 UnpretestedO 6 O 5

Quasi-Experimental Designs: Time Series D.T. Campbell and J.C. Stanley, Experimental and quasi-experimental designs for research. Copyright © 1963 by Houghton Mifflin Company. Reprinted with permission.

Quasi-Experimental Designs: Reversal

Quasi-Experimental Designs: Ex Post Facto This is one of the pre-experimental designs, but with the treatment not under the control of the experimenter. TO Statistical analysis? O 2

Quasi-Experimental Designs: Switched Replication Levels Trials Ss/Gps O 1 TO 2 O 3 O 4 O 5 2O 6 O 7 TO 8 O 9 O 10 3O 11 O 12 O 13 TO 14 TO 15 4O 16 O 17 O 18 O 19 O 20

Quasi-Experimental Designs: Single Participant Identify participant and follow over time. Does the treatment produce the same effect each time? Are treatment effects cumulative, or does participant return to baseline? Does participant’s response become less variable over treatment times? (continued)

Quasi-Experimental Designs: Single Participant (continued) Is participant’s magnitude of response sensitive to multiple treatment applications? Do varying intensities, frequencies, and lengths of treatment produce varying responses?