Randomized Experimental Design

What is an Experiment? Campbell & Stanley stressed random assignment to experimental treatments. I stress manipulation of the independent variable. Quasi-Experiments: C&S’s term for research where there is a manipulated IV but not random assignment to groups

Random Selection Refers to the selection of data points from a population into a random sample. This selection procedure will be random if each possible sample of size N is equally likely to be sampled. Random samples should be representative of the population. Our inferential statistics assume random sampling.

Note. The tabled values are probabilities. Y is random, X is not. In Y, each time a single score is sampled, all scores in the population are equally likely to be sampled.

Random Assignment Refers to the assignment of subjects to treatment conditions. Allows us to consider the populations (subjects who will get special treatment and those who will not) as equivalent prior to treatment. The samples will likely differ a little.

Two Basic Randomized Designs Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O Randomized Posttest Only Control Group Design R  X  O R     O

Noise Reducing Designs These designs reduce noise (error variance) and thus increase power.

Randomized Blocks Designs Matched pairs, randomized blocks, split-plot. Repeated measures or within-subjects. Variance due to the blocking variable is removed from error variance.

Analysis of Covariance Change a noise-producing extraneous variable into a covariate that is included in the statistical model. Must be able to measure the covariate. Variance due to covariate is removed from the error variance. Can have more than one covariate.

Factorial ANOVA Convert a categorical extraneous variable to an ANOVA factor. Variance due to that factor will be removed from the error term. 2 x 2 Factorial Design R  X11  O R  X12  O R  X21  O R  X22  O

Other Randomized Designs Solomon Four Group Design Controls threats to internal and external validity as well as the posttest only control group design. But has greater power. And greater cost Need more data More complex analysis R  O  X  O R  O     O R     X  O R        O

Solomon Four Group Design ANOVA Arrange all four groups’ posttest scores into a 2 x 2 ANOVA. Pretested or Not x Experimental Treatment or Not. Significant Interaction – Testing x Treatment threat to External Validity Main effect of pretesting. Main effect of treatment

Treatment effect but no testing or Testing x Treatment interaction Pretest means in parentheses

Treatment and testing effects but no Testing x Treatment interaction

Treatment and testing effects and a Testing x Treatment interaction

Solomon Four Group Design Pretest-Posttest Analysis To gain power, analyze the pretest-posttest portion of the design with ANCOV, using pretest scores as covariate Mixed factorial ANOVA planned comparisons using t control versus treatment change scores (independent t) pre versus post for control group (correlated t) pre versus post for treatment group (correlated t)

Randomized Switching- Replications Design R  O  X  O     O R  O     O  X  O Attempt to control social threats to internal validity. Both groups get the special effect, one early, one later. May still be social effects with respect to who gets it first.

Switching Replications Temporary Treatment Effect Group 1 got the treatment first. Treatment is anxiety-reducing drug DV = measure of anxiety reported by patient

Switching Replications Persistent Treatment Effect Treatment is psychotherapy DV = measure of anxiety reported by patient A third posttest could show the effect does not last indefinitely.

Switching Replications Continuing Treatment Effect Treatment = cognitive psychotherapy Anxiety continues to decline beyond the first post-treatment observation, as patients get better at employing the cognitive technique.