Experiments with More Than Two Groups Ryan M. Denney, Ph.D. The University of Southern Mississippi PSY 361

Multiple Groups Design A two-group design can tell you whether your IV has an effect. A multiple-group design is appropriate when you find the answer to your basic question and wish to go further. A multiple-group design compares three or more levels or amounts of an IV. A multiple-group design can have a control group and two or more experimental groups (but does not have to have a control group). We can compare three, four, five, or even more differing levels or amounts of an IV (when interested in impact of an IV) Multiple group designs sometimes compare several groups at only ONE level of the IV (when interested in how groups differ)

Multiple Groups Design Independent groups Groups of participants that are formed by random assignment. Random assignment serves as an important control procedure. You must take into account the large number of participants you will need to make random assignment feasible and to fill the multiple groups. Correlated samples (nonrandom assignment to groups) Matched sets Participants are matched on a variable that will affect their performance on the DV (matching variable). Then sets of participants are created who are essentially the same on the matching variable. You must consider the difficulty of finding three (or more) participants to match on the extraneous variable you choose.

Multiple Groups Design Correlated samples (con’t) Repeated measures Each participant must participate in all of the treatment conditions. That is, each participant must be measured at least three times. Natural sets Analogous to using natural pairs except that sets must include more than two research participants. Many animal researchers use littermates as natural sets (siblings, people belonging to a social group).

Variations on Multiple Groups Design Comparing different amounts/levels of an IV If we already know that a particular IV has an effect, then we can use a multiple-group design to help us define the limits of that effect. Placebo effect An experimental effect that is due to expectation or suggestion rather than the IV. Controlling for the “power of suggestion” E.g., caffeine Ex post Facto Research Deals with “measured” rather than “manipulated” IV’s.

Analyzing Multiple Group Experiments Multiple-groups designs are measured with the analysis of variance (ANOVA). The ANOVA procedure used to analyze a multiple- group design with one IV is known as a one-way ANOVA. (2-way ANOVA is used to analyze differences in groups along 2 IVs) A one-way ANOVA for independent groups is known as a completely randomized ANOVA. A one-way ANOVA for correlated groups is known as a repeated-measures ANOVA.

ANOVA Between-groups variability Variability in DV scores that is due to the effects of the IV. Error variability (within-groups variability) Variability in DV scores that is due to factors other than the IV (individual differences, measurement error, and extraneous variation). An ANOVA compares the ratio of between-groups variability to within-groups variability.

ANOVA Source Table: a table displaying the results of an ANOVA and showing the “source” of different types of variation Sum of Squares Sum of the squared deviations around the mean Represents the amount of variability in the DV attributable to each source (between groups, within groups) Mean Squares The averaged variability of each source Calculated by dividing each source’s sum of squares by its degrees of freedom: N-1 (number of groups or participants minus one) Used to put sums of squares on equal footing—correcting for the fact that only a few groups can contribute to between-group variability and many participants can contribute to within-group variability

ANOVA Source Table Betwn grps 1141.74 2 570.88 4.71 .02 SOURCE SUM OF DF MEAN F PROB SQUARES SQUARES Betwn grps 1141.74 2 570.88 4.71 .02 Within grps 2546.25 21 121.25 Total 3688.00 23 N= 24 Groups= 3

ANOVA When the IV has a significant effect on the DV, the F ratio will be large. When the IV has no effect or only a small effect, the F ratio will be small (near 1).

ANOVA Possible Distributions of Variability in an Experiment. A depicts a large F ratio. B depicts an F ratio of 1.

ANOVA To discern where the significance lies in a multiple-group experiment, we must do additional statistical tests known as post hoc comparisons (also known as follow-up tests). Post hoc comparisons Statistical comparisons made between group means after finding a significant F ratio. Which groups differed significantly from each other? Tukey’s HSD (Honestly Significant Difference): a common post hoc test used to perform pairwise comparisons of group means.