1. Single-Variable, Independent-Groups Designs
Graziano and Raulin
Research Methods: Chapter 10
This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.
Copyright © Allyn & Bacon (2007)
Graziano & Raulin (1997)

2. Experimental Design
Tests hypotheses about causal effects of the independent variable (IV)
Includes at least two levels of the IV
Randomly assigns participants to conditions
Includes specific procedures for testing hypotheses
Includes control for the major threats to internal validity
Copyright © Allyn & Bacon (2007)

3. Variance Variance is necessary in any research
Without variance, there is nothing to test
Research designs control unwanted sources of variance in order to evaluate the effects of the independent variable
Copyright © Allyn & Bacon (2007)

4. Forms of Variance Systematic between-groups variance
Experimental variance (due to the IV)
Extraneous variance (due to confounding variables)
Nonsystematic within-groups variance
Due to chance factors and individual differences
We analyze the results of our study using the F-test (ANOVA)
Ratio of between-groups variation to within-groups variation
Copyright © Allyn & Bacon (2007)

5. Controlling Variance Maximizing experimental variance
Make sure that there are real differences between the groups (using a manipulation check)
Controlling extraneous variance
Make sure the groups are as similar as possible at the start of the study
Therefore, the only difference is the IV manipulation
Minimizing error variance
Control with careful measurement or with special designs (e.g., correlated-group designs)
Copyright © Allyn & Bacon (2007)

6. Controlling Variance
The relationship of the various sources of variance to the F ratio is shown here
You want to
Maximize experimental variance
Minimize error variance
Control extraneous variance
Copyright © Allyn & Bacon (2007)

7. Manipulation Check
A specific test of whether the independent variable manipulation actually worked the way it was intended
Example: A study testing the hypothesis that females, but not males, tend to turn anger inward rather than express it externally
Copyright © Allyn & Bacon (2007)

8. Expressed Hostility
The data on the dependent measure suggests that females really do respond with less hostility than males when frustrated
Copyright © Allyn & Bacon (2007)

9. Reported Anger
However, the manipulation check of reported anger suggests that the females may not have been angered by the frustration manipulation
Copyright © Allyn & Bacon (2007)

10. Physiological Arousal
And the second manipulation check of physiological arousal seems to indicate that the report of less anger by the female participants is real
Copyright © Allyn & Bacon (2007)

11. Nonexperimental Designs
Do not include the critical controls of experimental designs
May still be used, but caution is necessary
Four designs covered in this section
Ex post facto design
Single-group, posttest-only design
Single-group, pretest-posttest design
Pretest-posttest, natural control-group design
Copyright © Allyn & Bacon (2007)

12. Ex Post Facto Design A very weak design
What we do when we try to figure out, after the fact, what caused something to happen
Not good science
Does not control confounding variables
Copyright © Allyn & Bacon (2007)

13. Single-Group, Posttest-Only Design
Even with the manipulation, virtually no control over confounding variables
We tend to use an implicit control group (what we think would have happened if there had been no manipulation)
Copyright © Allyn & Bacon (2007)

14. Single-Group, Pretest-Posttest Design
The pretest documents change, but factors other than the treatment could have accounted for the change
History, maturation, regression to the mean, etc.
Copyright © Allyn & Bacon (2007)

15. Pretest-Posttest, Natural Control-Group Design
Copyright © Allyn & Bacon (2007)

16. Pretest-Posttest, Natural Control-Group Design
Like an experiment except that participants are not randomly assigned to the groups
A reasonably strong design except that it does not control for selection
Selection could be a powerful confounding factor in many studies
Copyright © Allyn & Bacon (2007)

17. Experimental Designs Meet all criteria for an experiment
Provide more powerful tests of hypotheses
Designs discussed in this chapter
Randomized, posttest-only, control-group design
Randomized, pretest-posttest, control-group design
Multilevel, completely randomized, between-subjects designs
Solomon’s four-group designs
Copyright © Allyn & Bacon (2007)

18. Randomized, Posttest-Only, Control-Group Design
Copyright © Allyn & Bacon (2007)

19. Randomized, Posttest-Only, Control-Group Design
Random assignment controls for selection
Other confounding variables are controlled by comparing the treatment and no treatment groups
For example, history and maturation should be the same in both groups
Copyright © Allyn & Bacon (2007)

20. Randomized, Pretest-Posttest, Control-Group Design
Copyright © Allyn & Bacon (2007)

21. Randomized, Pretest-Posttest, Control-Group Design
Adding a pretest allows us to quantify the amount of change following treatment
Also allows us to verify that the groups were equal initially
A strong basic research design, with excellent control over confounding
Copyright © Allyn & Bacon (2007)

22. Multilevel, Randomized, Between-Subjects Design
Copyright © Allyn & Bacon (2007)

23. Multilevel, Randomized, Between-Subjects Design
May or may not include a pretest
Multi-group extension of the basic experimental designs
Controls virtually all sources of confounding variables
Copyright © Allyn & Bacon (2007)

24. Solomon’s Four-Group Design
Copyright © Allyn & Bacon (2007)

25. Solomon’s Four-Group Design
Combines two basic experimental designs
Randomized, posttest-only, control-group design
Randomized, pretest-posttest, control-group design
Allows the assessment of an interaction between the pretest and the treatment
Copyright © Allyn & Bacon (2007)

26. Statistical Analysis Issues
If the data are nominal, use chi-square
If the data are ordinal, use the Mann-Whitney U-test (two groups only)
If the data are interval or ratio
If two groups, a t-test of the posttest measures will test the hypothesis
More complex designs require an ANOVA
Copyright © Allyn & Bacon (2007)

27. Analysis of Variance Evaluates differences in group means
It does this evaluation by comparing different variance estimates (termed mean squares)
The F statistic is a ratio of
the mean square between-groups and
the mean square within-groups
The larger the differences between the group means, the greater the F value
Copyright © Allyn & Bacon (2007)

28. Specific Mean Comparisons
A significant F-test means that at least one group is significantly different from at least one other group
If you have more than two groups, you have to do follow-up tests to see which groups differ
Specific mean comparisons can be
Planned comparisons
Post hoc tests
Copyright © Allyn & Bacon (2007)

29. Other Experimental Designs
Other experimental designs covered in later chapters
Correlated-groups designs (Chapter 11)
Within-subjects designs
Matched-subjects designs
Single-subject designs
Factorial designs (Chapter 12)
Many variations on factorial designs are possible
Copyright © Allyn & Bacon (2007)

30. Summary
Research is designed to measure and control sources of variance
There are several non-experimental and experimental designs available
Experimental designs have two elements
Random assignment of participants to conditions
At least one control group
Copyright © Allyn & Bacon (2007)