1. Chapter 8 Experimental Design The nature of an experimental design
Pre-experimental models
True experimental models
Threats to validity

2. The Nature of Experimental Design
Changes in the independent variable (IV) result in (cause) changes in the dependent variable (DV); causation may be improperly attributed.
True experiments can address all three parts of causality

3. Methodology
Stimulus (or treatment): the independent variable in experiments; X
Observations: the dependent variables represented in experimental models; O
Procedures

4. A True Experiment A true experiment requires two things:
Random assignment
Control group

5. After Only Design
An experimental design in which a single measurement of the observation is taken, after the application of the stimulus; measurement of the DV is taken after the application of the IV

6. Random Assignment
Distribute available subjects into groups in some systematic way in an attempt to create equivalent groups

7. Control Group
The stimulus (treatment) group is subjected to some experience (the independent variable)
The control group has a similar experience to the stimulus group, but no exposure to the independent variable

8. 2 Primary Classes of Experiments
Pre-experimental models
True experimental models

9. Pre-Experimental Models
Case study
Simple pre-post
Static group comparison
In the models:
X = the IV, stimulus, treatment
O = the DV, usually a measurement

10. Case Study
X O
Most commonly used in examining naturally occurring events or phenomena; something happens (X), we measure its impact (by observation, O)
Sometimes referred to as firehouse research

11. Simple Pre-post (without control)
O1 X O2
We anticipate that an event will occur
Observations (O1 and O2) are made before and after the event (stimulus), X

12. Static Group Comparison
X O1 O2
The researcher separates a single group (O) into two parts (O1 and O2) based on some criteria. Differences are assessed based on exposure to the stimulus, X. O1 experiences the stimulus, O2 does not.

13. True Experimental Models
The researcher has control over who will get the stimulus
There is random assignment to experimental groups and a control group

14. True Experimental Models
After only design
Classic experimental model
Solomon 4-groups design

15. After Only Design R X O1 R O2
Two lines indicates two groups; the R indicates random assignment
The first group has the treatment (X) and an observation/measurement (O1)
The second group has no treatment, just an observation (O2), making it the control group

16. Classic Experimental Model
Sometimes, we want to be certain that the groups are equivalent in levels of the dependent variable
A pretest at time 1 (T1) and a posttest at time 2 (T2) allow us to confirm that random assignment created equivalent groups
However, every measurement potentially changes the phenomenon we are trying to observe

17. Classic Experimental Model
T T2
R O1 X O2
R O O4
Each group is measured, observed (O1, O2, O3, O4) before (T1) and after (T2) the experimental group receives treatment (X)

18. Solomon 4-Group Design
After only design and classical experimental design combined
Four randomly assigned groups, two stimulus groups, two control groups, six observations (two pretests and four posttests)
Rarely used

19. Solomon 4-Group Design T1 T2 R O1 X O2 R O3 O4 R X O5 R O6
Combines every possible configuration of after only design and classical experimental design

20. Threats to Validity
Potential main effects (direct impact of one variable on another variable):
History
Maturation
Testing or reactivity

21. Threats to Validity
History: becomes a threat to the validity of our design when something occurs in the world outside of our study that has an impact on our subjects
Can affect performance on posttest

22. Threats to Validity
Maturation: results from changes in the subjects, unrelated to the study, which will have impact on the results of the study

23. Threats to Validity
Testing or reactivity: the act of observing / measuring changes the very phenomenon we are studying
Demand characteristic (the Hawthorne effect): a situation where the subject is unconsciously trying to help the researcher

24. Threats to Validity The previous threats to validity are main effects
Sensitivity is a more subtle threat to validity
Interactions: occur when two or more variables create changes in the dependent variable
A pretest may interact with the stimulus to create an effect that appears to bolster the impact of the stimulus

25. Threats to Validity
Self selection: rather than a subject being selected into a sample, an individual decides whether to participate or not
Negatively impacts random assignment

26. Threats to Validity
Regression to the mean: multiple measures of the same thing can randomly give multiple results; a second measure may be closer to the “true” average