1. Approaches to social research Lerum
Experiments

2. Here are some new terms:
Internal validity
External validity
True experiments, quasi-experiments, pre-experiments
Laboratory experiments, field experiments, natural experiments
Reactivity
Random assignment
Test of statistical significance
Matching
Experimental and control groups

3. Experiments, overview
The main reason for conducting an experiment is to test a hypothesis that one variable causes a change in another variable. This allows one to strongly infer (but not prove) cause-effect.
Research designs that do this are called “true experiments” as opposed to “pre-experimental” or “quasi-experimental”

4. Features of a true (classical) experiment
Random and matched assignment of subject into experimental and control groups (both can be used, but one should never use matching alone)
Manipulation of an independent variable
Followed by measurement of the dependent variable
Tests for statistical significance

5. Example:
Experiment of impact of teacher comments on student performance (Page, 1958).
Students randomly put into two groups (w/o their knowledge)
Teachers wrote comments for one group, but not for the other.
Students’ subsequent test scores were compared, and the students w/ teacher comments scored better overall.

6. What is the point of a true experiment?
To see if there is a causal relationship between X and Y. in order to determine this you must find:
1) An association between the two variables
2) Direction of influence (which variable influences which?)
3) Elimination of plausible rival explanations

7. Association
In the teacher comment study, it was found the the IV (teacher comments) was associated with the DV (test scores).

8. Direction of Influence
Evidence that the IV influenced the DV and not the other way around is based on time order in experiments. Y (e.g. test scores) cannot be the cause of X (teacher comments) if it occurred after X.

9. Elimination of rival explanations
There may be lots of reasons for students doing better or worse on later exams (e.g. not just teacher comments).
How do we know it’s the comments, and not something else that improved test scores?
As long as you randomly assign people into groups, each participant has an equal chance of being in either group.
In an airtight experiment, there is only one rival explanation: the results occurred by chance.

10. Tests for statistical significance
Testing for statistical significance allow us to assess the likelihood that the results of an experiment cold have occurred by chance.
When we read that the results of an experiment were significant at the .05 level, this means that only about 5% of the time, or 5 times in 100, would differences this large between groups occur simply out of chance.

11. Internal validity
True experiments can provide relatively sound evidence of a causal relationship, and hence they generally have high internal validity.
An experiment is internally valid to the extent that it rules out the possibility that extraneous variables, rather than the manipulated IV, or responsible for the observed outcome.

12. External validity
External validity refers to the question of generalizability. In other words, what do the experimental results mean outside of the particular context of the experiment.
Would the experiment produce the same results on a different sample?
This can be ensured by replicating the study in a number of different settings, with different kinds of people, etc.

13. Weaknesses of “true” experiments
Typically low in external validity.
E.g., most experiments conducted in university settings, on white college sophomores taking psychology. Cannot generalize to larger populations from a small, uniquely situated sample.
Reactivity and social desirability effects
Ethical issues (abuse of power more likely due to extremely controlled conditions; valid results harder to achieve if people have full consent)

14. Strengths of “true” experiments
Generally high in internal validity
Can establish strong inference for a causal relationship (non-spurious)

15. In addition to laboratory experiments, two other types exist: 1)Field experiments, and 2) Natural experiments
Field experiments refer to studies that meet all the requirements of a true experiment but are conducted in a “natural” setting
Both field and lab experiments can be “true” experiments
As in a lab experiment, there is a staging of events.
But this staging occurs in an environment that is familiar to the subject.

16. Example of a field experiment
Common in public health – otherwise known as randomized, controlled Trials
E.g. randomized controlled male circumcision trials

17. Example of a natural experiment
(from Babbie, p. 245): After WWII, researchers studied impact of mass bombing on civilian morale, comparing residents of heavily bombed cities to those in cities w/o bombing.
Natural experiments occur “after the fact,” and hence can not be randomized or subjected to pre and post analysis. Thus natural experiments are always “quasi” experiments.