Part Three Modes of Observation

Chapter 8 Experiments

Chapter Outline Introduction Introduction Topics Appropriate to Experiments Topics Appropriate to Experiments The Classical Experiment The Classical Experiment Selecting Subjects Selecting Subjects

Chapter Outline Variations on Experimental Designs Variations on Experimental Designs An Illustration of Experimentation An Illustration of Experimentation Web-Based Experiments Web-Based Experiments "Natural" Experiments "Natural" Experiments Strengths and Weaknesses of the Experimental Method Strengths and Weaknesses of the Experimental Method

INTRODUCTION EXPERIMENTS: The design most linked to structured science. EXPERIMENTS: The design most linked to structured science. EXPERIMENTS: Involve 3 things: EXPERIMENTS: Involve 3 things: 1. group of selected subjects 2. do something to them 3. observe the effect Hence we are getting at CAUSATION.

Topics Appropriate to Experiments Projects with limited and well- defined concepts. Projects with limited and well- defined concepts. Projects that are explanatory rather than descriptive. Projects that are explanatory rather than descriptive. Studies of small group interaction. Studies of small group interaction.

Components of Experiments Three Pairs Independent and dependent variables Independent and dependent variables Pretesting and posttesting Pretesting and posttesting Experimental and control groups Experimental and control groups

INDEPENDENT & DEPENDENT VARIABLES Experiments examine the effect of the i.v. on the d.v. Experiments examine the effect of the i.v. on the d.v. I.V. has 2 attributes—present (give) or not present (don’t give) I.V. has 2 attributes—present (give) or not present (don’t give) Measure what happens to d.v. if i.v. is present or not. Measure what happens to d.v. if i.v. is present or not. E.G. 1 Group watch film; Other group does not—D.V. prejudice among subjects. E.G. 1 Group watch film; Other group does not—D.V. prejudice among subjects.

PRE-TESTING & POST- TESTING SIMPLEST DESIGN—Pre- testing occurs first—subjects are measured on D.V. SIMPLEST DESIGN—Pre- testing occurs first—subjects are measured on D.V. Then, subjects are exposed to the treatment (film) Then, subjects are exposed to the treatment (film) Post-testing measures the effect of the I.V. (What would a lower level of prejudice tell us?) Post-testing measures the effect of the I.V. (What would a lower level of prejudice tell us?)

Experimental and Control Groups Must be as similar as possible. Must be as similar as possible. Control group represents what the experimental group would have been like had it not been exposed to the stimulus. Control group represents what the experimental group would have been like had it not been exposed to the stimulus. In the film example---if prejudice is reduced in both groups (but more in one than the other) we know film had an effect. (E.G., Hawthorne Effect) In the film example---if prejudice is reduced in both groups (but more in one than the other) we know film had an effect. (E.G., Hawthorne Effect)

DOUBLE-BLIND EXPERIMENT DBE—neither subjects nor experimenters know which is the exp. group vs. cont. group DBE—neither subjects nor experimenters know which is the exp. group vs. cont. group E.G., Medical-Drug Study— researchers would not be told which subjects were receiving the drugs E.G., Medical-Drug Study— researchers would not be told which subjects were receiving the drugs D.V. needs to be clearly operationalized and precisely defined. D.V. needs to be clearly operationalized and precisely defined.

Selecting Subjects Representative sampling—still needed (College Students often used—remember you cannot generalize to the public with your guys!) Representative sampling—still needed (College Students often used—remember you cannot generalize to the public with your guys!) Probability sampling—as a rule you need at least 100 in the sample to be representative. Experiments rarely involve that number. Don’t use this—but use its logic!) Probability sampling—as a rule you need at least 100 in the sample to be representative. Experiments rarely involve that number. Don’t use this—but use its logic!)

EXPERIMENTAL SAMPLING Randomization—first the researcher selects a group of subjects from the population— may be random; may not. Randomization—first the researcher selects a group of subjects from the population— may be random; may not. Then---you randomly assign subjects to control vs. experimental groups. (The more alike these two groups, the better) Then---you randomly assign subjects to control vs. experimental groups. (The more alike these two groups, the better)

EXPERIMENTAL MATCHING Another way to make the 2 groups comparable—matching. Another way to make the 2 groups comparable—matching. We simply look at the population and match the control and experimental groups proportionately to this. (50% are White—make sure both groups have 50% whites) We simply look at the population and match the control and experimental groups proportionately to this. (50% are White—make sure both groups have 50% whites) See Quota Matrix—using several matching variables—p.235. (COULD EVEN MATCH ON THE D.V) See Quota Matrix—using several matching variables—p.235. (COULD EVEN MATCH ON THE D.V)

Randomization and Matching (Why R is better) May not know which variables will be relevant for matching process. May not know which variables will be relevant for matching process. Most statistics used to analyze results assume randomization. Most statistics used to analyze results assume randomization. Randomization only makes sense if you have a large pool of subjects. (If small number—use matching) Randomization only makes sense if you have a large pool of subjects. (If small number—use matching)

Preexperimental Research Designs One-shot case study - single group of subjects is measured on a variable following experimental stimulus. (W/O Pre-test—can’t be sure of the effect) One-shot case study - single group of subjects is measured on a variable following experimental stimulus. (W/O Pre-test—can’t be sure of the effect) One-group pretest-posttest design - adds a pre-test for the group, but lacks a control group. (Does not rule out other variables making a contribution to the change) One-group pretest-posttest design - adds a pre-test for the group, but lacks a control group. (Does not rule out other variables making a contribution to the change)

Pre-Exp Continued…. Static-group comparison - includes experimental and control group, but no pre-test. (How do we know the exp. Group wasn’t less prejudiced to start with????) Static-group comparison - includes experimental and control group, but no pre-test. (How do we know the exp. Group wasn’t less prejudiced to start with????) See Chart—p. 237 See Chart—p. 237

Sources of Internal Invalidity Internal Validity—perhaps the conclusions of the experiment may not reflect what actually went on. Internal Validity—perhaps the conclusions of the experiment may not reflect what actually went on. Historical events may occur during the course of the experiment. (Af- Am leader gets shot during study) Historical events may occur during the course of the experiment. (Af- Am leader gets shot during study) Maturation of the subjects. (Subjects change during time on their own) Maturation of the subjects. (Subjects change during time on their own)

INTERNAL VALIDITY…cont. Testing and retesting can influence behavior. (Subjects figure out we are looking for prejudice reduction) Testing and retesting can influence behavior. (Subjects figure out we are looking for prejudice reduction) Instrumentation (We might use different measures of D.V.— but what if they aren’t comparable) Instrumentation (We might use different measures of D.V.— but what if they aren’t comparable)

INTERNAL VALIDITY…cont. Statistical regression of subjects starting out in extreme positions. (Subjects low in prejudice might just naturally move up—not even based on the experiment.) Statistical regression of subjects starting out in extreme positions. (Subjects low in prejudice might just naturally move up—not even based on the experiment.) Selection biases. (Groups must be comparable at the start.) Selection biases. (Groups must be comparable at the start.) Experimental mortality - subjects drop out of the study before it's completed. Experimental mortality - subjects drop out of the study before it's completed.

INTERNAL VALIDITY…cont. Demoralized control group subjects. (They may stop trying—no treatment) Demoralized control group subjects. (They may stop trying—no treatment) SEE PAGE 239—if we conduct an experiment like this, we can usually tell what is going on—we have internal validity. SEE PAGE 239—if we conduct an experiment like this, we can usually tell what is going on—we have internal validity.

Limiting External Invalidity Let’s say did experiment correctly— would it have the same affect in the theater? (Can’t control sensitizing affect of the experiment itself.) Let’s say did experiment correctly— would it have the same affect in the theater? (Can’t control sensitizing affect of the experiment itself.) Solomon four-group design –see page 241 Solomon four-group design –see page 241 Posttest-only control group design— assumes subjects are randomly assigned and comparable in exp. And cont. groupings. Posttest-only control group design— assumes subjects are randomly assigned and comparable in exp. And cont. groupings.

Solomon Four-group Design Four groups of subjects, assigned randomly: Groups 1 and 2 are the control and experimental group. Groups 1 and 2 are the control and experimental group. Group 3 does not have the pre- test. Group 3 does not have the pre- test. Group 4 is only posttested. Group 4 is only posttested.

Posttest-only Control Group Design Includes Groups 3 and 4 of the Solomon design. Includes Groups 3 and 4 of the Solomon design. With proper randomization, only these groups are needed to control problems of internal invalidity and the interaction between testing and stimulus. With proper randomization, only these groups are needed to control problems of internal invalidity and the interaction between testing and stimulus. Explain Pygmalion Effect—e.g., Spurters or Bloomers Explain Pygmalion Effect—e.g., Spurters or Bloomers

"Natural" Experiments Important social scientific experiments occur outside controlled settings and in the course of normal social events. Important social scientific experiments occur outside controlled settings and in the course of normal social events. Raise validity issues because researcher must take things as they occur. Raise validity issues because researcher must take things as they occur.

Web-based Experiments Increasingly, researchers are using the World Wide Web to conduct experiments. Increasingly, researchers are using the World Wide Web to conduct experiments. Because representative samples are not essential in most experiments, researchers use volunteers who respond to invitations online. Because representative samples are not essential in most experiments, researchers use volunteers who respond to invitations online.

Experimental Method Strengths: Isolation of the experimental variable over time. Isolation of the experimental variable over time. Experiments can be replicated several times using different groups of subjects. Experiments can be replicated several times using different groups of subjects.

Experimental Method Weaknesses: Artificiality of laboratory setting. Artificiality of laboratory setting. Social processes that occur in a lab might not occur in a more natural social setting. Social processes that occur in a lab might not occur in a more natural social setting.