Lecture 09: Internal Validity and Experiments

Outline Causal Relations Internal Validity Threats to Internal Validity Video Games and Violence Designs

To Show Cause and Effect (J. S. Mill) X must precede Y in Time X must covary with Y No other explanation besides X causes Y is plausible. “Typically, we infer from an effect to a cause by eliminating other possible causes.” (Mackie, 1974, p. 67)

Experiments (Shadish et al., 2002) Definition: A study in which an intervention is deliberately introduced to observe its effects. Randomized Experiment: An experiment in which units are assigned to received the treatment or alternative condition by a random process (e.g., coin flip, table of random numbers)

Design 1: Randomized Two-Group Design Pool of Participants Treatment Control Outcome (DV)

Internal Validity: Concerns the validity of inferences about whether an association between two variables is causal. Shadish et al. (2002, p. 508)

Classic Ideas Campbell (1957) defined internal validity as this question: “Did the experimental stimulus make some significant difference in this specific instance?” Later, Cook and Campbell (1979) argued the internal validity referred to whether observed associations result from a causal relation.

Working Definition of Internal Validity (Source: Shadish, Cook, & Campbell, 2002, p. 38) Internal Validity refers to inferences about whether the observed covariation between X (the presumed treatment) and Y (the presumed outcome) reflects a causal relation from X to Y. Threats to internal validity are the other possible causes of the X and Y association. Validity is not a property of a method, it is a characteristic of inferences.

Threats to Internal Validity Remember definitions of independent and dependent variables.

Selection Preexisting differences between individuals in the different conditions that may influence the dependent variable. When is selection a particular problem?

Maturation Naturally occurring processes within persons that could cause a change in their behavior. Setting: You want to evaluate whether or not a parenting program promoted knowledge of child development and parenting efficacy. Pretest – Intervention – Posttest Design Also known as the O 1 – Intervention – O 2 design (Campbell & Stanley, 1966)

History An event that coincides with the independent variable that could affect the dependent variable. Often we think of major historical events in the social, economic, or political lives of people. Setting: A study about attitudes toward capital punishment occurs right after a well-publicized and brutal murder has occurred. A big problem if experimental conditions are run at different times.

Instrumentation Changes that occur over time in measurement procedures or devices. Basic Rule: Never switch measures in the middle of a study. Willett, Singer, and Martin (1998): “the time for instrument modification is during pilot work, not data collection” (p. 411). Still the meaning of measures can change. What if raters get more experienced between pretest and posttest? What if measures lack invariance?

Attrition (called Mortality in Book) People leave the study. Attrition is always a threat to external validity. When is it a threat to internal validity? Differential Attrition: Attrition is different for each experimental condition. Think of this as selection (out of a study) after random assignment. How can you account for this?

Example: Video Games and Violence (Source: Anderson & Dill, 2000)

Theory: General Affective Aggression Model Basic Idea: Enactment of Aggression is based on knowledge structures created by social learning processes. Playing violent video games increases the accessibility of aggressive cognitions by semantic priming. So the primary path is from Exposure to Cognitions – Not from Exposure to Affect or Exposure to Arousal.

How can we test this?

Pretest Video Games First Step: Select a pair of games that differ in amount of violence NOT arousal and NOT affect. Measure Blood Pressure and Heart Rate: No differences. Measure Ratings of Difficulty, Enjoyment, Frustration, and Action Speed: No differences Measure Excitement: Men thought Wolfenstein was more exciting than Myst. Difference not evident for women. Wolfenstein 3D versus Myst.

Study Details 210 Undergraduate Participants (49.5% Women) IV: Random Assignment to Video Game Condition. DV: B-data - Competitive Reaction Time Task. Use Duration of White Noise Blasts after the Lose Trials. – Participants in Violent Condition Gave Longer Blasts than Participants in Non-Violent Condition (6.81 versus 6.65)

We Could Also Study This Question with a Correlational Design

Study Details 227 Undergraduate Participants (54% Women). Reported playing video games an average of 2.14 hours per week. At least 90% reported that they currently play video games. Most popular games: Super Mario Brothers, Tetris, Mortal Kombat Measured` Aggressive Behavior, Aggressive Personality, Time Spent Playing Games, Violent Video Game Exposure, and GPA

GenABEVGTimeAPGPA Gender- Aggressive Behavior.20- Exposure to Violent Games Time Spent Aggressive Personality GPA

Designs

Design 2: Pretest-Posttest Two- Group Design Treatment Control Posttest (DV: Post - Pre) Pretest

Design 3: Solomon Four-Group Treatment Control Posttest Pretest Treatment Control “Posttest”

Design 4: Between-Participants Factorial Design Z1 Z2 Outcome (DV) X1 Z1 Z2 Outcome (DV) X2