1 Chapter 6: The Internal Validity of Research p.157 Confounds Threats to Internal Validity Reactivity Demand Characteristics Experimental Expectancies Summary

2 Chapter 6: The Internal Validity of Research Extraneous Variables (so what? Who cares?) –They contaminate the experiment by providing Alternative explanations jeopardizing internal validity –i.e. does the experiment test what is says it tests? –Esp. for “training” type studies Rule them out –Logically (e.g. cohort age differences in early development) –Control for them random assignment direct control (e.g. limit them, “females only”) Measure and estimate their effects on DV Alternative explanations (2) –Confounds: Two vars cannot be separated; e.g. black females; white males –Artifacts: Something other than the IV is causing changes to DV

3 Internal Validity: Confounds Confound: –“when two vars are combined so that the effect of one cannot be separated from the effects of the other” p.156 –? What are some possible confounding vars in your study? How to avoid –Untangle them, use each as IV (factorial design) –Eliminate the confound Selection of Ps Random assignment (how does that do it?) Blocking and measuring Measure the confounding variable

4 Internal Validity: Confounds Natural Confounds (Vars that naturally occur together) –Age & maturity; ethnicity & wealth; gender & toy choices –What are some more? Treatment Confounds –IV is connected with another var or treatment E.g. female Es conduct exp. Groups; males, control groups Critical question: –Do Ps experience exactly the same physical, social, temporal environment except for IV? P. 159 Measurement Confounds –DV measures more than one Hypothetical construct E.g. depression; anxiety

5 Internal Validity: Threats (8) Campbell, D. T., & Stanley, J. ‘63 Time-Related (5) 1.History 2.Maturation 3.Testing 4.Instrumentation change 5.Statistical Regression –Selection (3) 1.Nonrandom assignment 2.Preexisting groups 3.Mortality

6 Internal Threats: Time-Related (5) History: events outside of research setting (e.g. abortion attitudes) Maturation: natural changes over time (e.g. age, experience) Testing: Pretesting effects Instrumentation change: e.g. with experience, E’s assignment to behavioral category Statistical Regression Pre-selection of Ps based on extremes (e.g. anxiety) (see Huck and Sandler, ’79) Air Force cadets

7 Internal Validity Threats: Control Groups in Pre-Post Control group equivalence –Random assignment (not a guarantee, though) –Test by pretesting (not needed most of the time) Important to make sure the pretest is the DV or some measure closely related Possible to pretest on several measures related to the DV –Can use ANCOVA to remove these effects from DV –Advantages of not using a pretest Avoid testing effects Less costly

8 Internal Validity Threats: Selection Threats –Selection bias: when exp group differs from control group Non-random assignment –Avoid self-selection (e.g. volunteering) –Avoid data collection bias (collecting one condition first) How would that cause bias? Preexisting groups –Ps already self-selected into groups E.g. work setting: employees more eager for training in trn group What common characteristics do those in preexisting groups have? Other examples? Mortality –Survivors = drop outs?

9 Internal Validity Threats: Reactivity (ouch!) –Whenever measuring affects DV score Sources –Evaluation apprehension (judgeaphobia) Behavior and self-report (faking) May inhibit or disinhibit behavior (socially desirable responding) Distraction: –May divert attention from experiment instructions to others It’s what’s the Participant thinks, not the researcher that’s important –Novelty effects Participant tries to anticipate what behavior norms should be –E.g. Milgram, S. Asch

10 Internal Validity Threats: Controlling Reactivity General control measures: –Hide your identity don’t call yourself a psychologist! –Be informal, friendly, put them at ease as much as possible –Distract them –Entertain them –Trick them with deception Mislead, lie, whatever it takes to diver them

11 Control Reactivity: Control with Behavioral Measures –Surreptitious, unobtrusive observation (i.e. candid camera) –Embed IV in environment (Piliavin et al., ’69) –“Waiting room ploy” ?? (Aronson et al., ’90) –Use natural” observers (teachers, parents) Control with Self-Report Measures –?? Student evals? Social influence? –Anonymity, confidentiality –Bogus pipeline (Jones & Segall) Noting Instances of Reactivity

12 Reactivity: Demand Characteristics (Martin Orne, ’62) “ Purposely behaving in ways that affect the outcome of research” p.171 Sources: Participant Roles Good participant Bad participant Apathetic participant Impact of P roles Controlling Demand Cue reduction Motivation Role-play control groups Separate DV measure from study

13 Threats: Experimenter Expectancies Types of Experimenter Effects –Biased Observation –Influencing P responses Techniques of Control –Rehearsal and Monitoring –Minimizing E’s Role –Condition Blindness –Avoidance of Data Snooping

14 Internal Validity: Summary Confounds –What are they? –How do you control them? Threats to Internal Validity –Time related; control groups; selection threats Reactivity –Evaluation apprehension, Novelty –Controlling reactivity Demand Characteristics –What is it? How do you control/eliminate them? Experimental Expectancies –Types: biased observations; influencing Ps responses