1. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Chapter Eight: Internal and External Validity Chapter Eight: Internal and External Validity

2. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Internal Validity: Evaluating Your Experiment from the Inside Internal Validity The concept of internal validity revolves around the question of whether your IV actually caused any change that you observe in your DV.

3. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Internal Validity: Evaluating Your Experiment from the Inside Internal Validity The concept of internal validity revolves around the question of whether your IV actually caused any change that you observe in your DV. If you use adequate control techniques, your experiment should be free from confounding and you can, indeed, conclude that your IV caused the change in your DV.

4. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History History refers to events that occur between the DV measurements in a repeated measures design.

5. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Maturation refers to changes in participants that occur over time during an experiment.

6. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Maturation refers to changes in participants that occur over time during an experiment. These changes could include actual physical maturation or tiredness, boredom, hunger, and so on.

7. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Testing is a threat to internal validity that occurs because measuring the DV causes a change in the DV.

8. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Testing is a threat to internal validity that occurs because measuring the DV causes a change in the DV. Campbell (1957) noted that if you take the same test more than once, scores on the second test may vary systematically from the first scores simply because you took the test a second time.

9. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect A practice effect is a beneficial effect on a DV measurement caused by previous experience with the DV.

10. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Reactive measures are DV measurements that actually change the DV being measured.

11. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Reactive measures are DV measurements that actually change the DV being measured. Many attitude questionnaires are reactive measures. If we ask you a number of questions about how you feel about people of different racial groups, or about women’s rights, or about the president’s job performance, you can probably figure out that your attitude is being measured.

12. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Instrumentation is a threat to internal validity that occurs if the equipment or human measuring the DV changes the measuring criterion over time.

13. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Statistical Regression Statistical regression occurs when low scorers improve or high scorers fall on a second administration of a test due solely to statistical reasons.

14. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Statistical Regression Selection If we choose participants in such a way that our groups are not equal before the experiment, we cannot be certain that our IV caused any difference we observe after the experiment.

15. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Statistical Regression Selection Mortality Mortality can occur if experimental participants from different groups drop out of the experiment at different rates.

16. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Statistical Regression Selection Mortality Interactions with Selection Interactions with selection can occur when the groups we have selected show differences on another variable (i.e., maturation, history, or instrumentation) that vary systematically by groups.

17. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to Internal Validity Threats to Internal Validity History Maturation Testing Practice Effect Reactive Measures Instrumentation (Instrument Decay) Statistical Regression Selection Mortality Interactions with Selection Diffusion or Imitation of Treatments Diffusion or imitation of treatments can occur if participants in one treatment group become familiar with the treatment of another group and copy that treatment.

18. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Protecting Internal Validity Protecting Internal Validity Two Approaches You can (and should) implement the various control procedures discussed in Chapters 6 and 7.

19. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Protecting Internal Validity Protecting Internal Validity Two Approaches You can (and should) implement the various control procedures discussed in this chapter. Use a standard procedure Experimenters used standard procedures called experimental designs to help ensure internal validity.

20. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Protecting Internal Validity Protecting Internal Validity How Important is Internal Validity?

21. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Protecting Internal Validity Protecting Internal Validity How Important is Internal Validity? It is the most important property of any experiment. If you do not concern yourself with the internal validity of your experiment, you are wasting your time.

22. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside The second type of evaluation that you must make of your experiment involves external validity.

23. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside The second type of evaluation that you must make of your experiment involves external validity. When you consider external validity, you are asking a question about generalization.

24. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside External Validity A type of evaluation of your experiment that asks whether your experimental results apply to populations and situations that are different from those of your experiment.

25. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside External Validity A type of evaluation of your experiment that asks whether your experimental results apply to populations and situations that are different from those of your experiment. Generalization Applying the results from an experiment to a different situation or population. In essence, we would like to take our results beyond the narrow confines of our specific experiment. Generalization is an important aspect for any science.

26. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside There are three customary types of generalization in which we are interested: Population generalization Environmental generalization Temporal generalization

27. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside There are three customary types of generalization in which we are interested: Population generalization Applying the results from an experiment to a group of participants that is different and more encompassing than those used in the original experiment. Environmental generalization Temporal generalization

28. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside There are three customary types of generalization in which we are interested: Population generalization Environmental generalization Applying the results from an experiment to a situation or environment that differs from that of the original experiment. Temporal generalization

29. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education External Validity: Generalizing Your Experiment to the Outside There are three customary types of generalization in which we are interested: Population generalization Environmental generalization Temporal generalization Applying the results from an experiment to a time that is different from that when the original experiment was conducted.

30. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966):

31. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966): Interaction of Testing and Treatment A threat to external validity that occurs when a pretest sensitizes participants to the treatment yet to come. Occurs for the pretest-posttest control group design Because of a pretest, your participants’ reaction to the treatment will be different.

32. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966): Interaction of Testing and Treatment Interaction of Selection and Treatment A threat to external validity that can occur when a treatment effect is found only for a specific sample of participants. Occurs when the effects that you demonstrate hold true only for the particular groups that you selected for your experiment. Treatment interaction becomes greater as it becomes more difficult to find participants for your experiment (Campbell & Stanley, 1966).

33. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966): Interaction of Testing and Treatment Interaction of Selection and Treatment Reactive Arrangements A threat to external validity caused by an experimental situation that alters participants’ behavior, regardless of the IV involved. We cannot be sure that the behaviors we observe in the experiment will generalize outside that setting because the artificial conditions of the experiment do not exist in the real world.

34. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966): Interaction of Testing and Treatment Interaction of Selection and Treatment Reactive Arrangements Demand Characteristics Features from the experiment that inadvertently lead participants to respond in a particular manner. Demand characteristics make generalizations difficult because it is not clear from a set of research findings whether the participants are responding to an experiment’s IV, its demand characteristics, or both.

35. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Methods) Here is a summary of factors relating to external validity from Campbell and Stanley (1966): Interaction of Testing and Treatment Interaction of Selection and Treatment Reactive Arrangements Demand Characteristics Multiple-Treatment Interference A threat to external validity that occurs when a set of findings results only when participants experience multiple treatments in the same experiment (repeated measures designs).

36. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat

37. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat If you are interested in the behavior of subhumans, generalizing from rats (and pigeons) to all other animals may be a stretch. If you are interested in generalizing from animal to human behavior, there are certainly closer approximations to humans (and pigeons) in the animal kingdom.

38. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student

39. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student Psychologists who want to conduct human research turn to a ready, convenient source of human participants – students in introductory psychology courses (a technique referred to as convenience sampling ).

40. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student Psychologists who want to conduct human research turn to a ready, convenient source of human participants – students in introductory psychology courses (a technique referred to as convenience sampling ). Convenience Sampling A researcher’s sampling of participants based on ease of locating the participants; often does not involve true random selection.

41. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex

42. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex All four of these derogatory labels have been applied to women at various points in time.

43. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex All four of these derogatory labels have been applied to women at various points in time. The supposed inferiority of women has carried over into some psychological theories. Freud’s theories Erikson’s theory of psychosocial crises (“Eight Stages of Man”)

44. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Carol Tavris’s (1992) thesis is that “despite women’s gains in many fields in the last twenty years, the fundamental belief in the normalcy of men, and the corresponding abnormality of women, has remained virtually untouched” (p. 17)

45. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Even the Rats and Students Were White

46. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Even the Rats and Students Were White Just as history has failed to record the accomplishments of many women throughout time, it has largely ignored the accomplishments of African Americans and other minority groups. When we conduct research and make generalizations, we should be cautious that we do not exclude minority groups from our considerations.

47. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Even the Rats and Students Were White Even the Rats, Students, Women, and Minorities Were American

48. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Even the Rats and Students Were White Even the Rats, Students, Women, and Minorities Were American Although experimental psychology’s early roots are based in Europe, this aspect of the discipline quickly became Americanized, largely due to the influence of John B. Watson’s behaviorism.

49. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) The Infamous White Rat The Ubiquitous College Student The “Opposite” or “Weaker” or “Inferior” or “Second” Sex Even the Rats and Students Were White Even the Rats, Students, Women, and Minorities Were American In the mid-1960’s, psychologists started taking culture and ethnicity more seriously. The field of cross-cultural psychology has evolved from those changes that began in the 1960s.

50. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Threats to External Validity (Based on Our Participants) Cross-Cultural Psychology A branch of psychology whose goal is to determine the universality of research results. Ethnocentricity Other cultures are viewed as an extension of one’s own.

51. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Mook (1983) pointed out four alternative goals of research that do not stress external validity:

52. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Mook (1983) pointed out four alternative goals of research that do not stress external validity: We may merely want to find out if something can happen (not whether it actually happens).

53. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Mook (1983) pointed out four alternative goals of research that do not stress external validity: We may merely want to find out if something can happen (not whether it actually happens). We may be predicting from the real world to the lab – seeing a phenomenon in the real world, we think it will operate in a certain manner in the lab.

54. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Mook (1983) pointed out four alternative goals of research that do not stress external validity: We may merely want to find out if something can happen (not whether it actually happens). We may be predicting from the real world to the lab – seeing a phenomenon in the real world, we think it will operate in a certain manner in the lab. If we can demonstrate that a phenomenon occurs in a lab’s unnatural setting, the validity of the phenomenon may actually be strengthened.

55. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Mook (1983) pointed out four alternative goals of research that do not stress external validity: We may merely want to find out if something can happen (not whether it actually happens). We may be predicting from the real world to the lab – seeing a phenomenon in the real world, we think it will operate in a certain manner in the lab. If we can demonstrate that a phenomenon occurs in a lab’s unnatural setting, the validity of the phenomenon may actually be strengthened. We may study phenomena in the lab that don’t even have a real- world analogy.

56. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Replication An additional scientific study that is conducted in exactly the same manner as the original research project. When we replicate an experimental finding, we are able to place more confidence in that result.

57. The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education The Devil’s Advocate: Is External Validity Always Necessary? Replication An additional scientific study that is conducted in exactly the same manner as the original research project. When we replicate an experimental finding, we are able to place more confidence in that result. Replication With Extension An experiment that seeks to confirm (replicate) a previous finding but does so in a different setting or with different participants or under different conditions.