1. 2 Research Methods Safeguards against error
Slides prepared by Matthew Isaak

2. Lecture Preview The beauty and necessity of good research design
The scientific method
Ethical issues in research design
Statistics
Evaluating psychological research

3. Why We Need Research Designs
LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
In the early 1990s, an autism treatment was developed called "facilitated communication."
The developers thought that autism was a motor disorder.
The facilitator sat next to child with autism and guided the child's hand over a keyboard, allowing the children to type out words. 3

4. Why We Need Research Designs LO 2
Why We Need Research Designs LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
Students seemed to make stunning progress in communication, telling parents "I love you" and writing poetry.
However, some students began making allegations of sexual abuse against parents.
There was no physical evidence, just the communicators via the facilitators.

5. Why We Need Research Designs LO 2
Why We Need Research Designs LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
Dozens of controlled studies examined the phenomenon and found that the words came solely from the minds of the facilitators.
Still, some people continue to practice facilitated communication. 5

6. Figure 2.1 Putting Facilitated Communication to the Test.6

7. Why We Need Research Designs LO 2
Why We Need Research Designs LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
Even well-educated, intelligent people can be fooled.
Well-planned designs can help to eliminate biases when examining phenomena. 7

8. Why We Need Research Designs LO 2
Why We Need Research Designs LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
Prefrontal lobotomy is example of what happens when we rely on subjective impressions.
Developer won the Nobel Prize
In it, the neural fibers connecting frontal lobes to the thalamus were severed.
Control studies showed it didn't work.

9. Common Sense LO 1.1 Explain why psychology is more than just common sense.
Most of us trust our gut intuitions about how the world works.
Birds of a feather flock together.
Absence makes the heart grow fonder.
Better safe than sorry.
Two heads are better than one.
Actions speak louder than words.
Opposites attract.
Out of sight, out of mind.
Nothing ventured, nothing gained.
Too many cooks spoil the broth.
The pen is mightier than the sword.
LO 1.1
Even though each of these ring true, they are in fact opposites!

10. Naïve Realism LO 1.1 Explain why psychology is more than just common sense.
The belief that we see the world precisely as it actually is in truth – “seeing is believing” Works well in ordinary life, but consider:
The earth seems flat.
We seem to be standing still, yet the earth is moving around the sun 18.5 miles/sec.
LO 1.1

11. These two tabletops are identical in length.
Which table is longer? LO 1.1 Explain why psychology is more than just common sense.
These two tabletops are identical in length.

12. When Common Sense is Right LO 1
When Common Sense is Right LO 1.1 Explain why psychology is more than just common sense.
Not all common sense is wrong.
Common sense should serve as a generator for hypotheses, which can then be tested.
But learning to think like a scientist means learning when—and when not—to trust our common sense.

13. Theories and Hypotheses LO 1
Theories and Hypotheses LO 1.2 Explain the importance of science as a set of safeguards against biases.
A scientific theory is an explanation for a large number of findings in the natural world.
A hypothesis is a specific prediction based on a theory, which can then be tested.
Theories are general explanations; hypotheses are specific predictions derived from them.

14. Theory Misconceptions LO 1
Theory Misconceptions LO 1.2 Explain the importance of science as a set of safeguards against biases.
“A theory explains one specific event.” “A theory is just an educated guess.” Why are these both wrong?
LO 1.2

15. Science as a Safeguard against Bias LO 1
Science as a Safeguard against Bias LO 1.2 Explain the importance of science as a set of safeguards against biases.
Confirmation bias
Tendency to seek out evidence that supports our hypothesis and neglect or distort contradicting evidence
Scientists need to design studies that may disprove their theories.
LO 1.2

16. Figure 1. 3 Diagram of Wason Selection Task
Figure Diagram of Wason Selection Task. In the Wason selection task, you must pick two cards to test the hypothesis that all cards that have a vowel on one side have an odd number on the other. Which two will you select?

17. Science as a Safeguard against Bias LO 1
Science as a Safeguard against Bias LO 1.2 Explain the importance of science as a set of safeguards against biases.
Belief perseverance
Tendency to stick to our initial beliefs even when evidence contradicts them
The “don’t confuse me with the facts” bias
LO 1.2

18. Metaphysical Claims LO 1
Metaphysical Claims LO 1.2 Explain the importance of science as a set of safeguards against biases.
Non-testable assertions that fall outside the realm of science
The existence of God, the soul, or the afterlife
LO 1.2

19. FIGURE 1. 4 Nonoverlapping Realms
FIGURE Nonoverlapping Realms. Scientist Stephen Jay Gould (1997) argued that science and religion are entirely different and nonoverlapping realms of understanding the world. Science deals with testable claims about the natural world that can be answered with data, whereas religion deals with untestable claims about moral values that can’t be answered with data. Although not all scientists and theologists accept Gould’s model, we adopt it for the purposes of this textbook. (Source: Gould, 1997)
LO 1.2

20. We Might Be Wrong LO 1.2 Explain the importance of science as a set of safeguards against biases.
Good scientists are aware they might be wrong.
Scientific knowledge is always tentative and open to revision.
Science forces us to question our findings and conclusions.

21. Thinking Clearly LO 1.4 Identify reasons we are drawn to pseudoscience.
Learning to think scientifically can help us avoid falling prey to pseudoscience.
Emotional reasoning fallacy—using emotions rather than evidence as the guide
Bandwagon fallacy—Lots of people believe it, so it must be true.
“Not me” fallacy—Other people may have those biases, but not me.

22. Why Should We Care? LO 1.4 Identify reasons we are drawn to pseudoscience.
Because pseudoscience can be very dangerous.
Three major reasons to be concerned:
Opportunity cost
Direct harm
Inability to think scientifically
Although not foolproof, scientific thinking is our best safeguard against human error.

23. Scientific Skepticism LO 1
Scientific Skepticism LO 1.5 Identify the key features of scientific skepticism.
Being scientifically skeptical does not mean being closed-minded.
Evaluate claims with an open mind, but insist on persuasive evidence before accepting them.
Skeptics are willing to change their minds, but must have good evidence before doing so.

24. Critical Thinking LO 1.6 Identify and explain the text’s six principles of scientific thinking.
A set of skills for evaluating all claims in a open-minded and careful fashion
This allows us to overcome our own biases (especially the confirmation bias).
Six critical thinking principles will be emphasized in this course.

25. Critical Thinking Principles LO 1
Critical Thinking Principles LO 1.6 Identify and explain the text’s six principles of scientific thinking.
Ruling out rival hypotheses
Have important alternate explanations for the finding been considered?
Correlation isn’t causation
Can we be sure A causes B?
Falsifiability
Can the claim be disproven?

26. Critical Thinking Principles LO 1
Critical Thinking Principles LO 1.6 Identify and explain the text’s six principles of scientific thinking.
Replicability
Can the results be duplicated in other studies?
Extraordinary claims require extraordinary evidence
Is the evidence as convincing as the claims?

27. Critical Thinking Principles LO 1
Critical Thinking Principles LO 1.6 Identify and explain the text’s six principles of scientific thinking.
Occam’s razor
Does a simpler explanation fit the data just as well?
Parsimony: logical simplicity

28. Figure 2.2 The Prefrontal Lobotomy.

29. Two Modes of Thinking LO 2
Two Modes of Thinking LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
System I or intuitive thinking
Quick, reflexive, almost automatic
Relies on heuristics
System 2 or analytical thinking
Slow, reflexive, effortful

30. Two Modes of Thinking LO 2
Two Modes of Thinking LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
Heuristics are mental shortcuts or rules of thumb that we use daily.
They reduce the cognitive energy required to solve problems but we oversimplify reality.
Imagine yourself driving from Reno, Nevada to San Diego, California—which compass direction would you take?

31. San Diego is actually EAST of Reno LO 2
San Diego is actually EAST of Reno LO 2.1 Identify two modes of thinking and their application to scientific reasoning.
LO 2.1
FIGURE In Which Compass Direction Would You Travel to Get from Reno, NV to San Diego, CA? 31

32. So, how do we prevent ourselves from being fooled by our own (and other people's) biases?
LO 2.1 Identify two modes of thinking and their application to scientific reasoning.

33. The Scientific Method Toolbox
LO 2.2 Describe the advantages and disadvantages of using naturalistic observation,
case studies, self-report measures, and surveys.
Allows us to test specific hypotheses derived from broader theories of how things work.
Theories are never "proven," but hypotheses can be confirmed or disconfirmed.
We can use a number of different types of SM tools to gain information and test hypotheses.

34. Naturalistic Observation LO 2
Naturalistic Observation LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
Watching behavior in real-world settings
High degree of external validity - extent to which we can generalize our findings to the real world
Low degree of internal validity - extent to which we can draw cause-and-effect inferences
LO 2.2 34

35. Case Study Designs LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
Studying one person or a small number of people for an extended period of time
Common with rare types of brain damage or mental illness
Helpful in providing existence proofs, but can be misleading and anecdotal
LO 2.2 35

36. Self Report Measures and Surveys LO 2
Self Report Measures and Surveys LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
Psychologists often need to ask people about themselves or others.
Self-report measures or questionnaires asses characteristics such as personality or mental illness.
Surveys ask about a person's opinions or abilities.
Not all measures and surveys are equal.

37. Random Selection LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
The key to generalizability in surveys and questionnaire studies
Ensures every person in a population has an equal chance of being chosen to participate
Non-random selection can skew results and make them inaccurate when applied to the population as a whole.

38. Evaluating Measures LO 2
Evaluating Measures LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
To trust results, the measures must have:
Reliability—consistency of measurement
Validity—extent to which a measure assesses what it claims to measure
A test must be reliable to be valid, but a reliable test can still be completely invalid.

39. Self-Report Measures LO 2
Self-Report Measures LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
Pros
Easy to administer
Direct (self) assessment of person's state
LO 2.2 39

40. Self-Report Measures LO 2
Self-Report Measures LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
Cons
Accuracy is skewed for certain groups (narcissists)
Potential for dishonesty
Response sets - tendencies of research subjects to distort their responses
Positive impression management
Malingering
LO 2.2 40

41. Rating Data LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys.
People can also be asked to rate others on different characteristics.
This can do away with some biases in self-report, but still has problems.
Halo effect—tendency of ratings of one positive characteristic to spill over to influence the ratings of other characteristics

42. Correlational Designs
LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.
Examine how two variables are related
Correlations vary from -1 to +1 and can be:
Positive (as one increases, so does the other)
Negative (as one increases, the other decreases)
Zero (no relationship between variables)
Depicted in a scatterplot

43. Figure 2.4 Diagram of Three Scatterplots.
Scatterplots LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.
LO 2.3
Figure Diagram of Three Scatterplots. 43

44. Correlational Designs LO 2
Correlational Designs LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.
Illusory Correlation—perception of a statistical association where none exists
Crime rates and the full moon
Arthritis and weather
Examining a probability table helps to explain why we are all prone to seeing relationships where none exists.

45. Humans tend to overemphasize cell A and ignore the non-events.
The Great Fourfold Table of Life LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.
Did a crime occur?
Yes No
Yes
Did a full moon occur?
LO 2.3 No
Humans tend to overemphasize cell A and ignore the non-events.

46. Correlation vs. Causation LO 2
Correlation vs. Causation LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.
Just because two things are related, does not mean that one causes another.
There are three possible explanations:
A causes B
B causes A
C causes both A and B

47. Determining Causation
LO 2.4 Identify the components of an experiment, the potential pitfalls that can
lead to faulty conclusions, and how psychologists control for these pitfalls.
The only way to determine if one thing is casually related to another is via an experimental design.
This is because in an experiment, you purposefully manipulate variables, rather than just measure already existing differences.

48. What Makes a Study an Experiment. LO 2
What Makes a Study an Experiment? LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Random assignment of participants groups
Experimental Group - receives the manipulation
Control Group - does not receive the manipulation
LO 2.4 48

49. What Makes a Study an Experiment. LO 2
What Makes a Study an Experiment? LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Manipulation of an independent variable
The dependent variable is what the experimenter measures to see whether manipulation had an effect.
LO 2.4 49

50. What Makes a Study an Experiment. LO 2
What Makes a Study an Experiment? LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Confounds - any difference between the experimental and control groups aside from IV
Makes IV effects uninterpretable
Cause and effect - possible to infer, with random assignment and manipulation of independent variable
LO 2.4 50

51. Pitfalls in Experimental Design LO 2
Pitfalls in Experimental Design LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Placebo effect - improvement resulting from the mere expectation of improvement
Participants must be blind to their assignment to groups.
Placebos show many of the same characteristics as real drugs.
Nocebo effect - harm resulting from the mere expectation of harm
LO 2.4 51

52. Pitfalls of Experimental Design LO 2
Pitfalls of Experimental Design LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Experimenter expectancy effect – when researchers' hypotheses lead them to unintentionally bias a study outcome
Clever Hans, the mathematical horse
Rosenthal's undergrads and maze-bright or maze-dull rats
Using a double-blind design can decrease this.
LO 2.4 52

53. Pitfalls of Experimental Design LO 2
Pitfalls of Experimental Design LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls.
Demand characteristics - cues that participants pick up allowing them to guess at the researcher's hypotheses
Disguising the purpose of the study or using "filler" items can help to decrease these.
LO 2.4 53

54. Ethical Issues in Research Design
LO 2.5 Explain the ethical obligations of researchers toward their research participants.
Tuskegee study ran from 1932 to 1972
African American men living in rural Alabama diagnosed with syphilis
U.S. Public Health Service never informed, or treated, the men; they merely studied the course of the disease.
28 men died of syphilis, 100 of related complications, 40 wives were infected, and 19 children were born with it.
LO 2.5 54

55. Ethical Guidelines for Human Research LO 2
Ethical Guidelines for Human Research LO 2.5 Explain the ethical obligations of researchers toward their research participants.
Today, research has to go through a careful process of review to ensure that it is conducted ethically.
Institutional Review Board (IRB)
Informed consent
Justification of deception
Debriefing of subjects afterwards

56. Ethical Issues in Animal Research
LO 2.6 Describe both sides of the debate on the use of animals as research subjects.
Only 7-8% of psychological research uses animals.
Vast majority are rodents and birds
Goal is to generate ideas about the brain and behavior without harming people

57. Statistics: The Language of Research
LO 2.7 Identify uses of various measures of central tendency and variability.
Descriptive statistics—numerical characterizations of the data set
Central tendency—where the group tends to cluster
Mean: average of all scores
Median: middle score in the data set
Mode: most frequent score in the data set

58. Figure 2.7 Distribution Curves.

59. Statistics: The Language of Research LO 2
Statistics: The Language of Research LO 2.7 Identify uses of various measures of central tendency and variability.
Variability—sense of how loosely or tightly bunched scores are
Range—difference between the highest and lowest scores
Standard deviation—measure that takes into account how far each data point is from the mean

60. Range vs. Standard Deviation LO 2
Range vs. Standard Deviation LO 2.7 Identify uses of various measures of central tendency and variability.
Both sets of data have the same range, but very different standard deviations.
Standard deviations are less susceptible to extreme scores than ranges are.
LO 2.7

61. FIGURE 2.8 The Range versus the Standard Deviation.
LO 2.7

62. Statistics: The Language of Research
LO 2.8 Explain how inferential statistics can help us to determine whether we can
generalize from our sample to the full population.
Inferential statistics allow us to determine whether we can generalize findings from our sample to the population.
Statistical significance - finding would have occurred by chance less than 1 in 20 times
Practical significance - real-world importance
LO 2.8, 2.9 62

63. How People Lie With Statistics
LO 2.9 Show how statistics can be misused for purposes of persuasion.
People can misuse statistics to persuade – and mislead – others. They can:
Report unrepresentative measures, like the mean instead of the median for skewed data
Truncate the axes of graphs
Neglect base rate probabilities

64. Figure 2.9 Arrest Rates Before and After Transcendental Meditation.
LO 2.9

65. Evaluating Psychological Research
LO 2.10 Identify flaws in research designs and how to correct them.
The process of peer review helps to identify and correct flaws in research and research conclusions.
Remember to keep a look out for confounds, placebos, experimenter expectancy, correlation vs. causation, and others.

66. Evaluating Psychology in the Media
LO 2.11 Identify skills for evaluating psychological claims in the popular media.
Most reporters are not scientists, so consider the source.
Tabloids vs. science magazines
Beware of sharpening, leveling, and pseudosymmetry.
LO 2.11 66