1. Research Methods & Statistics

2. Methods Is Like Whac-A-Mole…
a fun game for which there is no perfect solution

3. Whac-A-Mole
If I’m conducting my own research, the goal is to minimize the moles.
If I’m considering the research of others, the goal is to identify as many moles as I can.

4. Things to Keep in Mind about Statistics
Our focus should be conceptual, not computational.
Statistics are necessary to understand the meaning of a set of numbers.
We need to demonstrate the importance of statistics throughout the entire course, not just in the methods unit.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

5. Frequency Distributions
Putting scores in order adds meaning
Bar graphs (histograms) are visual representations of frequency distributions.

6. What is the center of the distribution?
Measures of Central Tendency
Quiz Scores 4 3 5
Mode
--Most common = 4
Mean
--Arithmetic avg = 20/5 = 4
Median
--Middle score = 4

7. Central Tendency: Mean vs. Median 1968 TOPPS Baseball Cards
Nolan Ryan $1500
Billy Williams $8
Luis Aparicio $5
Harmon Killebrew $5
Orlando Cepeda $3.50
Maury Wills $3.50
Jim Bunning $3
Tony Conigliaro $3
Tony Oliva $3
Lou Pinella $3
Mickey Lolich $2.50
Elston Howard $2.25
Jim Bouton $2
Rocky Colavito $2
Boog Powell $2
Luis Tiant $2
Tim McCarver $1.75
Tug McGraw $1.75
Joe Torre $1.5
Rusty Staub $1.25
Curt Flood $1
With Ryan:
Median=$2.50
Mean=$74.14
Without Ryan:
Median=$2.38
Mean=$2.85

8. The median is a better measure of central tendency than the mean when there are extreme scores.

9. How spread out are the data?
Measures of variation
Range
The spread between the highest number & the lowest number.
Only considers two numbers
Standard deviation

10. Calculation Example for Standard Deviation
Punt
Distance
Deviation
from Mean
Deviation
Squared
std. dev. = 36 38 41 45 -4 -2 +1 +5 16 4 1 25
Variance =
11.5 = 3.4 yds 46
Mean =
160/4 =
40 yds
46/4 = 11.5 =
variance

11. Properties of the Normal Curve
In a large, randomly distributed data set
68% of scores will be within 1 SD of the mean.
95% of scores will be within 2 SDs of the mean.
99.7% of scores will be withing 3 SDs of the mean.

12. Properties of the Normal Curve
Marilyn vos Savant: claimed IQ of 228.
Is it more meaningful to express her IQ as points above average or as standard deviations above average?

13. Correlation
A measure of the strength of the relationship between two variables.
Can be positive or negative.
Useful for making predictions.
You can calculate correlations with Excel or Google Docs.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

14. What does a correlation looks like?
Scatterplots
Positive Correlation
Negative Correlation
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

15. Correlation
No Correlation

16. Correlation How do you express a correlation numerically?
The Correlation Coefficient
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

17. Correlation
A strong correlation is not enough to establish a cause and effect relationship.
Example: There is a correlation between TV watching and grades.
Do you think it’s positive, or negative?
From this, what do we know about cause-and-effect.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

24. Correlation
Even correlations that are clearly not cause-and-effect relationships can be used for prediction.
Ex: College entrance exams and freshman GPA.
Ex: Shoe size and vocabulary size in elementary school children.
Ex: Ice cream sales and the rate of violent crimes.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

25. Statistical Significance
A measure of the likelihood that a result is caused by chance.
In an experiment, we want that likelihood to be low so we can conclude a cause-and-effect relationship exists between the IV and the DV.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

26. Statistical Significance
Several statistics (e.g., chi square, t-test) can be used to calculate statistical significance, but our students don’t need to know these 
They do need to know how to interpret the results of these tests—the p value.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

27. Statistical Significance
P value is an estimate of the probability that a result was caused by chance.
In an experiment, it’s the likelihood that the difference between the experimental and control conditions as measured by the DV was caused by chance.
We want this difference to be caused by our manipulation—the IV—not by chance.
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

28. Statistical Significance
To say that the results of an experiment are statistically significant means that there is a small likelihood that the results were caused by chance; that is, a high likelihood they were caused by the IV.
The threshold for statistical significance is no more than a 5% likelihood the results were caused by chance.
We express this: p ≤ .05
Socrates “How many things there are that I do not want!”
Socrates & Plato
Mind and body separate (mind continues after death)
Knowledge built in. (nature)
Aristotle driven by data & observation over logic
Knowledge comes from experience (nurture)

29. Experimentation
The purpose of an experiment is to establish a cause-and-effect relationship.
Experiments are the only research method that can establish cause-and-effect.

30. Experimental Design Terms
Hypothesis
Operational definitions
Participant selection
IV & DV
Experimental & control groups
Confounding variables
Random assignment
Placebo control
Double blind procedure
Statistical significance (p value)
Replication

31. Importance of Operational Definitions
Students are more likely to smile for their senior pictures if they have a friendly photographer. IV? DV?
Photographer friendliness
Smiling
Operational definitions are needed for both of these variables. To illustrate the importance of this, have students determine how many of the students on the following slide are smiling.

32. How many smiles?

33. Importance of Operational Definitions
If we want our students to be critical consumers of research, we need to teach them to always ask how research variables were operationalized (“What do they mean by ‘best school’?”). Research cannot be replicated without operational definitions.

34. Replication
Research results that haven’t been replicated are termed “preliminary.” We cannot conclude cause-and-effect from preliminary results because the p value can’t be reduced to 0.

35. Random Sampling vs. Random Assignment
To select participants from population without bias
Allows you to generalize results from the research to the population
Random Assignment
To divide participants into groups
Controls confounding variables—an experimental requirement to “balance” the effects of individual differences

36. Importance of Replication
In 1970, Linus Pauling conducted a famous experiment indicating that vitamin C prevents colds. Over a dozen replication attempts failed. IV DV
Expt. Gp.
Vit C
Expt. Gp.
45%
Fewer colds
Cntrl. Gp.
Placebo

37. Overconfidence
Have students answer the questions on the following slide by writing a small number and a large number such that they are at least 98% certain that the correct answer is in between. Provide answers and by show of hands determine how many students get each wrong. It’s usually about half—far more than the 2% you’d expect if they were 98% certain of their correctness.
The obvious best strategy is to answer with extreme numbers, but few students do because of overconfidence. They are too certain that they can narrow it down.

38. 98% Certainty The area of the US in square miles?
The population of Australia 2007?
American battle deaths in Spanish-American War?
Female psychiatrists in the US in 2005?
Operating nuclear plants worldwide in 2007?

39. 98% Certainty Area of US: Australian pop.: Battle deaths:
Female psychiatrists:
Nuclear plants:
3.6 million sq. miles
20.4 million
385
13,079
435

40. Difficulties with Surveys
Have one side of your class put their heads down while the other side writes the answer to the two questions on the next slide. Then, have the other side answer the questions on second slide.
Students who are primed with the 500 mile question have much smaller estimates of the length of the river than students who are primed with the 3,000 mile question.
Students know that the wording of a question can influence the answers to that question, but they are shocked to learn that the way a first question is worded can influence answers to a second question.

41. Survey Demonstration
1. Is the Mississippi River longer or shorter than 500 miles?
2. How many miles long is it?

42. Survey Demonstration
1. Is the Mississippi River longer or shorter than 3,000 miles?
2. How many miles long is it?