1. Reasoning in Psychology Using Statistics
2018

2. Quiz 5 Due Friday March 23rd
Announcements

3. Exam(s) 2 Lab Ex2, mean = 68.6/75 = 91.5%
Lecture Ex2, mean = 52.6/75 = 70.1%
Combined Ex2, mean = 121.2/150 = 80.1%
Exam(s) 2

4. Inferential statistics
Hypothesis testing
Testing claims about populations (and the effect of variables) based on data collected from samples
Estimation
Using sample statistics to estimate the population parameters
Inferential statistics used to generalize back
Sampling to make data collection manageable
Population
Sample
Inferential statistics

5. Inferential statistics
Hypothesis testing
Estimation
Inferential statistics

6. Example: Testing the effectiveness of a new memory treatment for patients with memory problems
No Memory
patients
Test
60 errors
4 error diff
64 errors
Is the 4 error difference:
a “real” difference due to the effect of the treatment
or is it just sampling error?
Hypothesis testing

7. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data from your sample
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
Testing Hypotheses

8. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data from your sample
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
Today’s
focus
Testing Hypotheses

9. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Null hypothesis (H0)
Alternative hypothesis (HA)
This is the one that you test
There are no differences between conditions (no effect of treatment)
Note: This is general form, more complexity is coming up
Not all conditions are equal
You are NOT out to prove the alternative hypothesis
If you reject the null hypothesis, then you are left with support for the alternative(s) (NOT proof!)
Testing Hypotheses

10. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
In our memory example experiment:
About
populations
One -tailed
Our theory is that the treatment should improve memory (fewer errors).
H0:
μTreatment > μNo Treatment
HA:
μTreatment < μNo Treatment
Testing Hypotheses

11. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
In our memory example experiment:
no direction
specified
direction
specified
One -tailed
Two -tailed
Our theory is that the treatment should improve memory (fewer errors).
Our theory is that the treatment has an effect on memory.
H0:
μTreatment > μNo Treatment
H0:
μTreatment = μNo Treatment
HA:
μTreatment < μNo Treatment
HA:
μTreatment ≠ μNo Treatment
Testing Hypotheses

12. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Your alpha (α) level will be your guide for when to reject or fail to reject the null hypothesis (see step 5)
Based on the probability of making making an certain type of error
Type I error
Type II error
This step is basically deciding how big a difference is big enough to feel safe in concluding that there is an effect. This decision is made before the data is collected.
Testing Hypotheses

13. Error types There really is not an effect Real world (‘truth’)
H0 is correct
H0 is wrong
Reject H0
Experimenter’s conclusions
Fail to Reject H0
Error types

14. Error types Real world (‘truth’) I conclude that there is an effect
H0 is correct
H0 is wrong
Reject H0
Experimenter’s conclusions
I cannot detect an effect
Fail to Reject H0
Error types

15. Concluding that there isn’t an effect, when there really is
Real world (‘truth’)
Concluding that there is a difference between groups (“an effect”) when there really isn’t
H0 is correct
H0 is wrong
Type I error
Reject H0
Experimenter’s conclusions
Fail to Reject H0
Type II error
Concluding that there isn’t an effect, when there really is
Error types

16. Error types: Courtroom analogy
Innocent person goes
to jail
Real world (‘truth’)
Defendant is innocent
Defendant is guilty
Type I error
Find guilty
Jury’s decision
Guilty person gets out of jail
Type II error
Find not guilty
Error types: Courtroom analogy

17. Error types Summary of error types
Type I error (α): concluding that there is a difference between groups (“an effect”) when there really is not.
Sometimes called “significance level” or “alpha level”
We try to minimize this (keep it low) by picking a low level of alpha
Psychology: 0.05 and 0.01 most common
Explicit acknowledgement of the uncertainty of our conclusion about the effect, our conclusions are based on likelihood (probability)
Type II error (β): concluding that there is not an effect, when there really is.
Related to the Statistical Power of a test (1-β)
How likely are you able to detect a difference if it is there
Error types

18. How do we estimate our sampling error?
Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data from your sample
Memory
treatment
No Memory
patients
Test
60 errors
64 errors
4 error diff
Is the 4 error difference:
a “real” difference due to the effect of the treatment
or is it just sampling error?
How do we estimate our sampling error?
Testing Hypotheses

19. Inferential Statistics
Consider two bags of marbles (populations)
We can estimate how likely a particular sample is
Bag 1
50 black marbles
50 white marbles
Bag 2
10 black marbles
90 white marbles
sample
p(4black) =
p(4black) =
= 0.5 * 0.5 * 0.5 * 0.5
= 0.1 * 0.1 * 0.1 * 0.1
Roughly 6 out of 100 samples
Roughly 1 out of 10,000 samples
Inferential Statistics

20. Distribution of sample means
A simpler case
Population: 2 4 6 8
All possible samples of size n = 2
Assumption: sampling with replacement
Distribution of sample means

21. Distribution of sample means
A simpler case
Population: 2 4 6 8
All possible samples of size n = 2
There are 16 of them
mean 2 2 2 6 4 5 6 4 7 8 4 6 8 2 2 4 3 8 4 6 2 4 6 4 8 2 8 2 5 4 2 3 4 4
Distribution of sample means

22. Distribution of sample means2 3 4 5 6 7 8 1
In long run, the random selection of tiles leads to a predictable pattern
The distribution of sample means
mean 2 2 2 6 4 5 6 4 7 8 4 6 8 2 2 4 3 8 4 6 2 4 6 4 8 2 8 2 5 4 2 3 4 4
Distribution of sample means

23. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
= 1/16
0.0625
Distribution of sample means

24. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
= 2/16
0.1250
Distribution of sample means

25. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
0.1250 3
= 3/16
0.1875
Distribution of sample means

26. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
0.1250 3
0.1875 4
= 4/16
0.2500
Distribution of sample means

27. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
0.1250 3
0.1875 4
0.2500 3
= 3/16
0.1875
Distribution of sample means

28. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
0.1250 3
0.1875 4
0.2500 3
0.1875 2
= 2/16
0.1250
Distribution of sample means

29. Distribution of sample means2 3 4 5 6 7 8 1 X f p 8 7 6 5 4 3 2 1
0.0625 2
0.1250 3
0.1875 4
0.2500 3
0.1875 2
0.1250 1
= 1/16
0.0625
Distribution of sample means

30. Distribution of sample means2 3 4 5 6 7 8 1
Using the distribution of sample means
Finding out how likely is a particular sample
Sample problem:
What is the probability of getting a sample (n = 2) with a mean of 6 or more? X f p 8 1
0.0625 7 2
0.1250 6 3
0.1875 5 4
0.2500
P(X > 6) =
= 0.375
Same as before, except now we are asking about sample means rather than single scores
Distribution of sample means

31. Distribution of sample means
Null Pop Dist
Experimental Pop Dist 2 4 6 8 4 6 8 10
Magic Trick: take the original bag, wave magic wand and add 2 to each tile
Did the trick work?
Can we tell by looking at a sample
If we got a sample with a mean = 8 which population would you think it came from?
p(X = 8)
This is like the earlier bag of marbles question
Distribution of sample means

32. Distribution of sample means
Null Pop Dist
Experimental Pop Dist
means 2 3 4 5 6 7 8 1 2 4 6 8
means 4 5 6 7 8 9 10 2 3 1 4 6 8 10
Magic Trick: take the original bag, wave magic wand and add 2 to each tile
.375
.0625
Did the trick work?
Can we tell by looking at a sample X f p 8 1
0.0625 7 2
0.1250 6 3
0.1875 5 4
0.2500 X f p 10 1
0.0625 9 2
0.1250 8 3
0.1875 7 4
0.2500 6 5
If we got a sample with a mean = 8 which population would you think it came from?
This one is more likely
= p(X = 8) =
Distribution of sample means

33. Distribution of sample means
Distribution of sample means is a “virtual” distribution between the sample and population
Population
Distribution of sample means
Sample
There is a different one these for each sample size
Distribution of sample means

34. Properties of the distribution of sample means
Shape
If population is Normal, then the dist of sample means will be Normal
If the sample size is large (n > 30), the DSM will be approximately Normal (regardless of shape of the population)
Distribution of sample means
Population
n > 30
Properties of the distribution of sample means

35. Properties of the distribution of sample means
Center
The mean of the dist of sample means is equal to the mean of the population
Population
Distribution of sample means
same numeric value
different conceptual values
Properties of the distribution of sample means

36. Properties of the distribution of sample means
Center
The mean of the dist of sample means is equal to the mean of the population
Consider our earlier example 2 4 6 8
Population
Distribution of sample means
means 2 3 4 5 6 7 8 1 4
μ =
= 5 16 =
= 5
Properties of the distribution of sample means

37. Properties of the distribution of sample means
Spread
Standard deviation of the population
Sample size
The stand. Dev. of the distrib. of sample mean depends on 2 things
Properties of the distribution of sample means

38. Properties of the distribution of sample means
Spread
Standard deviation of the population
The smaller the population variability, the closer the sample means are to the population mean μ X 1 2 3 μ X 1 2 3
Properties of the distribution of sample means

39. Properties of the distribution of sample means
Spread
Sample size
n = 1 μ X
Properties of the distribution of sample means

40. Properties of the distribution of sample means
Spread
Sample size
n = 10 μ X
Properties of the distribution of sample means

41. Properties of the distribution of sample means
Spread
Sample size μ
n = 100
The larger the sample size the smaller the spread X
Properties of the distribution of sample means

42. Properties of the distribution of sample means
Spread
Standard deviation of the population
Sample size
Putting them together we get the standard deviation of the distribution of sample means
- The smaller the population variability, … the smaller the spread
- The larger the sample size the smaller the spread
Commonly called the standard error
Properties of the distribution of sample means

43. Properties of the distribution of sample means
The standard error is the average amount that you would expect a sample (of size n) to deviate from the population mean
In other words, it is an estimate of the error that you’d expect by chance (it is our estimate of the sampling error)
Keep your distributions straight by taking care with your notation
Population σ μ
Distribution of sample means
Sample s X
Properties of the distribution of sample means

44. Properties of the distribution of sample means
All three of these properties are combined to form the Central Limit Theorem
For any population with mean μ and standard deviation σ, the distribution of sample means for sample size n will approach a normal distribution with a mean of μ and a standard deviation of as n approaches infinity
(good approximation if n > 30).
Properties of the distribution of sample means

45. Example: Testing the effectiveness of a new memory treatment for patients with memory problems
No Memory
patients
Test
60 errors
4 error diff
64 errors
Is the 4 error difference:
a “real” difference due to the effect of the treatment
or is it just sampling error?
So if our standard error (estimated sampling error) is
Small (e.g., 1 error), then a 4 error difference is unlikely due to chance, so we will probably conclude there is a treatment effect
Large (e.g., 4 errors), then a 4 error difference may well be due to chance, so we will probably NOT conclude there is a treatment effect
Hypothesis testing

46. In lab In Labs Next time Wed’s focus
Make hypotheses (both null and alternative)
Get a feel for distributions of sample means
Next time
Finishing up steps of hypothesis testing
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data from your sample
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
Wed’s
focus
In lab

47. In lab In Labs Next time Make hypotheses (both null and alternative)
Get a feel for distributions of sample means
Next time
Finish 5 steps of hypothesis testing
Statistics How To: Hypotheses (~4 mins)
StatsLectures: Hypotheses and Error types (~4 mins)
Hypothesis testing and Error types (~7 mins)
Central Limit Theorem (~13 mins)
In lab