1. Statistics for the Social Sciences
Psychology 340
Spring 2010
Hypothesis testing

2. Reminders Don’t forget to complete homework 4 for Feb 9 (Tues)
And Quiz 3 (Chapters 5, 6, & 7) by 11AM Thurs (Feb 4)
Exam 1 Feb 11 (Thurs)

3. Outline (for the week) Review of: Hypothesis testing framework
Stating hypotheses
General test statistic and test statistic distributions
When to reject or fail to reject
Effect sizes: Cohen’s d
Statistical Power

4. Hypothesis testing
Example: Testing the effectiveness of a new memory treatment for patients with memory problems
Our pharmaceutical company develops a new drug treatment that is designed to help patients with impaired memories.
Before we market the drug we want to see if it works.
The drug is designed to work on all memory patients (the population), but we can’t test them all.
So we decide to use a sample and conduct the following experiment.
Based on the results from the sample we will make conclusions about the population.

5. Hypothesis testing
Example: Testing the effectiveness of a new memory treatment for patients with memory problems
Memory
treatment
No Memory
patients
Test
55 errors
5 error diff
60 errors
Is the 5 error difference:
A “real” difference due to the effect of the treatment
Or is it just sampling error?

6. Testing Hypotheses Hypothesis testing
Procedure for deciding whether the outcome of a study (results for a sample) support a particular theory (which is thought to apply to a population)
Core logic of hypothesis testing
Considers the probability that the result of a study could have come about if the experimental procedure had no effect
If this probability is low, scenario of no effect is rejected and the theory behind the experimental procedure is supported

7. Inferential statistics
Hypothesis testing
A five step program (note: these steps are different than the book’s)
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis

8. Hypothesis testing Hypothesis testing: a five step program
Step 1: State your hypotheses: as a research hypothesis and a null hypothesis about the populations
Null hypothesis (H0)
Research hypothesis (HA)
This is the one that you test
There are no differences between conditions (no effect of treatment)
Generally, not all groups are equal
You aren’t out to prove the alternative hypothesis
If you reject the null hypothesis, then you’re left with support for the alternative(s) (NOT proof!)

9. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
In our memory example experiment:
One -tailed
Our theory is that the treatment should improve memory (fewer errors).
H0:
HA:

10. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
In our memory example experiment:
no direction
specified
direction
specified
One -tailed
Two -tailed
Our theory is that the treatment should improve memory (fewer errors).
Our theory is that the treatment has an effect on memory.
H0:
H0:
HA:
HA:

11. One-Tailed and Two-Tailed Hypothesis Tests
Directional hypotheses
One-tailed test
Nondirectional hypotheses
Two-tailed test

12. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Your alpha (α) level will be your guide for when to reject or fail to reject the null hypothesis.
Based on the probability of making making an certain type of error
Essentially this is the process of deciding, in advance of collecting your observations, how big a difference between groups is needed to reject the null hypothesis

13. Performing your statistical test
What are we doing when we test the hypotheses?
Real world (‘truth’)
H0: is true (no treatment effect)
H0: is false (is a treatment effect)
One population
Two populations XA
the memory treatment sample are the same as those in the population of memory patients. XA
they aren’t the same as those in the population of memory patients

14. Error types There really isn’t an effect Real world (‘truth’)
One pop
Real world (‘truth’)
There really is
an effect
Two pops
H0 is correct
H0 is wrong
Reject H0
Experimenter’s conclusions
Fail to Reject H0

15. Error types Real world (‘truth’) I conclude that there is an effect
H0 is correct
H0 is wrong
Reject H0
Experimenter’s conclusions
I can’t detect an effect
Fail to Reject H0

16. Error types Real world (‘truth’) H0 is correct H0 is wrong
Type I error
Reject H0
Experimenter’s conclusions
Fail to Reject H0
Type II error

17. Error types
Type I error (α): concluding that there is a difference between groups (“an effect”) when there really isn’t.
Sometimes called “significance level” or “alpha level”
We try to minimize this (keep it low)
Type II error (β): concluding that there isn’t an effect, when there really is.
Related to the Statistical Power of a test (1-β)

18. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data

19. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Descriptive statistics (means, standard deviations, etc.)
Inferential statistics (z-test, t-tests, ANOVAs, etc.)

20. Performing your statistical test
What are we doing when we test the hypotheses?
Computing a test statistic: Generic test
Could be difference between a sample and a population, or between different samples
Based on standard error or an estimate of the standard error

21. Distribution of sample means
A distribution of all possible sample means drawn from the population (of a particular sample size)
Population σ μ
Distribution of sample means X 3 X 1
Mean of all samples of n = # X 4 X 2
Much more detail about this in the next lecture

22. “Generic” statistical test
The generic test statistic distribution (a transformation of the distribution of sample means)
To reject the H0, you want a computed test statistics that is large
What’s large enough?
The alpha level gives us the decision criterion
Distribution of sample means
Distribution of the test statistic
Transform to using statistical test
α-level determines where these boundaries go

23. Testing Hypotheses Hypothesis testing: a five step program
Step 1: State your hypotheses
Step 2: Set your decision criteria
Step 3: Collect your data
Step 4: Compute your test statistics
Step 5: Make a decision about your null hypothesis
Based on the outcomes of the statistical tests researchers will either:
Reject the null hypothesis
Fail to reject the null hypothesis
This could be correct conclusion or the incorrect conclusion

24. “Generic” statistical test
The generic test statistic distribution (think of this as the distribution of sample means)
To reject the H0, you want a computed test statistics that is large
What’s large enough?
The alpha level gives us the decision criterion
Distribution of the test statistic
If test statistic is here Reject H0
If test statistic is here Fail to reject H0

25. “Generic” statistical test
The alpha level gives us the decision criterion
Two -tailed
One -tailed
Reject H0
Fail to reject H0
α = 0.05
0.025
split up
into the
two tails
Reject H0
Fail to reject H0
Reject H0
Fail to reject H0

26. “Generic” statistical test
The alpha level gives us the decision criterion
Two -tailed
One -tailed
Reject H0
Fail to reject H0
α = 0.05
0.05
all of it in one tail
Reject H0
Reject H0
Fail to reject H0
Fail to reject H0

27. “Generic” statistical test
The alpha level gives us the decision criterion
Two -tailed
One -tailed
Reject H0
Fail to reject H0
α = 0.05
all of it in one tail
0.05
Reject H0
Reject H0
Fail to reject H0
Fail to reject H0

28. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
Step 1: State your hypotheses
One -tailed
We give a n = 16 memory patients a memory improvement treatment.
H0:
the memory treatment sample are the same as those in the population of memory patients.
After the treatment they have an average score of = 55 memory errors.
μTreatment ≥ (μpop = 60)
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
HA:
the memory treatment sample make fewer errors the the population
μTreatment < (μpop = 60)

29. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
Step 2: Set your decision criteria
α = 0.05
After the treatment they have an average score of = 55 memory errors.
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?

30. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
α = 0.05
Step 3: Collect your data
After the treatment they have an average score of = 55 memory errors.
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?

31. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
α = 0.05
Step 4: Compute your test statistics
After the treatment they have an average score of = 55 memory errors.
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
Why isn’t this just σ?
This is the standard error (σX) rather than the population standard deviation (σ). It is the standard deviation of the distribution of sample means. The formula for this is:
We will cover in detail in the next lecture.

32. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
α = 0.05
Step 4: Compute your test statistics
After the treatment they have an average score of = 55 memory errors.
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
= -2.5

33. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
α = 0.05
After the treatment they have an average score of = 55 memory errors.
Step 5: Make a decision about your null hypothesis
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8? 5%
Reject H0

34. “Generic” statistical test
An example: One sample z-test
Memory example experiment:
H0: μTreatment ≥ (μpop = 60)
HA: μTreatment < (μpop = 60)
We give a n = 16 memory patients a memory improvement treatment.
One -tailed
α = 0.05
After the treatment they have an average score of = 55 memory errors.
Step 5: Make a decision about your null hypothesis
- Reject H0
How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
- Support for our HA, the evidence suggests that the treatment decreases the number of memory errors