1. How hypothesis testing works
The Right Questions about Statistics:
How hypothesis testing works
Maths Learning Centre
The University of Adelaide
A hypothesis test is designed to DECIDE the answer to a YES OR NO question using DATA.
You use probability to calculate how consistent your data is with a particular answer.

2. Is the median number of chapters in a novel = 20?
Randomly choose 12 books

3. Book
# Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20
How likely is this to happen if the median for all books really is 20?
How likely is this to happen if the median for all books really is 20?
To answer this, I need to know what all the other possibilities are and compare...
That’s too hard.
So let’s turn all this data into just one number...
c = 0.91
0.2
0.4
0.6
0.8
1.0
“TEST STATISTIC”

4. Suppose the median number of chapters for all books really is 20.
And imagine what would happen if we took a different sample.
“NULL HYPOTHESIS”
0.2
0.4
0.6
0.8
1.0

5. Book
# Chapters 1 22 2 3 17 4 15 5 19 6 7 18 8 20 9 25 10 11 12 21
c = 0.55
0.2
0.4
0.6
0.8
1.0

6. Book
# Chapters 1 23 2 17 3 26 4 21 5 25 6 7 24 8 9 12 10 20 11 15
c = 0.27
0.2
0.4
0.6
0.8
1.0

7. Book
# Chapters 1 16 2 22 3 15 4 14 5 18 6 17 7 19 8 30 9 21 10 20 11 29 12
c = 0.55
0.2
0.4
0.6
0.8
1.0

8. Book
# Chapters 1 16 2 14 3 29 4 15 5 25 6 7 8 17 9 13 10 11 23 12 20
c = 0.64
0.2
0.4
0.6
0.8
1.0

9. Book
# Chapters 1 14 2 21 3 20 4 19 5 26 6 15 7 8 12 9 27 10 18 11 22
c = 0.55
0.2
0.4
0.6
0.8
1.0

10. Book
# Chapters 1 15 2 3 21 4 20 5 12 6 7 22 8 18 9 10 11 13
c = 0.73
0.2
0.4
0.6
0.8
1.0

11. Book
# Chapters 1 29 2 18 3 16 4 25 5 13 6 26 7 30 8 20 9 10 12 11 24
c = 0.36
0.2
0.4
0.6
0.8
1.0

12. Book
# Chapters 1 20 2 18 3 12 4 21 5 17 6 15 7 8 9 30 10 13 11 23
c = 0.73
0.2
0.4
0.6
0.8
1.0

13. Book
# Chapters 1 32 2 19 3 28 4 20 5 14 6 25 7 27 8 24 9 12 10 11 21 15
c = 0.45
0.2
0.4
0.6
0.8
1.0

14. Book
# Chapters 1 25 2 17 3 4 23 5 6 16 7 8 18 9 10 20 11 13 12 15
c = 0.64
0.2
0.4
0.6
0.8
1.0

15. Book
# Chapters 1 ## 2 3 4 5 6 7 8 9 10 11 12
c = 0.18
c = 0.82
c = 0.55
c = 0.45
c = 0.64
c = 0.27
c = 0.55
c = 0.45
0.2
0.4
0.6
0.8
1.0

16. “DISTRIBUTION OF TEST STATISTIC”
This describes all the possibilities for what the test statistic could have been and how likely they all are.
We can usually go straight to this distribution if we use some statistical theory
0.2
0.4
0.6
0.8
1.0
“DISTRIBUTION OF TEST STATISTIC”

17. I need to chose a cut-off for when I declare this to be unlikely.
NOW: How likely is my test statistic if the median for all books really is 20?
That is, what percentage of the possible test statistics are just as bad as mine?
p = 0.01
“P VALUE”
I need to chose a cut-off for when I declare this to be unlikely.
Let’s choose 0.05.
p = 0.01 is less than 0.05
So our test statistic unlikely.
So conclude the median is not 20.
0.2
0.4
0.6
0.8
1.0

18. Let’s go over that again…
Book
# Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20
Is the median number of chapters = 20?
Take a random sample of books.
Calculate a test statistic.
c = 0.91
Figure out the distribution if the median really is 20.
B(11,0.5)
Calculate a P-value.
p = 0.01
Decide the answer.
The median for all books is not 20.
“HYPOTHESIS TEST”
0.2
0.4
0.6
0.8
1.0
“SIGN TEST”

19. Another hypothesis test
Book
Weight (g) 1
242 2
366 3
424 4
312 5
307 6
238 7
317 8
265 9 10
314 11
217 12
379
Is the mean weight equal to 300 g?
Take a random sample of books.
Calculate a test statistic.
t = 0.46
Figure out the distribution if the mean really is 300g.
t (11)
Calculate a P-value.
p = 0.65
Decide the answer.
The mean for all books could be 300g.
“ONE-SAMPLE T-TEST”
-2.0
-1.0
0.0
1.0
2.0

20. Another hypothesis test
Book
Title has “the” 1 No 2
Yes 3 4 5 6 7 8 9 10 11 12
Is the percentage of books with “the” in the title 50% ?
Take a random sample of books.
Calculate a test statistic.
x = 3
Figure out the distribution if the percentage really is 50%.
B(12,0.5)
Calculate a P-value.
p = 0.09
Decide the answer.
The percentage with “the” could be 50%.
“EXACT TEST FOR ONE PROPORTION” 2 4 6 8 10

21. Another hypothesis test
Is the standard deviation of weights equal to 40g?
Book
Weight (g) 1
242 2
366 3
424 4
312 5
307 6
238 7
317 8
265 9 10
314 11
217 12
379
Take a random sample of books.
Calculate a test statistic.
χ2 = 22.9
Figure out the distribution if the standard deviation really is 40g.
χ2(11)
Calculate a P-value.
p = 0.03
Decide the answer.
The standard deviation of weights is not 40g. 5 10 15 20 25
“CHI-SQUARED TEST FOR ONE STANDARD DEVIATION”

22. Another hypothesis test
Are SciFi books the same thickness as other books?
Book
Genre
Thickness (cm) 1
SciFi
4.2 2
Other
2.4 3
3.6 4
2.7 5
3.9 6
2.9 7
3.7 8
3.0 9
4.0 10
2.0 11 12
2.8
Take a random sample of books.
Calculate a test statistic.
t = 6.62
Figure out the distribution if the average thickness really is the same.
t(20)
Calculate a P-value.
p =
Decide the answer.
SciFi books are not the same thickness as other books. 2 6 4 -6 -2 -4
“TWO-SAMPLE T-TEST”

23. Another hypothesis test
Are SciFi books more likely to be written by men?
Book
Genre
Author Gender 1
SciFi
Male 2
Other
Female 3 4 5 6 7 8 9 10 11 12
Take a random sample of books.
Calculate a test statistic.
χ2 = 1.37
Figure out the distribution if the average thickness really is the same.
χ2(1)
Calculate a P-value.
p = 0.24
Decide the answer.
SciFi books are not more likely to be written by men. 6 8 12 10 4 2
“CHI-SQUARED TEST FOR ASSOCIATION”

24. So this is how you do a hypothesis test:
Have a yes-or-no question. (One of the answers is the “null hypothesis”.)
Collect data.
Calculate a test statistic.
Figure out what the distribution would be if you suppose the null hypothesis.
Calculate a p-value.
Decide the answer based on the p-value.

25. And this is how you interpret hypothesis test:
A low p-value means your data is not very consistent with the null hypothesis and you don’t believe you’re in that situation.
A high p-value means your data is consistent with the null hypothesis and you do believe you could possibly be in that situation.
very strong evidence that the null is false
strong evidence that the null is false
some evidence that the null is false.
no evidence that the null is false.
0.01
0.05
0.10
significance level
reject null
retain null