2. Tests of Significance The reasoning of significance tests
Stating hypotheses
Test statistics
p values
Statistical significance
Tests for a population mean
Two-sided tests versus confidence intervals
p values versus fixed alpha

3. Reasoning of significance tests
1 Make a statement (the null hypothesis) about some unknown population parameter.
2 Assuming the null hypothesis is true, what is the probability of obtaining data such as yours?
3 If the probability of the data is small, then reject the null hypothesis.

4. Example 6.6 Tim believes that his “true weight” is 187 pounds
Let’s assume that if Tim weighed himself over and over, the weights would have an approximately normal distribution with s=3
Tim weighs himself once a week for four weeks. The average of these four measurements is 190.5
Are the data consistent with Tim’s belief, or is Tim fooling himself?

6. Example 6.6
mu = 187

We reject the null hypothesis because, if it is true, there is only about a 1% chance of obtaining the data we have.

7. Stating hypotheses Null hypothesis
About the population, not the sample
H0 or NH
“Nothing interesting is happening”
Alternative hypothesis Ha
What a researcher thinks is happening
May be one- or two-sided

8. Test statistics
The test statistic, such as the sample mean, is the information we use to make the decision to reject or keep the null hypothesis.
Usually, the null hypothesis tells us how the test statistic would be distributed if the null hypothesis is true, and if we drew lots and lots of samples at random from the population.

9. p values
If the null hypothesis is true, what is the probability that we would see data such as ours?
P(data|H0) is called the p value
If our sample mean is very different from what the null hypothesis says the population mean is, then the p value will be small (because our data will be unusual, or surprising).

10. Statistical significance
When you do a hypothesis test, you must decide how small the p value must be to lead you to reject the null hypothesis.
It is very common that researchers reject H0 if the p value is less than .05. Sometimes values of .01 or .10 are used.
This arbitrary threshold is called the alpha level.

11. Tests for a population mean Example 6.12
Null hypothesis: mu=450
Alternative hypothesis: mu>450
(Assume population is approximately normal with standard deviation of 100.)
We have a sample of 500 students whose average score is 461.
We reject H0 (because if it is true, then our sample mean is unusually large).

12. Example 6.12 Histogram of means of samples of size 500 if mu=450
We reject the null hypothesis because sample means of 461 or
larger have a very small probability.
(We expect such large means less than 1% of the time.)

13. Two-sided significance tests and confidence intervals
A two-sided significance test which uses the .05 alpha level corresponds to a 95% confidence interval.
That is, if the hypothesized population mean is outside of the 95% confidence interval, then the p value for the hypothesis test will be less than .05.
Ditto for a 90% CI and a = .10, etc.

14. p values versus fixed alpha
In many journal articles you will see statements such as “the null hypothesis was rejected at the .05 level of significance.”
It’s more informative to report the p value.
For example, “the null hypothesis was rejected (p = .032).”

15. Use & Abuse of Tests Choosing a level of significance
What statistical significance doesn’t mean
Don’t ignore lack of significance
Statistical inference is not valid for all sets of data
Beware of searching for significance

16. Power and Inference Power Increasing the power Inference as decision
Two types of error
Error probabilities
The common practice of testing hypotheses

18. Error probabilities When the null hypothesis is true:
P(Type I Error) = alpha
When the null hypothesis is false:
P(Type II Error) = beta

19. Power
When you do a certain hypothesis test, the probability that the test will reject the null hypothesis is called the power of that test.

21. Power is a function of The alpha level What m really is
The size of the sample
The standard deviation of the population

22. Increasing the power
Increase the alpha level (from .05 to .10, for example)
Try to make m really different from the null-hypothesis value
Increase the size of the sample
Try to reduce the standard deviation of the population