Design and Data Analysis in Psychology I School of Psychology Dpt. Experimental Psychology Design and Data Analysis in Psychology I Salvador Chacón Moscoso Susana Sanduvete Chaves

Null hypothesis significance testing Lesson 8 Null hypothesis significance testing Pérez, J., Manzano, V., & Fazeli, H. (1999). Análisis de datos en Psicología. Madrid: Pirámide.

1. Introduction Statistical decision: procedure to choose an alternative based on statistics. E.g., is cognitive therapy an useful treatment to reduce anxiety? Group Experimental Control Mean 8 10

2. Steps 2. Calculations to check if H0 is true or false. 1. Concrete the null hypothesis (H0): there are not differences between groups; there is not relationship between variables. 2. Calculations to check if H0 is true or false. When H0 is rejected, then the alternative hypothesis (H1) is accepted: there are differences between groups; there is relationship between variables. 3. Take a final decision, assuming some level of risk (probability of being wrong in your decision).

3. Estimation vs. decision Similarity: both are based on the same probabilistic theory or background Differences: Estimation Decision Problems that solves Calculate the percentage of 14-year-old smokers Are there more than 60% of 14-year-old smokers? Solution A value Yes/no

4. Statistical decision. Example One year ago, the opinion about abortion in the population presented a mean of 10.57. Nowadays, we obtained a mean of 12 from a sample of 200 participants with a standard deviation of 7.2. Has the opinion changed? (α, the probability of being wrong when you accept the H0 ,is 0.05).

4. Statistical decision. Example Steps: 1. Concrete the null hypothesis (H0). 2. Calculations to check if H0 is true or false: 3. Take a final decision: - If the mean is in the interval = H0 is accepted (there are not differences). - If the mean is not in the interval = H0 is rejected (there are differences).

4. Statistical decision. Example Steps: 1. H0: there are not differences; the opinion has not changed. 2. Calculations to check if H0 is true or false (two ways):

4. Statistical decision. Example 3. Take a final decision (two ways): The mean (12) is not in the interval [9.57-11.57] or The mean (10.57) is not in the interval [11-13] - H0 is rejected. There are differences. The opinion about abortion has statistically changed in one year. H0 H1 H1 10.57 11.57 9.57

5. Statistical significance It is the probability of obtaining values out of the interval calculated (the most extreme values). When p ≤ α → H0 is rejected. When p > α → H0 is accepted.

5. Statistical significance Decisions about the H0 can be taken based on the confidence interval or the statistical significance. In both cases, the decision is going to be always the same, So you do not have to do both procedures; one is enough.

5. Statistical significance. Example 1 In the previous example, calculate the statistical significance. 10.57 12

5. Statistical significance. Example 1 0.4974 0.0026 10.57 12 9.57 2.8 -2.8

5. Statistical significance. Example 1 The probability of obtaining a value out of the interval is 0.0052. p ≤ α 0.0052 < 0.05 → H0 is rejected. There are differences. The opinion about abortion has statistically changed in one year. When we used the confidence interval to take the decision about the H0, the conclusion obtained was the same.

5. Statistical significance. Example 2 Previous studies found that the average intellectual coefficient in children between 10 and 12 years old is 105. In a sample of 30 children, the mean obtained was 113 and its standard deviation, 20.5. Can we consider that children between 10 and 12 years old have an intellectual coefficient of 105? (α = 0.05).

5. Statistical significance. Example 2 0.4821 0.0179 105 113 2.1 -2.1

5. Statistical significance. Example 2 0.0358 < 0.05 → H0 is rejected. There are differences. We can not consider that children between 10 and 12 years old have an intellectual coefficient of 105.

5. Statistical significance. Example 3 The 73% of the population presented at least one anxiety attack. 70 adults participated in a relaxation program. 30 of them never suffered an anxiety attack. Is the program effective? Does the program decrease significantly the number of anxiety attacks? (α = 0.05).

5. Statistical significance. Example 3 - If 30 did not suffer an anxiety attack, then 40 suffered it. - 0.57<0.73, so the program could be effective. 0.4990 0.001 0.73 0.57 -3.1

5. Statistical significance. Example 3 The probability of obtaining a value out of the interval is 0.002. p ≤ α 0.002 < 0.05 → H0 is rejected. There are differences. We can consider that the program is effective. It decreases significantly the number of anxiety attacks.

6. Decisions about α The value of α: Depends on the consequences of being wrong in your decision when you reject the null hypothesis. Higher seriousness implies lower α; e.g. Is a treatment against cancer effective? Vs. Is a program to learn to read effective? Should be chosen before the statistical analysis to increase the credibility of the results and decisions taken based on them.

7. Standardized distance The same decision can be taken comparing Z scores instead of statistical significances (so you do not need to use the tables). When Z0 ≤ Zα/2 → H0 is accepted. There are not differences between groups. There is not relationship between variables. When Z0 > Zα/2 → H0 is rejected. There are differences between groups. There is relationship between variables.

7. Standardized distance H0 H1 Zα/2

7. Standardized distance. Example 1 Take the decision in the previous exercise 1, concluding with Z scores (problem in slide 6; Z0 in slide 13).

7. Standardized distance. Example 1 2.8 > 1.96 → H0 is rejected. There are differences. The opinion about abortion has statistically changed in one year. Z0=2.8 Zα/2=1.96

7. Standardized distance. Example 2 Take the decision in the previous exercise 2, concluding with Z scores (problem in slide 15; Z0 in slide 16).

7. Standardized distance. Example 2 2.1 > 1.96 → H0 is rejected. There are differences. We can not consider that children between 10 and 12 years old have an intellectual coefficient of 105. Z0=2.1 Zα/2=1.96

7. Standardized distance. Example 3 Take the decision in the previous exercise 3, concluding with Z scores (problem in slide 18; Z0 in slide 19).

7. Standardized distance. Example 3 3.1 > 1.96 → H0 is rejected. There are differences. We can consider that the program is effective. It decreases significantly the number of anxiety attacks. Z0=-3.1 Z0=3.1 Zα/2=-1.96 Zα/2=1.96

7. Standardized distance. Example 4 Spanish books about Psychology have a mean of 250 pages. Taking into account that I have 42 books with a mean of 280 pages and a standard deviation of 100 pages, can I conclude that my books are larger than usual? (α = 0.05).

7. Standardized distance. Example 4

7. Standardized distance. Example 4 1.92 < 1.96 → H0 is accepted. There are not differences. I can not conclude that my books are statistically larger than usual.