1. S519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 8: Significantly significant

2. Last week

3. This week What is significance and why it is important Type I and Type II errors How inferential statistics works How to select the proper statistical test for your research

4. The concept of significance Significance Any difference between the attitudes of the two groups is due to some systematic influence and not due to chance.

5. Significance Example Hypothesis: There is a significant difference in attitude toward maternal employment between adolescents whose mothers work and adolescents whose mothers do not work, as measured by a test of emotional state. There are many other reasons to affect this hypothesis, for example?

6. Significant level Significant level is the risk associated with not being 100% confident that the null is true (there is no difference between data or variables) If the significant finding occurred at the 0.05 level (p<0.05), this means that there is 1 chance in 20 (5%) that there is no difference in data (null is true): any differences found were not due to the hypothesized reason, but to some other unknown reasons, or by chance.

7. Significant level p-value The probability for the null hypothesis to be true The probability for no difference in data or variables p>0.05 (non significant): more than 5% chance (5% to 99%) that the null is true (no difference in data) – accept null p<0.05 (statistically significant): less than 5% chance that the null is true (no difference in data) – reject null

8. Null and research hypothesis Research hypothesis There is a difference in the academic achievement of children who participated in a preschool program and children who did not participate. Null hypothesis The two groups are equal to each other on some measure of achievement. As a good researcher, your job to show that any difference that exists between these two groups is due only to the effects of the preschool experience.

9. Statistical significance A result is called statistically significant  it is unlikely to cause by chance A statistically significant difference  there is statistical evidence that there is a difference It does not mean that the difference is necessarily large, important, or significant in common understanding

10. Null hypothesis Either true or false But null cannot be tested directly (as it is applied to the population) The researchers do not know the real true nature of the null hypothesis, and it is hard to know and to test That is why we need inferential statistics

11. Table 8.1 (s-p179) Accept the null hypothesisReject the null hypothesis The null hypothesis is really true, you accepted a null when it is true [there is really no difference between the groups] Type I error: reject a null hypothesis when it is true (represented by the Greek letter alpha, α) [there is really no difference between the groups] The null hypothesis is really false Type II error: accepted a false null hypothesis (represented by Greek letter beta, β) [there really are differences between the two groups], you rejected the null hypothesis which is false. [there really are differences between the two groups]

12. Type I error Defines the risk that you are willing to take in any test of the null hypothesis Conventional: 0.01 ~ 0.05 Example: if the level of significance is 0.05  there is a 5% chance you will make the Type I error: to reject it when the null is true. It is not proper to say “on 100 tests of the null hypothesis, I will make an error on only 5” As it is normally associated with one test

13. Type I error p<.05 or p<.01 (reject the null) The risk to make the Type I error (reject the real true null) is less than 5% or 1% chance p>.05 or p=n.s. (nonsignificant) (accept the null) The probability to make the Type I error (reject the real true null) exceeds.05 DegreeClosenessBetweenness Citation counts 0.42230.56140.7282 p=0.0060p=0.0100p=0.0003

14. Type II error The risk of accepting the real false null hypothesis It is sensitive to the number of subjects in a sample The size of sample increases  Type II error decreases If the sample is more closer to the population, the likelihood that you will accept a false null hypothesis decreases

15. Significance Statistical significance means Example: Group 1 with training to read using computer, Group 2 with training to read using classroom teaching A reading test: Group 1=75.6, Group 2=75.7 When using t test: result is statistically significant at the.01 level How to interpret: computers do better than classroom teaching

16. Inferential statistics Descriptive statistics: describe the characters of a sample Inferential statistics: infer something about the population based on the sample’s character

17. How inference works Mother-work group and mother-not-work group Select representative samples of two groups Conduct a test for each member in these two groups, calculate the mean scores Select a proper statistical test Draw a conclusion (to a population): If statistically significant: the difference is due to moms If not significant: the difference is not due to moms

18. How to select which test Flow chart (s-p186) http://rimarcik.com/en/navigator/

19. A template for significant test 1. a statement of the null hypothesis 2. setting a level of risk associated with the null hypothesis (level of significance or Type I error, p) 3. select a proper statistical test (see Fig 8.1) 4. set up the sample and experiment, and compute the test statistic value 5. determine the value needed for rejection of the null hypothesis using proper tables – critical value (see appendix) 6. compare the computed value and the obtained value 7. if computed value > critical value: reject the null; if computed value < critical value: accept the null

20. Exercise Are the following statements true, and why: A Type I error of 0.05 means that in 5 tests out of 100 tests of the research hypothesis, I will reject a true null hypothesis It is possible to set the Type I error to 0 The smaller the Type I error rate, the better the results