1. Nonparametric Statistics Timothy C. Bates tim.bates@ed.ac.uk

2. Parametric Statistics 1 Assume data are drawn from samples with a certain distribution (usually normal) Compute the likelihood that groups are related/unrelated or same/different given that underlying model t-test, Pearson’s correlation, ANOVA…

3. Parametric Statistics 2 Assumptions of Parametric statistics 1. Observations are independent 2. Your data are normally distributed 3. Variances are equal across groups Can be modified to cope with unequal ∂ 2

4. Non-parametric Statistics? Non-parametric statistics do not assume any underlying distribution They estimate the distribution AND compute the probability that your groups are the related/the same or unrelated/different

5. Nonparametric ≠ No parameters Model structure is not specified a priori but is instead determined from data. The data are parameterised by the analysis AKA: “distribution free”

6. Non-parametric Statistics Assumptions of non-parametric statistics 1. Observations are independent

7. Non-parametric Statistics? Non-parametric statistics do not assume any underlying distribution Estimating or modeling this distribution reduces their power to detect effects… So never use them unless you have to

8. Why use a Non-parametric Statistic? Very small samples (<20 replicates) High probability of violating the assumption of normality Leads to spurious Type-1 (false alarm) errors

9. Why use a Non-parametric Statistic? Outliers more often lead to spurious Type- 1 (false alarm) errors in parametric statistics. Nonparametric statistics reduce data to an ordinal rank, which reduces the impact or leverage of outliers.

10. Error Type-I error: False Alarm for a bogus effect reject the null hypothesis when it is really true Type-II error: Miss a real effect fail to reject our null hypothesis when it is really false Type-III error: :-) lazy, incompetent, or willful ignorance of the truth

11. Power 1-alpha

12. Non-parametric Choices Data type? χ2χ2 discrete Question? continuous Number of groups? Spearman’s Rank associationDifferent central value Mann-Whitney U Wilcoxon’s Rank Sums Kruskal-Wallis test two-groupsmore than 2 Brown- Forsythe Difference in ∂ 2

13. Non-parametric Choices Data type? χ2χ2 discrete Question? continuous Number of groups? Spearman’s Rank Like a Pearson’s R Mann-Whitney U Wilcoxon’s Rank Sums Kruskal-Wallis test two-groupsmore than 2 Like ANOVA Like Student’s t No alternative Different central value Brown- Forsythe Difference in ∂ 2 Like F-test association

14. Chi-Squared (Χ 2 ) χ2 tests the null hypothesis that observed events occur with an expected frequency in large samples frequencies are distributed as Χ 2 e.g. Ho: “This six-sided dice is fair ” Expect all 6 outcomes to occur equally often Assumptions Observations are independent Outcomes mutually exclusive Sample is not small Small samples require exact test:, i.e., binomial test

15. Chi-Squared Χ 2 formula Χ 2 = the sum of each squared difference between the observed and expected frequencies divided its expected frequency

16. Χ 2 and contingency tables Χ 2 essentially tests if each cell in a contingency table has its expected value In a 2-way table, this expectation will be the value of an adjacent cell

17. Example: coin toss Random sample of 100 coin tosses, of a coin believed to be fair We observed number of 45 heads, and and 55 tails Is the coin fair?

18. Coin toss If h o is true, our test statistic is drawn from a Χ 2 distribution with df = 1 (45-50) 2 + (55-50) 2 = 0.5 + 0.5 = 1 50 50 Χ 2 (1) = 1, p > 0.3

19. Coin toss Χ 2 in R chisq.test(c(45,55), p=c(.5,.5)) Chi-squared test for given probabilities Χ 2 = 1, df = 1, p = 0.3173

20. Spearman Rank test (ρ (rho)) Named after Charles Spearman, Non-parametric measure of correlation Assesses how well an arbitrary monotonic function describes the relationship between two variables, Does not require the relationship be linear Does not require interval measurement

21. Spearman Rank test (ρ (rho)) Mathematically, it is simply a Pearson’s r computed on ranked data d = difference in rank of a given pair n = number of pairs Alternative test = Kendall's Tau (Kendall's τ)

22. Mann-Whitney U AKA: “Wilcoxon rank-sum test Mann & Whitney, 1947; Wilcoxon, 1945 Non-parametric test for difference in the medians of two independent samples Assumptions: Samples are independent Observations can be ranked (ordinal or better)

23. Mann-Whitney U U tests the difference in the medians of two independent samples n 1 = number of obs in sample 1 n 2 = number of obs in sample 2 R = sum of ranks of the lower-ranked sample

24. Mann-Whitney U or t-test? Should you use it over the t-test? Yes if you have a very small sample (<20) (central limit assumptions not met) Possibly if your data are inherently ordinal Otherwise, probably not. It is less prone to type-I error (spurious significance) due to outliers. But does not in fact handle comparisons of samples whose variances differ very well (Use unequal variance t-test with rank data)

25. Aesop: Mann-Whitney U Example Suppose that Aesop is dissatisfied with his classic experiment in which one tortoise was found to beat one hare in a race. He decides to carry out a significance test to discover whether the results could be extended to tortoises and hares in general…

26. Aesop 2: Mann-Whitney U He collects a sample of 6 tortoises and 6 hares, and makes them all run his race. The order in which they reach the finishing post (their rank order) is as follows: tort = c(1, 7, 8, 9, 10,11) hare = c(2, 3, 4, 5, 6, 12) Original tortoise still goes at warp speed, original hare is still lazy, but the others run truer to stereotype.

27. Aesop 3: Mann-Whitney U wilcox.test(tort, hare) Wilcoxon = W = 25, p-value = 0.31 Tortoises are not faster (but neither are hares) tort = c(1, 7, 8, 9, 10,11) (n 2 = 6) hare = c(2, 3, 4, 5, 6, 12) (n 1 = 6, R 1 =32)

28. Aesop 4: Mann-Whitney U Wilcoxon = W = 25, p-value = 0.31 Tortoises are not faster (but neither are hares). Welch Two Sample t-test t = 1.1355, df = 10, p-value = 0.28 Alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -2.25 ~ 6.91 sample estimates: mean of x = 7.6 mean of y = 5.3

29. Power comparison with continuous normal data tort = 1 74 79 81 100 121 hare = 4 9 16 17 18 144 Wilcoxon W = 25, p = 0.31 t.test t.test(tort, hare, var.equal = TRUE) t(10) = 1.5, p = 0.16

30. Wilcoxon signed-rank test (related samples) Same idea as MW U, generalized to matched samples Equivalent to non-independent sample t- test

31. Kruskall-Wallis Non-parametric one-way analysis of variance by ranks (named after William Kruskal and W. Allen Wallis) tests equality of medians across groups. It is an extension of the Mann-Whitney U test to 3 or more groups. Does not assume a normal population, Assumes population variances among groups are equal.