1. PHL - 541

2. Test of Significance: General Purpose The idea of significance testing. If we have a basic knowledge of the underlying distribution of a variable, then we can make predictions about how, in repeated samples of equal size, this particular statistic will "behave," that is, how it is distributed. Normal distribution (Gaussian distribution) of variables.  About 68 % of values drawn from a normal distribution are within one SD, σ > 0 away from the mean μ.  About 95 % of the values are within two SD.  About 99.7 % lie within three SD.  This is known as the 68-95-99.7 Sigma rule. Example: If we draw 100 random samples of 100 adults each from the general population, and compute the mean height in each sample, then the distribution of the standardized means across samples will likely approximate the normal distribution (Student's t distribution with 99 df). Now imagine that we take an additional sample in a particular city ("X") where we believe that people are taller than the average population. If the mean height in that sample falls outside the upper 95% tail area of the t distribution then we conclude that, indeed, the people of X city are taller than the average population. Are most variables normally distributed?  Income in the population.  Incidence rates of rare diseases.  Heights in different cities.  Number of car accidents,,,,,,,,,,,,,

3. Test of significance: General Purpose Sample size: Large enough or Very small. If very small, then those tests based on the assumption can be used only if we are sure that the variable is normally distributed, and there is no way to test this assumption if the sample is small (<5, unpaired). Measurement Scales: a)Interval variables allow us not only to rank order the items that are measured, but also to quantify and compare the sizes of differences between them. For example, temperature, as measured in degrees Fahrenheit or Celsius, represent an interval scale. We can say that a temperature of 40 degrees is higher than a temperature of 30 degrees, and that an increase from 20 to 40 degrees is twice as much as an increase from 30 to 40 degrees. b)Ratio variables are very similar to interval variables. Most statistical data analysis procedures do not distinguish between the interval and ratio properties of the measurement scales. c)Nominal variables (qualitative). For example, we can say the two individuals are different in terms of variable A (e.g., they are of different gender, colour, city, race etc.), but we cannot say which one "has more" of the quality represented by the variable. b)Ordinal variables allow us to rank order the items we measure in terms of which has less and which has more of the quality represented by the variable, but still they do not allow us to say "how much more". Examples; good-better-best, upper-middle-lower.

4. Differences between parametric and nonparametric methods Parametric Test ProceduresNonparametric Test Procedures Involve population parameters like mean and SD of population distribution. Do not involve population parameters and don’t assume data is normal, or t-distributed. The underlying measurements are at least of interval, meaning that equally spaced intervals on the scale can be compared in a consequential manner. For example, temperature, as measured in degrees F or C, constitutes an interval scale. We can say that a temperature of 40>30 degrees, and that an increase from 20 to 40 degrees is twice as much as an increase from 30 to 40 degrees. Dependent variable may be measured on any scale method  Interval or Ratio where distribution of the random variable of interest is unspecified  Ordinal-scaled data or ranking: good- better-best, upper-middle-lower.  Nominal-scaled data: gender, race, color, city. Often requires large sample sizes to call to normality Sample sizes can be small Based on normality assumptions i.e. have stringent assumptions: normal distribution Have few assumptions about the population distribution (only assume random sample). Examples: t-Test, Z Test, F test, ANOVAExample: Mann-Whitney U test, Chi-square test

5. Alternatives of Parametric and Nonparametric Methods Basically, there is at least one nonparametric equivalent for each parametric general type of test. In general, these tests fall into the following categories: Tests of differences between groups (independent samples); 1. t-test for independent samples  alternatives  the Mann-Whitney U test. 2. ANOVA  alternatives  Kruskal-Wallis analysis of ranks. Tests of differences between variables (dependent samples); 1. t-test for dependent samples  alternatives  Sign test and Wilcoxon's matched pairs test or McNemar's Chi-square (If the variables of interest are dichotomous in nature (i.e., "pass" vs. "no pass"). 2. Repeated measures ANOVA  alternatives  Friedman's two-way analysis of variance and Cochran Q test (if the variable was measured in terms of categories, e.g., "passed" vs. "failed"). Tests of relationships between variables. Correlation coefficient  equivalents to  Spearman R, Kendall Tau, Coefficient Gamma, Chi-square test, the Phi coefficient, and the Fisher exact test. If the two variables of interest are categorical in nature (e.g., "passed" vs. "failed" by "male" vs. "female")

6. The Most Frequently Used Nonparametric Ttests ExampleParametric testNon-parametricPurpose of test To compare girls ’ heights with boys ’ heights Two-sample (unpaired) t test Mann-Whitney U test Compares two independent samples drawn from the same population To compare weight of infants before and after a feed One sample (paired) t test Wilcoxon matched pairs test Compares two sets of observations on a single sample To determine whether plasma glucose is higher one, two, or three hours after a meal One way analysis of variance (F test) using total sum of squares Kruskal-Wallis analysis of variance by ranks Effectively, a generalization of the paired t or Wilcoxon matched pairs test where three or more sets of observations are made on a single sample In the above example, to determine whether the results differ in male and female subjects Two way analysis of variance Two way analysis of variance by ranks As above, but tests the influence (and interaction) of two different covariates To determine whether acceptance into medical school is more likely if the applicant was born in the same country χ2 test Fisher ’ s exact test Tests the null hypotheses that the distribution of a discontinuous variable is the same in two (or more) independent samples To assess whether and to what extent plasma HbA1 concentration is related to plasma triglyceride concentration in diabetic patients Product moment correlation coefficient (Pearson ’ s r) Spearman ’ s rank coefficient (r2) Assesses the strength of the straight line association between two continuous variables

7.  It is not easy to give simple advice concerning the use of nonparametric procedures.  Each nonparametric procedure has its peculiar sensitivities and blind spots.  In general, if the result of a study is important (e.g., does a very expensive and painful drug therapy help people get better?), then it is always useful to run different nonparametric tests; should discrepancies in the results occur dependent on which test is used, one should try to understand why some tests give different results.  On the other hand, nonparametric statistics are less statistically powerful (less sensitive) than their parametric counterparts, and if it is important to detect even small effects one should be very careful in the choice of a test statistic. When to Use Which Method?

8. Nonparametric Methods: Mann-Whitney U or Wilcoxon rank sum test (MWW) Frank Wilcoxon (1892-1965)  Mann-Whitney U test is one of the best-known non-parametric significance test and is included in most modern statistical packages. It is also easily calculated by hand for small samples.  It was proposed initially by Frank Wilcoxon in 1945 for equal sample sizes, and extended to arbitrary (random) sample sizes and in other ways by Mann and Whitney (1947).  MWW test is practically identical to performing an ordinary parametric two-sample t test on the data after ranking over the combined samples. They are excellent alternatives to the t test if your data are significantly skewed.  MWW test tests for differences in medians and for chances of obtaining greater observations in one population versus the other.  The null hypothesis in the MWW test is that both populations have the same probability of exceeding each other. i.e. no difference in the two population distributions.  The alternative hypothesis is that the variable in one population is stochastically greater.  The test involves the calculation of a statistic, usually called U (the sum of ranks), whose distribution under the null hypothesis is known. In the case of small samples, the distribution is tabulated, but for sample sizes above ~ 20 there is a good approximation using the normal distribution methods.

9. Nonparametric Methods: Mann-Whitney U test Assumptions for Mann-Whitney U Test:  The two samples under investigation in the test are independent of each other and the observations within each sample are independent.  The observations are ordinal or continuous measurements (i.e., for any two observations, one can at least say, whether they are equal or, if not, which one is greater). Data types that can be analysed with Mann-Whitney U-test:  Data points should be independent from each other.  Data do not have to be normal and variances (SD) do not have to be equal.  All individuals must be selected at random from the population.  All individuals must have equal chance of being selected.  Sample sizes should be as equal as possible but some differences are allowed.

10. Nonparametric Methods: Mann-Whitney U test Calculations: There are two procedures of doing this Procedure # 1  Stage 1: Call one sample A and the other B.  Stage 2: Place all the values together in rank order (i.e. from lowest to highest). If there are two samples of the same value, the 'A' sample is placed first in the rank.  Stage 3: Inspect each 'B' sample in turn and count the number of 'A's which precede (come before) it. Add up the total to get a U value.  Stage 4: Repeat stage 3, but this time inspects each A in turn and count the number of B's which precede it. Add up the total to get a second U value.  Stage 5: Take the smaller of the two U values and look up the probability value in the next table. This gives the percentage probability that the difference between the two sets of data could have occurred by chance. Example: The results of the cytogenetic analysis of abnormal cells after exposure to the drug (Y) are shown below together with the concurrent control (X) data. Test to see if there is a significant difference between the treated and the control groups. Group (X) = 7; 3; 6; 2; 4; 3; 5; 5 Group (Y) = 3; 5; 6; 4; 6; 5; 7; 5

11. Nonparametric Methods: Mann-Whitney U test Solution  Stage 1: Sample A = 7; 3; 6; 2; 4; 3; 5; 5 Sample B = 3; 5; 6; 4; 6; 5; 7; 5  Stage 2:  Stage 3: U b = 3 + 4 + 6 + 6 + 6 + 7 + 7 + 8 = 47  Stage 4: U a = 0 + 0 + 0 + 1 + 2 + 2 + 5 + 7 = 17  Stage 5: U = 17 The critical value from the table = 6.5. The probability that the quality of the measures in group Y is better than group X just by chance is 6.5 per cent (p = 0.065) (see the next Table). If you find that there is a significant probability that the differences could have occurred by chance, this can mean: 1.Either the difference is not significant and there is little point in looking further for explanations of it, OR 2.Your sample is too small. If you had taken a larger sample, you might well find that the result of the test of significance changes: the difference between the two groups becomes more certain. 3. GroupAAABABAABBBABBAB 2. Labels2333445555566677 1. Rank12345678910111213141516

12. n¹n¹ 12345678 u 011.12.20.60.20.10.0 122.24.41.20.40.20.10.0 233.38.92.40.80.30.1 0.0 344.413.34.21.40.50.20.1 455.620.06.72.40.90.40.20.1 5 26.79.73.61.50.60.30.1 6 35.613.95.52.31.00.50.2 7 44.418.87.73.31.50.70.3 8 55.624.810.74.72.11.00.5 9 31.514.16.43.01.40.7 10 38.718.48.54.12.01.0 11 46.123.011.15.42.71.4 12 53.928.514.27.13.61.9 13 34.117.79.14.72.5 14 40.421.711.46.03.2 15 46.726.214.17.64.1 16 53.331.117.29.55.2 17 36.220.711.66.5 18 41.624.514.08.0 19 47.228.616.89.7

13. Example: The results of the cytogenetic analysis of abnormal cells Males (♂) and Females (♀) are shown below. Test to see if there is a significant difference between these two gender groups. Group (♀) = 9; 4; 6; 8; 6 (% of cells) Group (♂) = 19; 16; 9; 19; 8 (% of cells) U B = 24 U A = 1 P = 0.2 (0.02) Nonparametric Methods: Mann-Whitney U test

14. Procedure # 2  Choose the sample for which the ranks seem to be smaller (the only reason to do this is to make computation easier). Call this "sample 1," and call the other sample "sample 2."  Taking each observation in sample 1, count the number of observations in sample 2 that are smaller than it (count a half for any that are equal to it).  Calculate sum of ranks R1 and R2 then use the following formula U1 = m x n + m (m + 1)/2 – R1 U2 = m x n + n (n + 1)/2 – R2 U1 + U2 should be equal to m x n NB: If you have ties ( equal), a correction should, strictly, be made for ties. Rank them anyway, pretending they were slightly different. Find the average of the ranks for the identical values, and give them all that rank. Carry on as if all the whole-number ranks have been used up.

15. Example Data 14 2 5 4 2 14 18 14 Nonparametric Methods: Mann-Whitney U test

16. 2 4 5 14 18 Data 14 2 5 4 2 14 18 14 Nonparametric Methods: Mann-Whitney U test Example Sorted Data

17. Example Sorted Data 2 4 5 14 18 Data 14 2 5 4 2 14 18 14 Ties Nonparametric Methods: Mann-Whitney U test

18. Example 2 4 5 14 18 Data 14 2 5 4 2 14 18 14 Rank them anyway, pretending they were slightly different Nonparametric Methods: Mann-Whitney U test Sorted Data Ties

19. Example Rank A 2 4 5 14 18 1234567812345678 Data 14 2 5 4 2 14 18 14 Nonparametric Methods: Mann-Whitney U test Sorted Data

20. Example Rank A 2 4 5 14 18 1234567812345678 Data 14 2 5 4 2 14 18 14 Find the average of the ranks for the identical values, and give them all that rank Nonparametric Methods: Mann-Whitney U test Sorted Data

21. Example Rank A 2 4 5 14 18 1234567812345678 Data 14 2 5 4 2 14 18 14 Average = 1.5 Average = 6 Nonparametric Methods: Mann-Whitney U test Sorted Data

22. Example Rank A 2 4 5 14 18 1234567812345678 Data 14 2 5 4 2 14 18 141.5 3 4 6 8 Rank Nonparametric Methods: Mann-Whitney U test Sorted Data

23. Example Rank A 2 4 5 14 18 1234567812345678 Data 14 2 5 4 2 14 18 14 1.5 3 4 6 8 Rank These can now be used for the Mann-Whitney U test Nonparametric Methods: Mann-Whitney U test Sorted Data

24. Nonparametric Methods: Mann-Whitney U test Solution of our example Group (X) = 7; 3; 6; 2; 4; 3; 5; 5 Group (Y) = 3; 5; 6; 4; 6; 5; 7; 5 Sample 2Ranks (orders)Sample 1Ranks (orders) 3455566734555667 3 5.5 9 13 15.5 (highest order) 2334556723345567 1 (lowest order) 3 5.5 9 13 15.5 m=8R1 (sum of the ranks)n=8R2 (sum of the ranks) R1 = should be 77, R2 = should be 59, U1 = 23, U2 = 41 Look at the next Table at n 8 and m 8 you will find the tabulated value at p 0.05 (= 16) is less than the calculated one (23) so the difference is not significant and we failed to reject H o.

25. Nonparametric Methods: Mann-Whitney U test