1. Experimental Design & Analysis
Nonparametric Methods
April 17, 2007
DOCTORAL SEMINAR, SPRING SEMESTER 2007

2. Nonparametric Tests Occasions for use?
The response variable (residual) cannot logically be assumed to be normally distributed, a key assumption of ANOVA models
Limited data
Counts and rank are relevant units instead of means

3. Wilcoxon Rank Sum Test
Nonparametric version of a paired samples t-test 
Example of corn yield as a function of weeding
Yield
153.1
156.0
158.6
165.0
166.7
172.2
176.4
176.9
Rank 1 2 3 4 5 6 7 8
Treatment Sum of Ranks
No weeds
Weeds

4. Wilcoxon Rank Sum Test
Calculate test statistic by calculating the mean μ and the standard deviation
proc univariate data = MYDATA;
var CORN;
run; μ
= n1(N+1)/2
= 4(8+1)/2
= 4*9/2
= 36/2
=18
= sqrt n1n2(N+1)/12
= sqrt (4)(4)(8+1)/12
= sqrt 144/12
= sqrt 12
= 3.464 σ

5. Wilcoxon Signed Rank Sum Test
Nonparametric version of a paired samples t-test
Study difference between two variables (Story 1 vs. Story 2)
Data step necessary to create the difference of the two scores for each subject
data MYDATA; set MYDATA;
diff = STORY1 – STORY2;
proc univariate data = MYDATA;
var diff; run;

6. Wilcoxon Mann-Whitney Test
Nonparametric version of independent samples t-test can be used when you do not assume that the dependent variable is a normally distributed interval variable
Assume that the dependent variable is ordinal
proc npar1way data = mydata wilcoxon;
class female;
var write;
run;

7. Kruskal Wallis Test
Used when you have one independent variable with two or more levels and an ordinal dependent variable
Nonparametric version of ANOVA
Generalized form of the Mann-Whitney test method, as it permits two or more groups
proc npar1way data = mydata;
class prog;
var write;
run;

8. Chi-Square Test
Used when you want to see if there is a relationship between two categorical variables
Chi-square test assumes that the expected value for each cell is 5 or higher
If this assumption is not met, use Fisher's exact test
In SAS, the chisq option is used on the tables statement to obtain test statistic and p-value
proc freq data = mydata;
tables school*gender / chisq;
run;

9. Fisher’s Exact Test
Used when you want to conduct a chi-square test, but one or more of your cells has an expected frequency of 5 or less
Fisher's exact test has no such assumption and can be used regardless of how small the expected frequency is
proc freq data = mydata;
tables school*race / fisher;
run;

10. Factorial Logistic Regression
Used when you have two or more categorical independent variables but a dichotomous dependent variable
The desc option on the proc logistic statement is necessary so that SAS models the odds of being female (i.e., female = 1).  The expb option on the model statement tells SAS to show the exponentiated coefficients (i.e., the odds ratios).
proc logistic data = mydata desc;
class prog schtyp;
model female = prog schtyp prog*schtyp / expb;
run;

11. Nonparametric Correlation
Spearman correlation is used when one or both of the variables are not assumed to be normally distributed and interval (but are assumed to be ordinal)
The values of the variables are converted in ranks and then correlated
The spearman option on the proc corr statement tells SAS to perform a Spearman rank correlation instead of a Pearson correlation
proc corr data = mydata spearman;
var read write;
run;

12. Nonparametric Tests: Advantages & Shortcomings

13. Common Nonparametric Tests
Normal theory based test
Corresponding nonparametric test
Purpose of test
t test for independent samples
Mann-Whitney U; Wilcoxon rank-sum
Compares two independent samples
Paired t test
Wilcoxon matched pairs signed-rank
Examines a set of differences
Pearson correlation coefficient
Spearman rank correlation coefficient
Assesses the linear association between two variables
One way analysis of variance (F test)
Kruskal-Wallis analysis of variance by ranks
Compares three or more groups
Two-way analysis of variance
Friedman two-way analysis of variance
Compares groups classified by two different factors

14. Why Use Nonparametric Tests?
When data are not normally distributed and the measurements at best contain rank-order information, computing the standard descriptive statistics (e.g., mean, standard deviation) is sometimes not the most informative way to summarize the data

15. Advantages of Nonparametrics
Nonparametric test make less stringent demands of the data (resistant to outliers, shape of distribution)
Nonparametric procedures can sometimes be used to get a quick answer with little calculation
Nonparametric methods provide an air of objectivity when there is no reliable (universally recognized) underlying scale for the original data

16. Why Not Use All the Time?
Parametric tests are often preferred because:
They are robust
They have greater power efficiency (greater power relative to the sample size)
They provide unique information (e.g., the interaction in a factorial design)
Parametric and nonparametric tests often address two different types of questions

17. Different Nonparametric Tests  Same Results?
Different nonparametric tests may yield different results
Advisable to run different nonparametric tests

18. Large Data Sets
Nonparametric methods are most appropriate when the sample sizes are small
When data set is large it often makes little sense to use nonparametric statistics at all
How large is large enough?

19. Small Data Sets
What happens when you use a nonparametric test with data from a normal distribution?
Greater incidence of Type II error
The nonparametric tests lack statistical power with small samples

20. Shortcomings
Non-parametric tests cannot give very significant results for very small samples as all the possible rank-sums are fairly likely
They do not, without the addition of extra assumptions, give confidence intervals for the means or medians of the underlying distributions
Assume that the data can be ordered – power of the test diminished if there are lots of ties

21. Standardization
Used in situations in which you need to adjust and rescale
observations to have a different mean and standard deviation
Example: Midterm test scores are to be rescaled to have a mean of 75 and a standard deviation of 10
data midterm;
input grade
datalines; ;
proc univariate data=midterm plot;
var grade;
run;

22. Moments N
Mean
Std Dev
Stem Leaf # Boxplot | | | |
| |
*--+--* | | | |
Multiply Stem.Leaf by 10**+1

23. proc standard data=midterm out=adjusted mean=75 std=10;
var grade;
run;
The new data set, ADJUSTED, has one variable, also called GRADE, which has a mean of 75 and a standard deviation of 10.
For example, the grade of 95 in the MIDTERM dataset becomes a grade of 97.2 in the ADJUSTED dataset.
0.86( ) +75=97.2

24. The STDIZE procedure standardizes one or more numeric variables in a SAS data set by subtracting a location measure and dividing by a scale measure. A variety of location and scale measures are provided, including estimates that are resistant to outliers and clustering. Some of the well-known standardization methods such as mean, median, std, range, Huber's estimate, Tukey's biweight estimate, and Andrew's wave estimate are available in the STDIZE procedure.
In addition, you can multiply each standardized value by a constant and add a constant. Thus, the final output value is
result = add + multiply ×[((original - location))/scale]
where
result = final output value
add = constant to add (ADD= option)
multiply = constant to multiply by (MULT= option)
original = original input value
location = location measure
scale = scale measure
PROC STDIZE can also find quantiles in one pass of the data, a capability that is especially useful for very large data sets. With such data sets, the UNIVARIATE procedure may have high or excessive memory or time requirements.

25. options ls=78 ps=200 nocenter nodate;
data midterm;
input grade
datalines; ;
run;
proc stdize data=midterm
out=adjusted method=std add=75 mult=10 ;
var grade;
proc print data=adjusted;
proc univariate data=adjusted;

26. One-Sample Tests of Location
What is the equivalent of the one-sample normal test or one-sample t test for the hypothesis that the true mean is equal to a specified value?
Sign Test
Wilcoxon Sign Rank Test
Where h is the (unknown) median of the population.

27. Wilcoxon Sign Rank Test
PROC UNIVARIATE in SAS automatically performs three tests of location but it does so by testing if the "typical" value is zero.
T- Test
Sign Test
Wilcoxon Sign Rank Test
data midterm;
input grade /*
If GRADE is typically 75,
then GRADE-75 should typically be zero. */
diff=grade-75;
label diff='Points above 75';
datalines; ;
run;
proc univariate data=midterm;
var diff;

28. Univariate Procedure
Variable=DIFF Points above 75
Moments
N Sum Wgts
Mean Sum
Std Dev Variance
Skewness Kurtosis
USS CSS
CV Std Mean
T:Mean= Pr>|T|
Num ^= Num >
M(Sign) Pr>=|M|
Sgn Rank Pr>=|S|
T-test (m=75)
Sign Test (h=75)
Wilcoxon Sign Rank Test (h=75)

29. Ranking
Many nonparametric procedures rely on the relative ordering, or ranking, of the observations.
Suppose a new Florida resident wants to see if the prices of houses in Gainesville are higher than the prices of homes in smaller towns near Gainesville. She collects a random sample of 10 prices for houses on the market in Gainesville and 10 prices of homes in other cities in Alachua County.

30. Does one location tends to have higher average prices than the other?
data homes;
input location $ price
datalines;
Gville Gville Gville 94500
Gville Gville 99900
Gville Gville Gville 78000
Gville Gville
County County County
County County
County County County 74500
County County ;
run;
Influential
observation
Does one location tends to have higher average prices than the other?
Does one location tends to have higher-ranked prices than the other?
More higher ranking homes in the county than expected at random.

31. How to get the ranks? proc rank data=homes out=rankdata ties=mean;
OBS LOCATION PRICE RANKCOST
1 Gville $74,
2 Gville $269,
3 Gville $94,
4 Gville $86,
5 Gville $99,
6 Gville $91,
7 Gville $70,
8 Gville $78,
9 Gville $289,
Gville $114,
County $32,
County $125,
County $105,
County $120,
County $139,
County $72,
County $85,
County $74,
County $199,
County $2,200,
How to get the ranks?
proc rank data=homes
out=rankdata
ties=mean;
var price;
ranks rankcost;
run;
proc print
data=rankdata;
format price dollar10.;

32. Another way to rank the data would be to create groups of least expensive, inexpensive, moderate, expensive, and very expensive price ranges. PROC RANK can do this with the GROUPS option.
OBS LOCATION PRICE PRICEGRP
1 Gville $74,
2 Gville $269,
3 Gville $94,
4 Gville $86,
5 Gville $99,
6 Gville $91,
7 Gville $72,
8 Gville $78,
9 Gville $289,
Gville $114,
County $32,
County $125,
County $105,
County $120,
County $139,
County $72,
County $85,
County $74,
County $199,
County $2,200,
proc rank data=homes
out=rankdata
groups=5;
var price;
ranks pricegrp;
run;
proc print
data=rankdata;
format price dollar10.;
Grouping starts at 0.

33. We use PROC RANK to calculate the normal scores.
PROC RANK can be used to produce a better normal probability plot than the one produced by PROC UNIVARIATE.
We use PROC RANK to calculate the normal scores.
If the data are indeed normally distributed, the Blomberg-calculated scores (BLOM option) should provide the best straight line.
Consider the 20 homes to be a random sample of all homes for sale in Alachua County, and we want to see if price or log(price) more closely follows a normal distribution.
data homes;
set homes;
logprice=log(price);
run;
proc rank data=homes out=rankdata normal=blom;
var price logprice;
ranks norm1 norm2;
proc plot data=rankdata;
plot price*norm1 logprice*norm2;

34. Plot of PRICE*NORM1. Legend: A = 1 obs, B = 2 obs, etc. |
| A
| A A
| A A AA AA A A A A A
0| A B B A
RANK FOR VARIABLE PRICE
Plot of LOGPRICE*NORM2. Legend: A = 1 obs, B = 2 obs, etc.

35. Not very normal looking!

36. LOGPRICE|
16+ |
| A
14+
| A A
| A A
| AA A A A
| B B A A A AA
| A
10+
RANK FOR VARIABLE LOGPRICE

37. Better, but still not very normal!

38. Comparing Two or More Groups
The nonparametric version of analysis of variance is based on ranks.
The Mann-Whitney test and the Wilcoxon rank sum test are equivalent nonparametric techniques to compare two groups, while the Kruskal-Wallis test is ordinarily used to compare three or more groups.
All of these are available in PROC NPAR1WAY (nonparametric 1-way analysis of variance) in SAS.

39. NPAR1WAY
PROC NPAR1WAY performs tests for location and scale differences based on the following scores of a response variable: Wilcoxon, median, Van der Waerden, Savage, Siegel-Tukey, Ansari-Bradley, Klotz, and Mood Scores. Additionally, PROC NPAR1WAY provides tests using the raw data as scores. When the data are classified into two samples, tests are based on simple linear rank statistics. When the data are classified into more than two samples, tests are based on one-way ANOVA statistics. Both asymptotic and exact p-values are available for these tests.
PROC NPAR1WAY also calculates the following empirical distribution function (EDF) statistics: the Kolmogorov- Smirnov statistic, the Cramer- von Mises statistic, and, when the data are classified into only two samples, the Kuiper statistic. These statistics test whether the distribution of a variable is the same across different groups

40. proc npar1way wilcoxon data=homes; class location; var price; run;
N P A R 1 W A Y P R O C E D U R E
Wilcoxon Scores (Rank Sums) for Variable PRICE
Classified by Variable LOCATION
Sum of Expected Std Dev Mean
LOCATION N Scores Under H0 Under H0 Score
Gville
Other
Average Scores Were Used for Ties
Wilcoxon 2-Sample Test (Normal Approximation)
(with Continuity Correction of .5)
S = Z = Prob > |Z| =
T-Test Approx. Significance =
Kruskal-Wallis Test (Chi-Square Approximation)
CHISQ = DF = Prob > CHISQ =

41. Other Rank Tests
When there are two factors with no interaction, as in a randomized complete block design, Friedman's chi-square test is a non-parametric test that can be used to examine treatment differences. Friedman's test is available in SAS under PROC FREQ, but it is fairly complicated to perform. PROC FREQ also offers versions of rank sum tests for ordinal data. See SAS documentation for details.

42. The LIFETEST procedure can be used with data that may be right- censored to compute nonparametric estimates of the survival distribution and to compute rank tests for association of the response variable with other variables. The survival estimates are computed within defined strata levels, and the rank tests are pooled over the strata and are therefore adjusted for strata differences.

43. Non-Parametric Correlation
The Pearson Product Moment correlation coefficient measures the strength of the tendency for two variables X and Y to follow a straight line.
Suppose we are more interested in measuring the tendency for X to increase or decrease with Y, without necessarily assuming a strictly linear relationship.
Spearman correlation coefficient - Pearson correlation between ranks of X and ranks of Y.
Kendall correlation coefficient - probability of observing Y2 > Y1 when X2 > X1.

44. proc plot data=students; plot grade*order; proc corr data=students
Twelve students take a test, and we want to see if the students who finished the test early had higher scores than those who finished later.
We don't know the exact time that each student spent on the test, but we do know the order in which the tests were turned in to be graded.
With only have rank data for the time variable, the Pearson linear correlation coefficient would not be appropriate. (The last person to turn in the test could have taken 30 minutes, an hour, or two hours, and the guess of the exact time would greatly influence the resulting Pearson correlation.)
Both the Spearman and Kendall correlation coefficients could legitimately be used.
data students;
input order grade
datalines; ;
run;
proc plot data=students;
plot grade*order;
proc corr data=students
spearman kendall;
var grade;
with order;

45. Plot of GRADE*ORDER. Legend: A = 1 obs, B = 2 obs, etc.
100+ |
| A |A
| A
80+
| A A A
| A
| A
| A
| A
A A
ORDER

47. Correlation Analysis
Spearman Correlation Coefficients / Prob > |R| under Ho: Rho=0
/ N = 12
GRADE
ORDER
0.5855
Kendall Tau b Correlation Coefficients
/ Prob > |R| under Ho: Rho=0 / N = 12
ORDER
0.5815