PSY 1950 Nonparametric Statistics November 24, 2008

Skeptics at the Pub: Twins, Telepathy, and Smut Samuel T. Moulton November 24, 2008

“Most of these developments have this feature in common, that the distribution functions of the various stochastic variables which enter into their problems are assumed to be of known functional form, and the theories of estimation and of testing hypotheses are theories of estimation of and of testing hypotheses about, one or more parameters… the knowledge of which would completely determine the various distribution functions involved. We shall refer to this situation… as the parametric case, and denote the opposite case, where the functional forms of the distributions are unknown, as the non-parametric case.” –Jacob Wolfowitz (1942)

Parametric versus Nonparametric Think of the difference in terms of sampling distributions –In parametric statistics, sampling distributions are based on assumed, theoretical type of population distributions Most commonly assume normal population distribution –In nonparametric statistics, sampling distributions are NOT based on assumed, theoretical type of population distributions

Example: Parametric Approach Given sample of 99 scores, what values of 100th score would have p <.01? –Assume distribution –Estimate distribution –Determine sampling distribution –Calculate probability

Example: Nonparametric Approach Given sample of 99 scores, what values of 100th score would have p <.01? –Determine distribution from data –Determine sampling distribution –Calculate probability

Nonparametric Analogs Two independent samples –Parametric: independent samples t-test –Nonparametric: Mann-Whitney U-test Two dependent samples –Parametric: dependent samples t-test –Nonparametric: Wilcoxon signed-ranks test More than two independent samples –Parametric: independent samples ANOVA –Nonparametric: Kruskal-Wallis test More than two dependent samples –Parametric: dependent samples ANOVA –Nonparametric: Friedman test

Rank-ordering Ordinal (rank-ordered) measurement scale –Originally ordinal –Originally interval/ratio Variable is continuous

Why are Ordinal Tests Nonparametric? Rank ordering defines our null distribution (and sampling distribution) with minimal assumptions

Mann-Whitney U-test Logic: –If IV does not have a systematic effect, then ranks of two groups should be similar –If IV does have a systematic effect, then ranks of two groups should be different

To quantify similarity/difference: –Collapse across group and rank-order –Give each subject a “point” for outranking a subject from the other group –For each group*, add up the points –Test statistic is the lowest group score To calculate significance, compare obtained test statistic to distribution of all possible test statistics –Use U table or SPSS –When n > 20, distribution is approximately normal Mann-Whitney U-test

U male = = 8 U female = = 8 check n male  n female = U male + U female p (U ≤ 8) >.05

Wilcoxon Signed-Ranks Test Analogous to dependent- measures t-test Rank-order difference scores, group by positive/negative Logic: –If IV does not have a systematic effect, the positive/negative rankings should be similar –If IV does have a systematic effect, the positive/negative rankings should be different

To quantify similarity/difference: –Collapse across group and rank-order absolute difference scores –Sum positive and negative ranks –Test statistic is the lowest sum To calculate significance, compare obtained test statistic to distribution of all possible test statistics –Use U table or SPSS –When n > 20, distribution is approximately normal Wilcoxon Signed-Ranks Test

T pos = = 18 T neg = = 18 p (T ≤ 18) >.05 Note: If dif scores = 0, split between groups (if odd #, discard one)

Kruskal-Wallis Test Analogous to independent- measures ANOVA Collapse across group, rank order scores Logic: –If IV does not have a systematic effect, then ranks of groups should be similar –If IV does have a systematic effect, then ranks of groups should be different

Kruskal-Wallis Test H is distributed as chi-square with df = k - 1

Friedman Test Analogous to dependent- measures ANOVA Rank order conditions within subjects Logic: –If IV does not have a systematic effect, then ranks of conditions should be similar –If IV does have a systematic effect, then ranks of conditions should be different

Friedman Test distributed as chi-square with df = k - 1

Why and Why Not? Why? –Ordinal measurement scale –Small sample sizes –Outliers –Non-normality –Before computers: Computationally easier Why not? –Lose information –Under normality, lose power –Assume identically shaped distributions

Resampling Bootstrapping –Determine sampling distribution by repeated sampling (with replacement) of original sample –In other words, treat the sample as though it were the population Jacknifing –Determine sampling distribution by resampling with one measurement missing Permutation –Randomly reassign subjects to groups (independent measures) or scores to conditions (dependent measures) –e.g., Monte-Carlo