1. Selecting Among 2-Sample Tests
Selection among tests is often based on the underlying distributions of the two populations. We often consider the ones below. Here x is a vector of values of the variable…
uniform (density is dunif(x, min, max) where min and max are the endpoints of the uniform density.
normal (density is dnorm(mean,sd)
exponential (density is dexp(rate), where rate is the parameter of the exponential (so 1/rate = mean of the exponential distribution)
Cauchy (density is dcauchy(location,scale) where location and scale are its parameters
Laplace (density is (1/2b)(exp(-abs(x-a)/b) where inf<x<+inf, and b>0 is the scale parameter and a is the location parameter)
Write R code that will plot all these densities on the same axis for comparison, especially comparison of the tails of the distributions. “Fat tailed” underlying densities often call for nonparametric methods …

2. Assume Xi from treatment 1 and Yj from treatment 2 and that they have cdf's given by:
Also assume the cdf's are continuous (no ties). The hypothesis tested is
If the cdfs are normal with unknown but equal variances, then the t-test is the best. Departure from normality has little effect on Type I error, because of the Central Limit Theorem, but there are problems with power if they are non-normal.
See Table for a comparison of the Wilcoxon rank-sum test versus the t-test. Note that for small samples and "light-talied" distributions (those without much chance of outliers), the t-test is probably better. Otherwise, use Wilcoxon
Another way of comparing these two tests is via relative efficiency (and ARE, asymptotic relative efficiency). See the definition on p. 62: basically, test 1 is more efficient than test 2 if it requires a smaller sample size to achieve the same power.

3. The ARE of the Wilcoxon test versus the t-test is >=
The ARE of the Wilcoxon test versus the t-test is >= but can be arbitrarily large (see Table 2.9.2). Even for normal cdfs, the ARE of Wilcoxon vs. t-test is .955
The permutation test vs. the Wilcoxon test and the t-test is discussed in There are a couple things to note here:
little is gained by doing more than 1600 randomly sampled permutations
see Table for comparison with t-test
see Table for comparison with Wilcoxon test