1. Lecture 15 - Hypothesis Testing A. Competing a priori hypotheses - Paired-Sites Tests Null Hypothesis : There is no difference in support for one tree over the other. Test statistic: In Parsimony (or minimum evolution);  = Length A – Length B In Likelihood;  = lnL A – lnL B

2. Early methods looked at the difference in optimality between the two trees on a site- by-site basis. Thus, these are called paired-sites method. So we would accept the H o that there is no difference in support for either tree. KH-Test: Parsimony – normal distribution Tree # 1 2 Length 354 357 [So the test statistic is 3 steps] Kishino-Hasegawa test: Tree Length diff s.d.(diff) t P* --------------------------------------------------------------------- 1 354 (best) 2 357 3 9.65019 0.3109 0.7560 * Probability of getting a more extreme T-value under the null hypothesis of no difference between the two trees (two-tailed test). Example Normal (mean = 0)

3. KH-Test: Likelihood – normal distribution Tree 1 2 -ln L 2558.60898 2529.81009 [So the test statistic is 28.8 lnL units] Kishino-Hasegawa test: KH test using normal approximation, two-tailed test KH-test Tree -ln L Diff -ln L P --------------------------------------------- 1 2558.60898 28.79889 0.000* 2 2529.81009 (best) * P < 0.05 Here, the same data and test lead to rejection of H o under a different optimality criterion Same data, but using ML as criterion. KH-Test: Likelihood – RELL bootstrapping Tree 1 2 -ln L 2558.60898 2529.81009 Kishino-Hasegawa test: KH test using RELL bootstrap, two-tailed test Number of bootstrap replicates = 1000 KH-test Tree -ln L Diff -ln L P --------------------------------------------- 1 2558.60898 28.79889 0.396 2 2529.81009 (best)

4. Same Data, Same Two Trees, Different Inferences

5. Most applications of these tests have involved comparing a suboptimal tree to the best tree. This results in use of a mismatched null distribution and the tests should not be used in this way. SH Test - corrects for the a priori requirement. Uses a collection of trees (rather than just ML v. Hypothesis). Uses RELL bootstrap to generate an average lnL for the collection. Uses this average to center the null distribution. The test is very conservative, though, and very sensitive to the collection of trees used in centering the null distribution. If there are only two trees in the collection, the ML tree and the tree we’re interested in testing, the SH test is identical to the KH test. Shimodaira has applied multiscale bootstrapping to correct for this bias in the AU (almost unbiased) test

6. Parametric Bootstrap Tests Generate the null distribution via simulation under the hypothesis being tested. Assuming some hypothesis is true, what is the probability of an observed test statistic? Ancient Fragmentation Hypothesis Northern Dispersal Hypothesis N. Rockies N. Cascades S. Cascades Southern Dispersal Hypothesis N. RockiesS. Cascades N. Cascades PNW Biogeography

7. Parametric Bootstrap Tests This tree has a likelihood score of lnL = -1593.01499

8. Parametric Bootstrap Tests Can we reject an ancient fragmentation? Force the tree to fit the predictions of our hypothesis, we get a much worse tree: lnL = -1612.50229  = lnL (uconstrained) – lnL (constrained) = - 1593.01499 – ( - 1612.50229) = 19.487 lnL units. If the true tee actually were the constrained tree (i.e., if the hypothesis being examined were actually true), what is the probability that we would see a tree that is 19.487 lnL units better, simply due to stochasticity? So we simulate many replicate data sets under the hypothesis. We first run an unconstrained search to find the ML tree for the replicate. We then run a search constrained to find the best tree consistent with the hypothesis for each replicate.

9. Parametric Bootstrap Tests Can we reject an ancient fragmentation? We can reject a southern dispersal hypothesis as well and are left with the northern dispersal hypothesis that the tree suggests.

10. Bayesian Hypothesis Testing If we run a typical MCMC, we have sample of topologies that represent the posterior distribution of trees. The proportion of the trees in the distribution that are consistent with the topological predictions of each hypothesis provide the posterior probability of that hypothesis. Remember the difference in Bayesian and frequentist perspectives, and the different interpretations of p-values this causes.