Choosing Between Diversity Indices James A. Danoff-Burg Dept. Ecol., Evol., & Envir. Biol. Columbia University

Alpha Diversity Indices A diversity of diversities Log Alpha Log-Normal Lambda Q-Statistic Simpson McIntosh Berger-Parker Shannon-Wiener Brillouin How to choose between these? Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Appropriateness Index assumptions need to be met Abundance model of data Sensitivity to sample size Each index needs to be considered for all of these aspects determines whether can be used for your data Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Assumptions Alpha diversity indices do not make many assumptions No assumptions made about species abundance distributions Cause of distribution  not needed species abundance models have assumptions about these Geometric – niche pre-emption, regular arrivals Log – niche pre-emption, irregular arrival intervals Log-Normal – successively apportioning available niche space of all resources in proportion to abundance Broken Stick – simultaneously apportioning available niche space of one resource in proportion to abundance Shape of curve “Non-parametric” Normality is not needed Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Abundance Model Some indices perform better under a specific abundance model Example: Simpson – probability that two individuals are of the same species Geometric Underestimate Simpson diversity value Log Log-Normal Best for Simpson diversity analysis analysis Broken Stick May overestimate Simpson diversity value Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Abundance Model Fitting Main problem: Often have multiple models that fit the data Occasionally because of low number of abundance classes Log2 has only 11 classes (octaves) even possible Most data have less than 11 E.g., less than 256 individuals in a species Resulting in only 8 classes Fewer classes, mean fewer opportunities for departures from fit Small data sets  fit many models Few spp in each abundance class  decreased discriminability Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Abundance Model Fitting Secondary problem: Log-Normal is a frequent consequence Often because of the central limit tendency of large data sets If a data set has many species  often log-normal distribution results Does not necessarily mean that the community has assembled by a successive breaking of the available nice space As is the assumption with the log-normal distribution Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Appropriateness – Sample Effort Some indices are tremendously sensitive to sample size Low replication  skewed values Idiosyncratic results Not truly representative of the environment Indices sensitive to inadequate sampling S = very sensitive to sampling effort Dominance indices (Simpson, Berger-Parker, McIntosh) Information statistics indices (Shannon) Evenness indices Indices insensitive to sampling effort Always: Log series a, 1/d (influenced by abd of most abd sp) If more than 50% of spp represented: Q Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Appropriateness - Sampling Effort How to determine when you have completely sampled the environment? Assuming prior information Leveling off of S with adding more samples If interest is largely richness Leveling off of Pielou’s t point If interest is proportional abundance Leveling off point = adequate sample size Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Appropriateness, Sample Size - When is Enough Enough? Leveling off of S with adding more samples If interest is largely richness S is more sensitive to sample size than diversity indices Need more samples Leveling off of Pielou’s t point If interest is proportional abundance Any diversity index can be used 50 S Pielou’s t 40 Pielou’s t Diversity Index Value 30 H’ Shannon Pielou’s t 20 1/D Simpson Pielou’s t 1/d Berger-Parker 10 10 20 30 40 Sample Addition Sequence Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Sampling Effort Need consistency in sampling effort Need to use the same effort throughout experiment Helps to ensure comparability of indices All would then be equal(ly biased) When sample sizes are unequal? Rarefy the larger sample to the smaller More next week on rarefaction (WE 1) Better to have many small samples than few large samples Increases replication Decreases thoroughness of each replicate Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Discriminant Ability of the Index Differences are usually very subtle Need analytical rigor to differentiate Assuming differences exist, how best to see them? No two sites are identical in terms of S, N, relative abundance All sites will differ, how can we detect this? Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Studies of Discriminability I Taylor (1978) Using 8 indices on moths, 9 sites, over 4 years Rothamsted Insect Survey, England Best: Log a (by far) Next: H’, S, log-normal l, 1/D, log biomass Useless: log-normal S* and s Kempton (1979) Using 4 indices on same data, 14 sites, 7 years Best: S, H’ Useless: 1/D, 1/d Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Studies of Discriminability II Kempton & Taylor (1976) Transformed indices > untransformed form exp H’ > H’ 1 / D > D Kempton & Wedderburn (1978) a and Q > any H’ and any D Magurran (1981) Best: Margalef (Dmg = (S-1) / ln N); U, S Less well: HB > H’, exp H’ Worst: 1/d, D or 1/D, McIntosh D, H’ E, HB E Morris & Lakhani (1979) H’ > D or 1/D Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Diversity Aspects Measured Not just discriminability, but what is the index measuring? Richness, Dominance, Evenness, Abundance… What most affects the index? Rare species or species richness? Type 1 Measures log a, log-normal l, Q, S, H’, HB, Dmg, McU Abundance of the most common species or dominance? Type 2 Measures 1/D or D, 1/d, McD, H’E, HBE Within each type  significant correlation Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Statistical Comparability Historically, statistical comparisons not made Mostly descriptive comparisons in past Statistical Options H’, Var H’, t-test Replication Most sets of replicated estimates are normally distributed Non-normal data can be transformed for normality Jackknife data Provides standard error and confidence limits as well Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Widespread Utility of the Index Important for ensuring comparability Between studies Between sites Between researchers Most commonly used S, H’, 1/D, log a Less commonly used Log-Normal l, Q Even though highly valuable as discussed above Dmg, McU, McE, HB, 1/d Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Widespread Utility - Cautions Be careful with the H’ Shannon Heavily criticized, despite widespread use “no direct biological interpretation” (Goodman 1975) Be careful with the log a Based only on S & N Insensitive to changes when both stay constant Uncommon situation “there can be no universal best buy but there are rich opportunities for inappropriate usages” (Southwood 1978) Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Indices Performance Summary Index Discriminant Ability Sample Size Sensititivy Richness, Evenness, Dominance Calculation Widely used? Sensitivity to Abd models Log a Good Low Richness Simple Yes N (?) Log Normal l Moderate Complex N Q S High Margalef Shannon Intermediate Brillouin McIntosh U Simpson Dominance Berger-Parker Poor Shannon E Evenness Brillouin E McIntosh D = Desirable Traits in an Index Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Index Choice Guidelines Clearly formulate question you are studying Ensure equal sample sizes Draw a Rank Abundance graph Calculate Margalef (Richness) and Berger-Parker (Dominance) indices Determine Log a and Q Test fit to abundance models Use ANOVA to test for treatment differences Use Jackknife to improve estimate of indexes Be consistent in choice of index across your studies Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Bases for Choice Appropriateness of each index for your data Discriminant ability of the index Statistical Comparability Widespread utility of the index Your Question Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Your Question What you want to know determines how you analyze your data How important is each aspect of diversity? Richness? Evenness? Dominance? Abundance? Per-species (relative) abundance? Taxon diversity? Trophic structuring? Guild diversity? Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Your Question What answers your question? What are the most important aspects of diversity? What data directly addresses your question? How should it be presented? How should it be emphasized? Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Your Question Work through the gardens example Abundance model? Diversity aspects? Form of the data? Appropriate analyses? Answer for a few subcomponent questions Assignment: Do above using your thesis (or the gardens data) 3-5 pages Due 2nd April before class Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Studies of Discriminability III – Gotelli & Colwell 2001 Purpose / Goal To discuss the different ways of presenting richness To discuss the ways in which we can approximate total species richness To discuss the difficulties encountered when using richness Proportional abundances Species Density Standardizing number of species across different sized areas Species / Genus Important in biogeography Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Gotelli & Colwell 2001 Differentiates between Individual-based and Sample-based assessment methods Individual: life lists, Christmas bird counts, collector’s curves Sample: replicated quadrats, mist nets, trap data Hybrid: m-species lists (observing to a point) Differentiates between accumulation and rarefaction curves (either individual or sample based) Accumulation – total # of spp during process of data collection Rarefaction – repeatedly subsampling without replacement from the data (progressive data reduction) Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Sample Addition Sequence Gotelli & Colwell 2001 Richness Samples always below Individuals Individuals are clumped Random assortments of traps through time unclumped Rarefaction smoothed curves Replicated data removal process (like Pielou) Built “right to left” Accumulation stepped line Observed fact Built “left to right” 50 Individuals: Rarefaction Individuals: Accumulation 40 30 Samples: Rarefaction Samples: Accumulation 20 10 10 20 30 40 Sample Addition Sequence Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Gotelli & Colwell 2001 Factors influencing richness estimates Underlying species richness Relative abundance distributions Sampling effort Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Gotelli & Colwell 2001 Cautions when using richness Scaling different sized areas to species density Susceptible to non-linearity of increasing area and richness Leads to an inability to extrapolate from smaller to larger areas Species per comparable unit area Problem with non-linear relationship between sampling efforts and richness Species per genus Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Gotelli & Colwell 2001 Solution? Use rarefaction on your data to standardize sampling effort Bring larger sample down to size of smallest one Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

© 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu

Hypothetical Model Curves 100 Geometric Series 10 Log-Normal Series Broken Stick Model Log Series 1 Per Species Abundance 0.1 0.01 0.001 10 20 30 40 Species Addition Sequence Lecture 5 – Choosing Between Diversity Indices © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu