1. Guide to Handling Missing Information Contacting researchers Algebraic recalculations, conversions and approximations Imputation method (substituting missing data)

2. Imputation Method - When recalculations not possible -e.g. no standard deviation for a study -Use available data from other studies or other meta-analysis a.Within study imputation b. Multiple imputations Imputation Method

3. Within-study imputation = Standard deviation (SD) for missing data from study j =Mean from study with missing SD =Summation of all known SD from different studies =Summation of means from different studies other than j Method 1. (Means) SD j ~ XjXj _ Ʃ i k SD i (Ʃ i k X i ) _ SD j = X j Ʃ i k SD i ______ _ Ʃ i k X i _ ~

4. Assumptions Assumes SD to mean ratio is at the same scale for all studies - Experimental scales can differ tremendously between different taxonomic groups or experimental designs SD j = X j Ʃ i k SD i ______ _ Ʃ i k X i ~ -

5. -Regression techniques -Reports sample size but missing information to calculate pooled SD (required for Hedge’s d). α = Intercept β = slope of the linear regression of n vs s n j = observed sample size of the study with missing data Method 2. (sample size) s j= α+β(n j ) ~

6. Assumptions Assumes n (observed sample size of the study with missing data) is a good predictor s. s j= α+β(n j ) ~ K= number of studies with complete information on s and n (sample size of individual study) Method 3. No. of studies s j = Ʃ i k s j √n i _____ K √n j ~

7. Method 4. Follman et al. (1992) Furukawa et al. (2006) s j = √Ʃ i k [(n i -1)Ϭ 2 i ] __________ √Ʃ i k (n i -1) Ϭ 2= variance n= sample size of individual study ~

8. Assumptions Some degree of homogeneity among the observed SD and X across studies Assume information is missing at random and not due to reporting biases (non-random) -Imputations retain their original units. -Large variations among estimates will bias imputations. _

9. Multiple imputations Use random sampling approach Average repeated sampling for missing data Overall imputed synthesis

10. Advantage of multiple imputations Variability is explicitly modeled therefore do no treat imputed value as true observation e.g. Does not account for error associated with α or β. s j= α+β(n j ) ~

11. Methods: Multiple imputations Various methods: use maximum likelihood or Bayesian models. Requires specialized software e.g. Hot Deck- To calculate pooled s but several SD values missing -Random sample of s drawn with replacement possible s -Process repeated with replacement from possible s -Repeat till we get “m” number of complete data sets

12. Methods: Hot deck calculate effect size= δ for each(m) data Calculate variance = Ϭ 2 (δ l ) set δ = Ʃ l m = 1 δ l _ _ _. ___ m Variance= Ϭ 2 (δ)= Ʃ l m = 1 Ϭ 2 (δ l ) + (1+1) Ʃ l m = 1 (δ l – δ) 2 m _________ m _ m-1. _ _. Pooled effect size Rubin and Schenker (1991) If 30% data missing->m= 3 If 50% data missing->m= 5

13. Non-parametric analyses and bootstrapping Alternative to Hedge’s d Using weighting scheme Does not require SD E.g log response ratio lnR= ln X T X C If sample size available but no SD Ϭ 2 =(lnR)= n T n C n T +n C ___ _ _ T= treatment C= control ___ Inverse of a simplified estimate of variance

14. Effects of Imputation No standardized method for imputation-> bias Rubin and Schenker (1991) e.g. Appropriateness of imputed data can be evaluated using a sensitivity analysis Benefits despite potential bias -Improved variance estimate (i.e. smaller CI) over exclusion -May potentially improve representation of null studies