1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September 2007

2 Motivation Missing outcome data compromise trials So they also compromise meta-analyses We may want to –correct for bias due to missing data –down-weight trials with more missing data NB missing data within trials, not missing trials

3 Plan Meta-analysis of binary data Haloperidol example Standard approaches to missing data Imputation methods IMORs Methods that allow for uncertainty Demonstration

4 Haloperidol meta-analysis HaloperidolPlacebo % missing r1f1m1n1r2f2m2n2 Arvanitis % Beasley % Bechelli % Borison % Chouinard % Durost % Garry % Howard % Marder % Nishikawa % Nishikawa % Reschke % Selman % Serafetinides % Simpson % Spencer % Vichaiya % r=successes f=failures m=missing n=total

5 Standard approaches to missing data Available cases (complete cases): ignore the missing data –assumes MAR: missingness is independent of outcome given arm Assume missing=failure –implausible, but not too bad for health-related behaviours Neither assumption is likely to be correct

6 Other ideas Sensitivity analyses, e.g. do both missing=failure and available cases –but these could agree by chance Explore best / worst cases Use reasons for missingness Explicit assumptions about informative missingness ( IM) –IM: missingness is dependent on outcome

7 metamiss.ado Processes data on successes, failures and missing by arm & feeds results to metan Available cases analysis (ACA) Imputed case analyses (ICA): –impute as failure: ICA-0 –impute as success: ICA-1 –best-case: ICA-b (missing=success in E, failure in C) –worst-case: ICA-w –impute with same probability as in control arm: ICA-pC –impute with same probability as in experimental arm: ICA-pE –impute with same probability as in own arm: ICA-p (agrees with ACA) –impute using IMORs: ICA-IMOR (see next slide)

8 More general imputation: IMORs Measure Informative Missingness using the Informative Missing Odds Ratio (IMOR): –Odds ratio between outcome and missingness Can’t estimate IMOR from the data, but given any value of IMOR, we can analyse the data Generalises other ideas: e.g. –ICA-0 uses IMORs 0, 0 –ICA-1 uses IMORs ,  –ICA-b uses IMORs , 0 –ICA-p uses IMORs 1, 1 –ICA-pC uses IMORs OR, 1 where OR is odds ratio between arm and outcome in available cases

9 Getting standard errors (weighting) right Weight 1: treat imputed data as real Weight 2: use standard errors from ACA Weight 3: scale imputed data to same sample size as available cases Weight 4: algebraic standard errors –same as weight 1 for ICA-0, ICA-1, ICA-b, ICA-w –same as weight 2 for ICA-p –uses Taylor expansion for ICA-IMOR –for ICA-pC & ICA-pE, we condition on the IMOR (I can explain…)

10

11

12

13

14

15

16

17

18

19

20

21 Allowing for reasons (ICA-R) Specify number of missing individuals in each arm to be imputed by each scheme ICA-0, ICA-1, ICA-pC, ICA-pE, ICA-p, ICA-IMOR. Can take these data from a different outcome: metamiss scales to #missing If missing in a particular study, metamiss imputes using combined studies

22

23 Allowing for uncertainty So far we have pretended we really know the IMORs This is never really correct Now we allow them to be unknown but from a user-specified distribution

24 Bayesian approach allowing for uncertain IMORs (Rubin, 1977)

25 Bayesian analysis Elicit prior for  E,  C or use N(0,1 2 ) or N(0,2 2 ) Get posterior distribution by integrating over the 2- dimensional distribution of  E,  C. metamiss does this fast & accurately by: 1.Standard normal approximation to posterior given  E,  C 2.Integrate using Gauss-Hermite quadrature. Alternatives: –Taylor expansion (inaccurate for large SD of log IMOR) –Full Bayesian Monte Carlo (slow, little gain in accuracy)

26 Understanding priors for log IMOR: implied prior for P(success | missing) when P(success | observed) = 1/2

27

28

29

30 Proposal: 4 sensitivity analyses IMORsOptions (e.g.) Sensitive to:Works via: fixed equal imor(2 2) Imbalance in missingness Point estimates fixed opposite imor(2 1/2) Amount of missing data random equal sdlogimor(2) corr(1) Imbalance in missingness Weightings random uncorrelated sdlogimor(2) corr(0) Amount of missing data

31 Summary Tool for sensitivity analysis Requires thought about plausible missing data mechanisms Would be nice to overlay sensitivity analysis with ACA Further work includes combining uncertainty with reasons I also have a program mvmeta for multivariate meta-analysis

32 References 1 st part: Higgins JPT, White IR, Wood A. Imputation methods for missing outcome data in meta-analysis of clinical trials. Clinical Trials, submitted. 2 nd part: White IR, Higgins JPT, Wood AM. Allowing for uncertainty due to missing data in meta-analysis. 1. Two-stage methods. Statistics in Medicine, in press. Related: White IR, Welton NJ, Wood AM, Ades AE, Higgins JPT. Allowing for uncertainty due to missing data in meta- analysis. 2. Hierarchical models. Statistics in Medicine, in press. metamiss.ado available from

33 Extra slides

34