1. Lessons Learned From Recent Safety Meta-Analyses Mark Levenson, Ph.D. Quantitative Safety and Pharmacoepidemiology Group Office of Biostatistics Center for Drug Evaluation and Research, FDA v. 1 Oct. 2009

2. Disclaimer The views expressed in this presentation represent the opinions of the author, and do not necessarily represent the views of the United States Food and Drug Administration

3. Focus Today Safety Regulatory setting Pre- and post-market Clinical trials Access to relevant data Not: locating, accessing quality of, and extracting data from studies

4. Outline 1.Study inclusion criteria 2.Endpoints 3.Methodology 4.Examples –Suicidality meta-analyses –Aprotinin

5. Meta-Analysis Steps 1.Define research goals 2.Research/understand relevant trials 3.Define analysis plan (Prespecify!) –Research questions –Study inclusion criteria –Analysis set and subgroups –Endpoints –Primary methods –Sensitivity methods 4.Make request to sponsor(s) / obtain data 5.Implement analysis plan 6.Report and interpret findings

6. Study Inclusion Criteria

7. Valid comparison groups Similarity in –Design –Interventions –Study population –Studied indication Data availability

8. 8

9. 9

10. Duration: Example Assume event of interest takes some time to develop (increasing hazard) Survival Years 12

11. Duration: Example (Cont.) Scenario 1: –10 trials –100 patients per trial –3 month duration 1,000 patients 250 person-years Scenario 2: –1 trial –100 patients per trial –2 year duration 100 patients 200 person-years

12. Endpoints

13. First choice: prospectively collect and adjudicate endpoints Second choice: use common post-hoc adjudication procedure across trials Last choice: make do with existing information from trials

14. Endpoints: Follow-Up Time Randomization Event End of Treatment End of Follow-up Time

15. Endpoints: Follow-Up Time Follow-up should be long enough to capture event of interest Use common cut-off point across trials when possible Need to balance lasting effect of drug versus confounding with post-trial therapy and dilution of drug effect (see: NEJM Vioxx APPROVE discussion, 2006)

16. Data Availability Patient-level data allows more thorough analysis Time-to-event Subgroup Treatment duration effects Internal validation

17. Methodology

18. Need to use appropriate methods for problem Need to justify method Need to perform sensitivity analyses along several fronts

19. Methodology Considerations Number of trials Number of subjects per trial Rates of events Zero-event trials Heterogeneity of effect

20. Methods Inverse variance weighting Mantel-Haenszel odds ratio or risk ratio Exact method for odds ratio Mantel-Haenszel risk difference Bayesian methods –Encompass fixed- and random-effect models and hierarchical models

21. Sensitivity Analysis Consequences of low event rate Consequences of zero-event trials Consequences of heterogeneity of trials –Random effects models –Trials with large influence

22. Sensitivity Strategy Primary method: Exact method for OR Sensitivity methods: –Mantel-Haenszel RD –GLMM, qualitatively compare results with primary method

23. Suicidality Meta-Analyses

24. Concern that drugs may be associated with suicidality Requested all patient-level data from all placebo-controlled trials from sponsors Patients retrospectively classified into suicidality outcomes by blinded experts

25. 25

26. 26 * * Reanalysis of FDA/Hammad 2004 data

27. 27 Suicidal Behavior or Ideation Odds Ratio Estimates

28. Antiepileptic AC Paraphrase Does committee agree with agency that findings should apply to all 11 drugs? Yes: 18, No: 3, Abstain: 0 Does committee agree with agency that findings should apply to all approved antiepileptics? Yes: 15, No: 5, Abstain: 1

29. Aprotinin

30. The Aprotinin Story Aprotinin: used to reduce blood loss and transfusion in patients undergoing coronary artery bypass graft surgery (CABG) with cardiopulmonary bypass 2006 NEJM Mangano paper raised safety concerns FDA held 2 Advisory Committee meetings on the safety of aprotinin motivated by 3 observational studies

31. Disparate Findings Mangano in-hospital death: no effect relative to no drug Mangano 5-year death: 1.37 HR, p- value=0.008 relative to no drug Sponsor Global CABG RCT Database: Death 2.9% aprotinin, 2.5% placebo (9/06 AC) Meta-analyses Henry et al. Cochran Review 2007: RR=0.90 (0.67, 1.20) relative to control (no drug). No effects relative to other drugs.

32. BART Blood Conservation Using Antifibrinolytics in a Randomized Trial (BART) Compared aprotinin to two active drugs 30-day death secondary endpoint

33. Jan. 2007 Second Interim Analysis Aprotinin 5.0% vs. 3.9% and 4.3% for comparator drugs DSMB: “did not consider the results of [Mangano Study] convincing.” DSMB: [Systematic reviews] “less biased than the observational studies” Four systematic reviews showed no death effect

34. October 2007 Aprotinin 6.5% vs. 4.2% and 4.3% for comparator drugs (p-value near nominal significance) DSBM recommends terminating trial

35. What Happened to the Meta-Analysis? Ray Editorial (NEJM) Trials not designed to collect follow-up mortality information Limited data on head-to-head comparisons with other drugs Trial and patient heterogeneity (surgical procedure, patient risk) may hide signal

36. Conclusions Prospectively plan trials for pooled analyses, e.g. endpoint definition and ascertainment Prespecify analysis plan Select trials with similar and appropriate designs Consider methodology issues of sparse events and perform sensitivity analyses Understand the limitations of meta-analysis Draft FDA Guidance for safety meta-analyses December 2009

37. References Guidance for Industry Premarketing Risk Assessment (FDA, 2005) Much ado about nothing: a comparison of the performance of meta-analytic methods with rare events (Statis. Med., Bradburn et al., 2007) The Aprotinin Story – Is BART the final chapter? (NEJM, Ray, 2007) Time-to-Event Analysis for Long-Term Treatments – The APPROVe trial (NEJM, Lagakos, 2006) Understanding the New Drug Safety Standards: The Emerging Science of Meta-Analysis (The Pink Sheet, 2007)