Case Studies of the Use of Meta- Analyses of Drug Safety Questions in the Regulatory Setting Jesse A. Berlin, ScD Johnson & Johnson Pharmaceutical Research and Development New Jersey Chapter of the ASA 10 October 2008 Jesse A. Berlin, ScD Johnson & Johnson Pharmaceutical Research and Development New Jersey Chapter of the ASA 10 October 2008

2 2 Outline Case studies of the use of historical randomized trial data to address potential safety concerns Emphasis on exploration of patient-level characteristics as potential effect modifiers –Some methodologic “heads up” Case studies of the use of historical randomized trial data to address potential safety concerns Emphasis on exploration of patient-level characteristics as potential effect modifiers –Some methodologic “heads up”

3 3 Is it sampling variability? Problem: How do we distinguish sampling variability from “real” variability (possibly) associated with different effects of treatment in different subgroups of patients (or with different dosing algorithms or other specific aspects of treatment)?

4 4 Example 1: Galantamine Acetylcholinesterase inhibitors (AchEIs) are used as a standard treatment for Alzheimer’s Disease (AD) Galantamine, an AChEI, has been extensively studied in patients with mild to moderate AD Galantamine has also been studied in patients with AD with concomitant cerebrovascular disease (CVD) and in patients with VaD (16). The benefit is to slow the progress of cognitive decline (relative to placebo) Acetylcholinesterase inhibitors (AchEIs) are used as a standard treatment for Alzheimer’s Disease (AD) Galantamine, an AChEI, has been extensively studied in patients with mild to moderate AD Galantamine has also been studied in patients with AD with concomitant cerebrovascular disease (CVD) and in patients with VaD (16). The benefit is to slow the progress of cognitive decline (relative to placebo)

5 5 Safety “signal” for Galantamine in Mild Cognitive Impairment Two 2-year randomized controlled trials –Individuals with mild cognitive impairment –Findings replicated in both studies –13 deaths versus 1 death Higher mortality observed in galantamine-treated patients, compared with placebo –Overall mortality rates were low in both groups The findings prompted a reevaluation in patients with dementia Two 2-year randomized controlled trials –Individuals with mild cognitive impairment –Findings replicated in both studies –13 deaths versus 1 death Higher mortality observed in galantamine-treated patients, compared with placebo –Overall mortality rates were low in both groups The findings prompted a reevaluation in patients with dementia

6 6 Galantamine Methods All galantamine trials (J&J or Shire-sponsored) for which J&J could access data Also searched MEDLINE and the Cochrane Controlled Trials Register (2005) Issue 4 Trials included were independently reviewed, verified by two readers, and met the following criteria: –a) randomized –b) placebo-controlled –c) parallel group –d) blinded –e) at least one treatment arm with galantamine All galantamine trials (J&J or Shire-sponsored) for which J&J could access data Also searched MEDLINE and the Cochrane Controlled Trials Register (2005) Issue 4 Trials included were independently reviewed, verified by two readers, and met the following criteria: –a) randomized –b) placebo-controlled –c) parallel group –d) blinded –e) at least one treatment arm with galantamine

7 7 Meta-analysis of survival in galantamine randomized trials (6 months duration)

8 8 Other Galantamine Analyses Nested case-control study of deaths was used to investigate potential mechanism for the mortality increase –Baseline ECG findings –Comorbidities –Concomitant medications Findings were inconclusive due to small sample size Mortality analyses in press (Feldman et al.; Acta Neurologica Scandinavica) We are doing a large, placebo-controlled study with mortality as the primary endpoint Nested case-control study of deaths was used to investigate potential mechanism for the mortality increase –Baseline ECG findings –Comorbidities –Concomitant medications Findings were inconclusive due to small sample size Mortality analyses in press (Feldman et al.; Acta Neurologica Scandinavica) We are doing a large, placebo-controlled study with mortality as the primary endpoint

9 9 Example: Erythropoietin-stimulating agents in oncology Is meta-analysis the appropriate tool?

10 “Houston, we have a problem...” A randomized trial of the use of erythropoietin in breast cancer patients found an increased risk of mortality in the EPO arm, relative to the placebo arm. (More on this later.) This led to extensive discussions with FDA and, ultimately, a presentation in May of 2004 to the Oncologic Drugs Advisory Committee (ODAC) An obvious question is, what if we look at prior studies used in support of the current indication... A randomized trial of the use of erythropoietin in breast cancer patients found an increased risk of mortality in the EPO arm, relative to the placebo arm. (More on this later.) This led to extensive discussions with FDA and, ultimately, a presentation in May of 2004 to the Oncologic Drugs Advisory Committee (ODAC) An obvious question is, what if we look at prior studies used in support of the current indication...

11 Summary of Survival Over Time EPO-INT At-risk pts Placebo Epoetin Time, mo 25 Vertical line represents end of double-blind phase at 12 mo. †Based on Kaplan-Meier estimate. 12-mo mortality N n (%) † Censored P value Placebo (24) Epoetin alfa (30)81 12-mo hazard ratio (95% CI) 1.37 (1.07, 1.74)

12 Definitions Supportive anemia care: studies evaluating the established indication, treatment of anemia –Goal is to obtain sufficient rise in hemoglobin to reduce need for transfusion Beyond correction of anemia: investigational studies treating to higher hemoglobin levels –Includes most of the recent studies evaluating effects of erythropoietic agents on cancer treatment outcomes (response, progression, survival) Supportive anemia care: studies evaluating the established indication, treatment of anemia –Goal is to obtain sufficient rise in hemoglobin to reduce need for transfusion Beyond correction of anemia: investigational studies treating to higher hemoglobin levels –Includes most of the recent studies evaluating effects of erythropoietic agents on cancer treatment outcomes (response, progression, survival)

13 Supportive Anemia Care Studies Analysis Combined analysis from all completed, randomized, double-blind trials of anemic patients (n = 1,976; 10 trials) –Mortality hazard ratios –Thrombotic vascular event (TVE) frequency Considerations for this analysis –Variations in design and patient heterogeneity –Studies primarily evaluated transfusion reduction over 12 to 24 weeks Combined analysis from all completed, randomized, double-blind trials of anemic patients (n = 1,976; 10 trials) –Mortality hazard ratios –Thrombotic vascular event (TVE) frequency Considerations for this analysis –Variations in design and patient heterogeneity –Studies primarily evaluated transfusion reduction over 12 to 24 weeks

14 Supportive Anemia Care Studies No Survival Effect Seen in Combined Analysis Tumor type/study Hazard ratio Mixed (non-chemo)0.89 Mixed (non-cisplatin)1.08 Mixed (cisplatin)0.86 CLL (J89-040)1.68 CLL (P-174)0.42 Ovarian (EPO-INT-1)1.58 MM (EPO-INT-2)0.15 Mixed (EPO-INT-3)1.56 Mixed (EPO-INT-10)0.81 Mixed (PR )1.17 OVERALL0.99 Mortality, n/N (%) PlaceboEpoetin alfa 13/59 (22.0) 13/65 (20) 9/76 (12) 10/81 (12) 9/65 (14) 8/67 (12) 6/79 (8)16/142 (11) 1/12 (8) 1/33 (3) 2/80 (3) 6/164 (4) 7/76 (9) 1/69 (1) 3/65 (5) 9/135 (7) 22/124 (18)41/251 (16) 26/165 (16)31/168 (19) (0.76, 1.28) Favors Epoetin alfa Favors placebo HR (95% CI) log scale Test for heterogeneity, P =.66

15 J&J was not alone Two other companies presented results at the ODAC meeting: Roche and Amgen

16 Pooled 1 Analyses: Progression-free Survival Hazard Ratios Associated with Aranesp  (A) vs. Placebo (P) by Tumor Type 1 Studies , , , Hazard Ratio (95% CI) Ovarian (A = 11/49, P = 3/12) Breast (A = 16/94, P = 6/23) GI Other (A = 15/54, P = 4/13) SCLC (A = 40/60, P = 42/47) NSCLC (A = 99/146, P = 109/130) Lymphoma (A = 29/70, P = 32/60) Myeloma (A = 66/105, P = 57/86) CLL (A = 26/39, P = 22/28) Other (A = 23/91, P = 5/22) Overall (A = 325/708, P = 280/421)

17 Meta-analysis: Survival (Roche) 34 MF4249 MF4250 MF4252 MF4253 MF4266 MF4313 MF4321 MF4421 MF4467 Solid Hematological Other Category Total Study Tumor class Subgroup N Risk ratio Better with epoetin beta Better with placebo

18 Policy implications (1) Concerns about the meta-analyses –Studies are too heterogeneous  tumor types  cancer treatment regimens  other aspects of patient populations and treatment protocols (2004) –Meta-analysis may be “masking” signals in individual studies (2007) –Short-term follow-up, few events? –Bias toward the null due to crossover and other design flaws Concerns about the meta-analyses –Studies are too heterogeneous  tumor types  cancer treatment regimens  other aspects of patient populations and treatment protocols (2004) –Meta-analysis may be “masking” signals in individual studies (2007) –Short-term follow-up, few events? –Bias toward the null due to crossover and other design flaws

19 Policy implications (2) Next question to ODAC (2004, paraphrased): –We are asking for a large study in a single tumor type, with restricted treatment options to control extraneous sources of variability. Is it fair to generalize from a single tumor type to other sites? –(this is part of the dilemma) Next question to ODAC (2004, paraphrased): –We are asking for a large study in a single tumor type, with restricted treatment options to control extraneous sources of variability. Is it fair to generalize from a single tumor type to other sites? –(this is part of the dilemma)

20 Now here we are in 2008 Further studies (in head and neck cancer and in anemia of cancer) have found a “signal” Conducted outside the current label (high target Hb, or not chemotherapy-induced anemia) So, we were back at ODAC in 2007 and again in March, 2008 Further studies (in head and neck cancer and in anemia of cancer) have found a “signal” Conducted outside the current label (high target Hb, or not chemotherapy-induced anemia) So, we were back at ODAC in 2007 and again in March, 2008

21

22 Study-level Meta-analysis of Overall Death 46 Controlled CIA Studies (n=12,034; 6505 ESA; 5529 Control)

23 Risks of ESAs (from 2008 FDA presentation) 6 studies→ statistically significant evidence of ↑ tumor promotion and/or ↓ survival –BEST (Breast)* –ENHANCE (Head/Neck) –DAHANCA (Head/Neck) –161 (Lymphoid Ca)* –CAN-20 (NSCLC) –103 (Anemia of Cancer) (Many tumor types) 2 studies→ trends of ↑ tumor promotion and/or ↓ survival –PREPARE (neoadjuvant breast)* –GOG 191 (cervical cancer) † 6 studies→ statistically significant evidence of ↑ tumor promotion and/or ↓ survival –BEST (Breast)* –ENHANCE (Head/Neck) –DAHANCA (Head/Neck) –161 (Lymphoid Ca)* –CAN-20 (NSCLC) –103 (Anemia of Cancer) (Many tumor types) 2 studies→ trends of ↑ tumor promotion and/or ↓ survival –PREPARE (neoadjuvant breast)* –GOG 191 (cervical cancer) † * = pts receiving chemo † †=pts receiving chemoRT

24 Study N1° Endpoint ESA Adverse (from label) Outcome Chemo BEST (breast)93912 mo OS↓ 12 mo OS 161 (Lymphoid)344∆ Hgb↓ OS PREPARE (breast)733RFS, OS↓ RFS*, ↓ OS* GOG 191 (cervical)114PFS↓ OS* RT ENHANCE (H/N)351LR PFS↓ LR PFS, ↓ OS DAHANCA (H/N)522LRC↓ LRC, ↓ OS* No Chemo or RT CAN-20 (NSCLC)70QOL↓ OS 103 (Heterogenous)989Transfusion↓ OS *=trend

25 GOG-191 Cervical Results (N=114) 0.68, , % CI %61%Survival (3 yr) Local+Distant Recurrence PFS (3 yr) Endpoint 27% 33% 1.06 HR 59% ESA 62% Control HR = ESA : Control

26 Published meta-analysis (shortly before ODAC) Bennett et al. found mortality relative risk of 1.10 (95% CI: 1.01, 1.20) 27 Feb 2008 JAMA In an earlier presentation (Gleason et al., ASCO 2007, Dr. Bennett as senior author), analyses separated by “label status” (defined by baseline hemoglobin 12: –Off-label: RR = 1.14 (1.02, 1.27), I 2 = 12.7% –On-label: RR = 0.97 (0.86, 1.11), I 2 = 0.0% Bennett et al. found mortality relative risk of 1.10 (95% CI: 1.01, 1.20) 27 Feb 2008 JAMA In an earlier presentation (Gleason et al., ASCO 2007, Dr. Bennett as senior author), analyses separated by “label status” (defined by baseline hemoglobin 12: –Off-label: RR = 1.14 (1.02, 1.27), I 2 = 12.7% –On-label: RR = 0.97 (0.86, 1.11), I 2 = 0.0%

27 Mortality And ESA Response a : (Landmark At 4 Weeks On Treatment) All Randomized Studies, ESA-treated Subjects Study/Contrast Hazards Ratio (b) 95% Confidence Interval (b)P value (b) All studies, ESA-treated subjects (N=3750, # of events=1293) ESA Response: Increase vs. Stable0.84(0.72, 0.98)0.02 ESA Response: Decrease vs. Stable1.22(1.03, 1.45)0.02 ESA Response: Missing vs. Stable1.13(0.92, 1.38)0.24 a) Increase: Increase of >0.5 g/dL; Stable: change of ≤ 0.5 g/dL; Decrease: a drop of >0.5 g/dL; In hemoglobin over the pre-treatment level, independent of transfusion b) Analyses were based on Cox's proportional hazards model, adjusting for ECOG score, cancer type, advanced disease, pre-treatment hemoglobin and other baseline variables, stratified by study.

28 Mortality And ESA Response a : (Landmark At 4 Weeks On Treatment) Beyond-anemia-correction studies, ESA-treated subjects Study/Contrast Hazards Ratio (b) 95% Confidence Interval (b)P value (b) Beyond-anemia-correction studies, ESA-treated subjects (N=2089, # of events=758) ESA Response: Increase vs. Stable0.91(0.75, 1.12)0.38 ESA Response: Decrease vs. Stable1.50(1.21, 1.86)0.00 ESA Response: Missing vs. Stable1.19(0.88, 1.60)0.26

29 Mortality And ESA Response a : (Landmark At 4 Weeks On Treatment) Anemia-correction studies, ESA-treated subjects Study/Contrast Hazards Ratio (b) 95% Confidence Interval (b)P value (b) Anemia-correction studies, ESA-treated subjects (N=1661, # of events=535) ESA Response: Increase vs. Stable0.75(0.60, 0.95)0.02 ESA Response: Decrease vs. Stable0.92(0.69, 1.22)0.56 ESA Response: Missing vs. Stable1.00(0.75, 1.32)0.99

30

31 Methodologic issues (general) Standardization of definitions across studies  Endpoints  Adverse events  Requires standardization of data collection across studies! (SPERT) What about adjudication?  Does it depend on the endpoint? (Can we count deaths or not?) Broad versus narrow definitions (numbers versus signal strength) Standardization of definitions across studies  Endpoints  Adverse events  Requires standardization of data collection across studies! (SPERT) What about adjudication?  Does it depend on the endpoint? (Can we count deaths or not?) Broad versus narrow definitions (numbers versus signal strength)

32 More methodologic issues –Long-term versus short-term follow-up  Non-constant hazards / hazard ratio?  When is it OK to discard data? –Studies with small numbers of events?  When is it OK to discard data?  Get down-weighted in analysis anyway –Beware of confounding of study-level characteristics!!!!!!!  E.g., low dose study of aspirin in women, higher dose study in men –Long-term versus short-term follow-up  Non-constant hazards / hazard ratio?  When is it OK to discard data? –Studies with small numbers of events?  When is it OK to discard data?  Get down-weighted in analysis anyway –Beware of confounding of study-level characteristics!!!!!!!  E.g., low dose study of aspirin in women, higher dose study in men

33 A Few Words on Endpoint Definitions Common view is that more sensitive definitions –Are more “conservative” by being inclusive –Increase power by generating more events Overly broad inclusion of events can lead to an underestimation of the true relative risk –might include events less likely to be related to the true (but possibly unknown) mechanism of action or –by their nature, are simply more likely to be misclassified in clinical trials Implications of “non-differential” misclassification in efficacy versus safety settings? O’Neill RT. Assessment of safety. Biopharmaceutical Statistics for Drug Development. Peace KE, ed. New York: Marcel Dekker;1988:543–604. Proschan MA, Lan KKG, Wittes JT. Statistical Methods for Monitoring Clinical Trials Springer, New York, 2006 Common view is that more sensitive definitions –Are more “conservative” by being inclusive –Increase power by generating more events Overly broad inclusion of events can lead to an underestimation of the true relative risk –might include events less likely to be related to the true (but possibly unknown) mechanism of action or –by their nature, are simply more likely to be misclassified in clinical trials Implications of “non-differential” misclassification in efficacy versus safety settings? O’Neill RT. Assessment of safety. Biopharmaceutical Statistics for Drug Development. Peace KE, ed. New York: Marcel Dekker;1988:543–604. Proschan MA, Lan KKG, Wittes JT. Statistical Methods for Monitoring Clinical Trials Springer, New York, 2006

34 Conclusions (1) Meta-analysis has valuable applications in pharmacoepidemiology –Evaluation of safety using existing randomized trials –Evaluation of safety using non-experimental studies (need more time to show) Meta-analysis has valuable applications in pharmacoepidemiology –Evaluation of safety using existing randomized trials –Evaluation of safety using non-experimental studies (need more time to show)

35 Conclusions (2) There are challenging methodologic issues in the meta-analysis of safety data –Rare events, multiplicity, adjudication, … Sensitivity analyses should always be performed –Then more sensitivity analyses should always be performed Use patient-level data when possible There are challenging methodologic issues in the meta-analysis of safety data –Rare events, multiplicity, adjudication, … Sensitivity analyses should always be performed –Then more sensitivity analyses should always be performed Use patient-level data when possible