1. 2006 FDA/Industry Workshop Advantages and Challenges of Bayesian Clinical Trials Increased Bayesian Submissions: A future state of Drug applications? Stacy R. Lindborg, Eli Lilly & Co. September 29, 2006

2. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 2 Acknowledgments Melissa E. Spann, Eli Lilly & Co. John W. Seaman, Jr., Baylor University

3. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 3

4. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 4 Investment in Research and Development Continues to Grow Source: PhRMA 2006 Industry profile, (http://www.phrma.org/files/2006%20Industry%20Profile.pdf )

5. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 5

6. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 6 Challenges for the Implementation of Bayesian methods Lack of Bayesian analyses in Regulatory interactions –Many reasons (e.g., computing, FDA acceptance?) –Focus on classical statistics by many graduate programs resulting in an unfamiliarity with Bayesian inference among many statisticians (PhRMA & regulatory globally) –Uninformed scientists (even statisticians) about how a Bayesian analysis is performed (i.e. prior elicitation). –Incorrect Conclusion: Bayesian inference is not acceptable within the pharmaceutical industry

7. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 7 2006 ENAR Roundtable Topic: The Use of Bayesian Inference in Drug Approval Un-informed comment from a (senior) colleague in the Pharmaceutical industry … Bayesian inference can never be used in credible settings. If you dont like the answer then one can simply change the prior to get the answer you like! The basic tenants of a good trial are the same for both Bayesian & frequentist trials…Bayesian trials start with a sound clinical design (source: FDA CDRH draft Guidance)

8. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 8 FDA/Johns Hopkins Workshop May 20-21, 2004 Can Bayesian approaches to studying new treatments improve regulatory decision-making? Senior FDA leadership: Lester Crawford, Janet Woodcock, Bob Temple, Center Division Directors (CVM, CDER, CBER, CDRH) Web-cast of meeting: www.prous.com/bayesian2004 Website to speaker slides: http://www.cfsan.fda.gov/~frf/bayesdl.html Clinical Trials (special issue, dedicated to discussion): Vol 2, No 4, Aug 2005

9. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 9 FDA/Johns Hopkins Workshop A few points… Use of Bayesian methods: if you use these models, all will get easier and drug development will speed up –Temple: This statement is not true in my opinion. In some cases my prior will be very negative, as in case of drugs for sepsis… (dont assume it will always speed up) –T.Louis: Sometimes better might require more patients to overwhelm the prior –G.Campbell: Its a lot of work for everyone –J.Seigel: Makes it harder to approve drugs that arent already a foregone conclusion. Also makes it easier to approve drugs we already know work. –Pfizer did 1000s of simulations to understand operating characteristics. What sample size is needed to accomplish traditional power estimates? Simulations need to plays important role – operating characteristics of posterior distribution (i.e., frequentist characteristics) are necessary (D.Rubin) –Operating characteristics: Type I error or some analog protects the public (G.Campbell) –Dont just do where we have to. They make sense (S.Ellenberg) The issue of transparency is very important (S.Ellenberg)

10. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 10 Critical Path is the FDA's premier initiative to identify and prioritize the most pressing medical product development problems and the greatest opportunities for rapid improvement in public health benefits. Its primary purpose is to ensure that basic scientific discoveries translate more rapidly into new and better medical treatments by creating new tools to find answers about how the safety and effectiveness of new medical products can be demonstrated in faster timeframes with more certainty, at lower costs, and with better information. FDA Critical Path Initiative

11. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 11 Critical Path Topic 2: Streamlining Clinical Trials Creating Innovative & Efficient Clinical Trials & Improved Clinical Endpoints Advancing Innovative Trial Designs Design of Active Controlled Trials Enrichment Designs Use of Prior Experience or Accumulated Info. in Trial Design Development of Best Practices for Handling Missing Data Development of Trial Protocols for Specific Therapeutic Areas Analysis of Multiple Endpoints

12. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 12 Bayesian Methods in Clinical Research? A few points worth considering… Ability to calculate probability of interest and interpret easily –No arguments that p-values indirectly aide in decision making –P-value assumes H 0 is true, cant represent if H 0 is true. –Bob Temple: using p=.05 is not an appropriate measure of strength of information. Everyone knows p-values are stupid Joint probability estimation on 2+ parameters (multi-level structure & compute joint posterior distribution for all unknowns) –Critical Success Factors Adaptive designs– flexible, no penalty for looking. Learn faster Borrow Strength (historical data, trials, across pt. groups & Dx) FDA Modernization Act of 1997 - least burdensome means of demonstrating effectiveness or substantial equivalence

13. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 13 Bayesian Methods in Clinical Research? Non-inferiority trials lend themselves to Bayesian analyses –Ease of incorporating variability appropriately Ability to predict - posterior predictive distributions. The ultimate goal of clinical trials is to predict an outcome variable of interest and to compare predictions across treatments. Other areas for predictive probabilities: e.g., pretrial predictions – used to compare different sample sizes & designs and aid to judge competing trials considering different treatments. e.g.2: wrt interim analyses, given the data what is the chance of obtaining a significant result? Ease in complex modeling Appropriately account for variability (e.g., different correlations over course of trial in key measures, between center differences, appropriate model for meta-analysis – including the case where the control rate varies across trials…)

14. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 14 NEW Draft CDRH Guidance Draft Guidance available: http://www.fda.gov/cdrh/guidance.html

15. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 15 Where do we go from here? B. Temple – apply Bayesian and frequentist methods to a classical study design; compare. –Consider retrospective studies to accomplish these types of comparisons and develop confidence. Olanzapine Intramuscular: phase III study in clinically agitated patients with Schizophrenia

16. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 16 Phase III Study Design Clinically agitated patients with schizophrenia Primary Objectives (IM): 2 hr change, PANSS-EC 1.Superiority of Olz to Pla 2.Non-inferiority of Olz to Hal Secondary Objectives: Safety of Olz relative to Pla and Hal

17. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 17 Why Bayesian Adaptive Design (AD)? Among other reasons, a Bayesian AD provides an ability to: Formally leverage prior data Ability to model key efficacy and safety variables and continuously monitor and report updated predictive probabilities of interest Bayesian ANOVA: same data model as employed in frequentist analysis

18. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 18 Olanzapine RAIM Adaptive Design benefits? Appropriately employed (the where & how): Decrease cycle time –Greater than expected effect size (vs. Hal & Pla), stop sooner? Improve p(TS) –Increase quantity and precision of information Ethics –Optimize patient treatment –Minimize patient exposure to ineffective doses/Tx Minimize Hal exposures decreased dystonia reactions? Minimize Placebo exposures? Additional Olz IM data?

19. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 19 Prior data explored Oral vs. IM PK data? IM & oral PK profiles very similar, efficacy timing very different Efficacy Scales from Phase II? No PANSS EC data Tranquallization scale (e.g., ACES) ­ Conflicting results between 2 trials ­ Early Tranquallization scales failed to discriminate between Agitation and Calming Scale: 1 = Alert (including agitated, aggressive), 2 = Tranquillization (calming of mood or motor activity) Efficacy data from Olanzapine Oral? HGAJ data, PANSS EC inclusion entry criteria imposed – provided estimate of treatment difference with Agitation (not very precise!) Responder analyses from Olanzapine and Haloperidol Safety data: Olanzapine Oral & Haloperidol literature? A lot of data for prior use

20. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 20 Prior Distributions: Mean Change in PANSS-EC 1 =IMHal7.5 2 =IMOlz10 3 =IMPla *Center all distributions at zero to reflect a prior belief that all treatments are ineffective **Assume a different data model with flat prior distributions across the support of the parameter ***Reflect potential lack of confidence in IMOlz10 relative to the IMPla and IMHal7.5 * ** *** Informative prior elicitation was performed by way of a planning committee comprised of statisticians, physicians, pharmacokinetic scientists, and regulatory scientists.

21. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 21 Other prior distributions elicited Probability of Dystonic Reaction (Safety) Probability of clinical response without Dystonia Informative prior elicitation was performed by way of a planning committee comprised of statisticians, physicians, pharmacokinetic scientists, and regulatory scientists. (See back-up slides for details)

22. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 22 Bayesian Adaptive Non-Inferiority Design with Safety Assessment Fixed Allocation Calculate Posterior Probabilities Safety STOP Efficacy STOP Futility STOP Minimum Sample Sizes STOP Maximum Sample Sizes Adaptive Allocation CONTINUE REPEAT PROCEDURE Hal Olz Pla Hal Olz Pla Select Treatment Treat Patient Update Data

23. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 23 Adaptation and Stopping Minimum sample size required before adaptation begins including stopping for futility or efficacy: 30 placebo, 60 Hal IM and 60 Olanzapine IM Stopping Criteria: Stop for Futility: P(Non-Inferiority) 0.90 and P(Superiority)>0.90 Maximum sample size allowed (# observed in trial given retrospective nature) Stop for Safety

24. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 24 MCMC Multiple Integration Estimation Gibbs sampling used to estimate the marginal distribution. Convergence for reliable parameter estimation was assessed by visual inspection of graphical diagnostics and the calculation of the Gelman-Rubin (Gelman 2004) convergence diagnostic Computing: SAS IML WinBugs

25. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 25 Frequentist vs. Bayesian Conclusion p<.001 Lower bound of 1-sided 97.5% confidence interval 0 but >LL Frequentist Olz Superior to Pla Olz Noninferior to Hal 3.05 Lower Limit IMHal7.5 Better IMOlz10 Better -1.12 1-sided 97.5% CI Non-inferiority margin between Olz and Hal, 40% i.e., max. allowable difference in pt response

26. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 26 Bayesian Approach to Non-Inferiority and Superiority 1 =Hal 2 =Olz 3 =Pla Probability Olz is non-inferior to Hal Probability Olz is superior to Pla j is the change in PANSS-EC 2 hrs post baseline

27. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 27 Frequentist vs. Bayesian Conclusion p<.001 posterior prob = 1 Lower bound of 1-sided 97.5% confidence interval 0 but >LL posterior prob =1 Frequentist Bayesian Olz Superior to Pla Olz Noninferior to Hal The probabilities of superiority of Olz vs. Pla and non- inferiority of Olz vs Hal were 1 for all priors

28. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 28 The true benefit of retrospective design 24 patients could have been spared Placebo treatment Fewer Haloperidol exposures (i.e., decreased dystonic reactions) $ 1 mil Savings ($6.4k/pt, clinical grant savings) 40% time savings (stop after 6 of 10 mo.) Trial stopped: criteria met to declare Olanzapine IM superior to Placebo and non-inferior to Haloperidol IM *note: max value based on prior sensitivity analysis (min=62)

29. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 29 Future role of Bayesian inference in the pharmaceutical industry?

30. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 30 Back-up Slides

31. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 31 Convergence A separate experiment was conducted to assess convergence of prediction estimates. We generated n=100 data sets then used over dispersed initial starting values for m=3 chains to calculate the Gelman-Rubin convergence diagnostic. A burn in of 100 with 1000 iterations yielded GR values ranging from.999537 to 1.002757 indicating that the estimated predictions converge under diffuse priors. The same type of experiment was performed for the effect size model with similar results.

32. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 32 Priors Distributions: Probability of Dystonia ***** 1 =IMHal7.5 2 =IMOlz10 3 =IMPla Diffuse priors for binomial model were used when unfavorable priors for mean change in patient condition were used **Assume a different data model with flat prior distributions across the support of the parameter ***Reflect potential lack of confidence in IMOlz10 relative to the IMPla and IMHal7.5 Informative prior elicitation was performed by way of a planning committee comprised of statisticians, physicians, pharmacokinetic scientists, and regulatory scientists.

33. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 33 Priors for Probability of Patient Responding without Dystonia ***** 1 =IMHal7.5 2 =IMOlz10 3 =IMPla Diffuse priors for binomial model were used when unfavorable priors for mean change in patient condition were used **Assume a different data model with flat prior distributions across the support of the parameter ***Reflect potential lack of confidence in IMOlz10 relative to the IMPla and IMHal7.5 Informative prior elicitation was performed by way of a planning committee comprised of statisticians, physicians, pharmacokinetic scientists, and regulatory scientists.

34. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 34 Joint Posterior Predictive Probability & Allocation Probabilities The joint posterior predictive probability of the next patient responding to treatment and not experiencing dystonia is The allocation probabilities are determined by

35. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 35 Predictive Probabilities: Responder & Safety The posterior predictive conditional probability that the next patient responds to treatment (40% reduction in agitation two hours post baseline) given that he does not experience dystonia The posterior predictive probability that the next patient does not experience dystonia is (See back-up slide for joint posterior predictive probability computations & allocation probabilities)

36. FDA/Industry Workshop 2006 S.Lindborg Company Confidential Copyright © 2000 Eli Lilly and Company 36 Examples of present Bayesian application at Lilly Multiple Bayesian designs (including adaptive) being used across the portfolio (Oncology, Endocrine, NSD) Bayesian inference used to determine Pr(technical success) at decision points for compound development. Used in Modeling and simulation Global Product Safety: meta-analysis work