1. Optimal Drug Development Programs and Efficient Licensing and Reimbursement Regimens Neil Hawkins Karl Claxton CENTRE FOR HEALTH ECONOMICS

2. Overview Societal and commercial value of Information Decision rules incorporating value of information Challenges

3. The Quantitative Estimate of the Value of Sample Information The value of additional sample information is the value of the increased likelihood of selecting the optimum treatment arising from the reduction in uncertainty regarding treatment effects (and costs).

4. What is the optimum treatment? The treatment with greatest expected net benefit in terms of costs and effects- Bayesian Decision Rule.

5. Decision Uncertainty

6. Expected Net Value of Sample Information (ENVSI) Expectation over potential future samples of: Net benefit from optimum decision made including the additional sample data — Net benefit from optimum decision based on existing data — Cost of collecting sample Note: ENVSI < 0, if the optimum decision does not change due to extra sample data

7. Bayesian Simulation of Future Samples for Binomial Parameter Sample P from current posterior distribution: P current ~ beta(a,b) Simulate Trial Data r T ~ bin(n, P Tx ) Calculate new posterior distribution P new ~ beta(a+r,b+n-r)

8. Small Sample (n=1)

9. Large Sample (n=2000)

10. Example Phase II Trial Results χ 2 Test: 2.0037, 1 df, p = 0.1569

11. Decision Analytic Model Net Benefit = P(Resp) x QALYs Gained | Resp x Monetary Value of a QALY - P(Death) x QALYs Lost | Death) x Monetary Value of a QALY - Treatment Cost

12. Example Parameter Estimates QALYs Gained | Response : ~ N(0.7,0.1 2 ) x 4 QALYs Lost | Death : ~ N(0.7,0.1 2 ) x 4 Value of a QALY: £30,000 Treatment Cost Per Course :(<10%  Response) £12,000 (≥10% & < 20%  Response) £14,000 (≥20%  Response) £16,000 Treatment Population:20,000 Production Costs: £150,000,000 Trial Costs: (Fixed) £10,000,000 (per Patient) £20,000

13. Based on Current Data New treatment is cost-effective New treatment would not get approval based on a frequentist hypothesis test

14. Societal Value of Sample Information (Efficacy Trial)

15. Commercial Value of Sample Information The value of increased sales due to the increased probability of regulatory and reimbursement approval arising from the extra information

16. ICH E9: Guidance on Statistical Principles for Clinical Trials Using the usual method for determining the appropriate sample size, the following items should be specified: probability of erroneously rejecting the null hypothesis probability of erroneously failing to reject the null hypothesis

17. ICH E1A: The Extent of Population Exposure to Assess Clinical Safety 100 patients exposed for a minimum of one-year is considered to be acceptable to include as part of the safety data base. It is anticipated that the total number of individuals treated with the investigational drug, including short- term exposure, will be about 1500.

18. Probability of Approval (Efficacy Endpoint – Current Regulatory Regimen)

19. Value of Sample Information (Efficacy Endpoint - Current Regulatory Regimen)

20. What happens if we just use a Bayesian CE decision rule?

21. Value of Sample Information (Efficacy Endpoint - Bayesian CE Decision Rule)

22. Societal Value of Sample Information (Utility Study)

23. Value of Sample Information (Utility Study – Current Regulatory Regimen)

24. Implications Under current regulatory system we might expect a lack of outcomes and long-term data We need to consider uncertainty and resulting VOI when making decisions, not just expectations based on current data How should we do this? Maybe we shouldn’t abandon frequentist hypothesis testing just yet?

25. The FDA view “...A reasonable basis for a claim [of cost-effectiveness] depends on a number of factors relevant to the benefits and costs of substantiating a particular claim. These factors include: the type of product, the consequences of a false claim, the benefits of a truthful claim, the costs of developing substantiation for the claim...”

26. Potential Industry Responses to Approval based on Value of Information Reduce cost of uncertainty by research or price reduction Trade-off between: Additional research -Cost, delay and uncertain outcome -Entry and free rider Price reduction -Reduces EVI (for payoffs > 0) but reduces revenues

27. Approval Based on Expected Net Value of Sample Information Approve new (more expensive?) treatment if expected net benefit of treatment is greater than existing treatment and expected net value of further sample Information is zero

28. Approval Based on Expected Net Value of Sample Information Hard to define set of endpoints, study designs and sample space over which we calculate value of sample information ENVSI is uncertain and will change as data become available. When is ENVSI defined? Many of the parameters required to estimate ENVSI are uncertain and may not be transparent Non-financial capacity restraints on further research What decision do we make in the interim? - Sunk costs, irreversibility and option value

29. Approval based on Expected Value of Perfect Information

30. Summary results of the NICE pilot study

31. Approval based on Expected Value of Perfect Information Approve new therapy if Expected Value of Perfect Information is below a given threshold at an acceptable cost-effectiveness threshold Requires an arbritary EVPI threshold for approval Parameters still uncertain. For example; relevant time horizon, future technological change.

32. Approval based on Decision Uncertainty

33. Approve new therapy if decision uncertainty is below a given threshold at an acceptable cost-effectiveness threshold Requires an arbritary uncertainty threshold for approval

34. Some Challenges How we consider uncertainty when decision making will influence the availability of evidence How do we frame explicit decision rules incorporating uncertainty?