1. Department Author Bayesian Sample Size Determination in the Real World John Stevens AstraZeneca R&D Charnwood Tony O’Hagan University of Sheffield

2. Department Author Reference: Bayesian Assessment of Sample Size for Clinical Trials of Cost-Effectiveness O’Hagan A, Stevens JW Medical Decision Making 2001;21:219-230

3. Department Author Contents The Sample Size Problem The Utility Function Study Objectives The Prior Distribution An Example Discussion

4. Department Author The (Real World) Sample Size Problem A study is to be designed to compare the efficacy of two treatments, Treatment 2 (the experimental treatment) and Treatment 1 (the control treatment). Consideration is to be given to the gain (i.e. profit) achieved from sales of the new treatment if it is successfully marketed, the loss (i.e. costs) associated with setting up the study and the cost per patient in the study, allowing for any delay in coming to market. It will be assumed that all gains and losses are associated with the conduct of a single study. What is the optimal sample size?

5. Department Author The Utility Function In order to determine sample size using decision theory it is necessary to define a Utility Function. The Utility Function can be written as : U(n 1, n 2, x) = u. L(x) - c. (n 1 + n 2 ) -c 0 where, n 1 and n 2 are the sample sizes in the two treatment arms, x denotes the data obtained in the study, L(x) takes values one if the data, x, are convincing enough to the regulator to approve the drug and zero otherwise, u is the profit to the company if the drug is approved, c is the cost per patient in the study (allowing for delays in marketing), c 0 is a set-up cost for the study.

6. Department Author Sample Size - The Two Stages There are two stages in the conduct of a study: Design stage. Plan the study to maximise the expected utility associated with the desired outcome. Analysis stage. Analyse the data from the study to see whether we can report the desired outcome. These two stages are reflected in setting the objectives, how we do the sample size calculations.

7. Department Author Study Objectives Analysis Objective: We will regard the outcome of the study as positive if the data obtained are such that there is a probability of at least ω that Treatment 2 is more efficacious than Treatment 1. Design Objective: We wish to maximise the expected utility, U(n 1, n 2 ), associated with the desired outcome.

8. Department Author Analysis Objective Frequentist formulation We wish to reject the null hypothesis that μ 2 - μ 1 = 0, at the 100(1  )% level of significance. e.g.  0.95 corresponds to usual 5% significance test in a one-sided test Bayesian formulation We wish to have at least a 100  % posterior probability that μ 2 - μ 1 > 0.

9. Department Author Design Objective Bayesian formulation We want to choose sample sizes that maximise the expected utility associated with achieving the desired posterior probability that μ 2 - μ 1 > 0.

10. Department Author Expected Utility L(x) is a function of the sample size in each treatment group. The expected value of L(x) is P(n 1, n 2 ), and is the probability of obtaining data to convince the regulator. The expected Utility is then, U(n 1, n 2 ) = u. P(n 1, n 2 ) - c. (n 1 + n 2 ).

11. Department Author Understanding the Bayesian Formulation At the Analysis stage: The Bayesian and frequentist formulations are similar (particularly if we employ weak prior information in the analysis of the data). The Bayesian statement is often how the p-value is interpreted anyway. At the Design stage: The frequentist and Bayesian formulations are different. The frequentist approach fixes the parameters at (more or less) arbitrary values. The Bayesian formulation defines prior distributions for the parameters.

12. Department Author The Prior Distribution When advocating a Bayesian approach, the usual question arises. What about the prior distribution? Various options are available: Realistic beliefs of the wider community The company’s genuine prior beliefs Non-informative priors Sceptical priors

13. Department Author Which Prior Distribution?

14. Department Author The Two Prior Distributions We will allow different prior distributions at the two stages of design and analysis. Analysis prior distribution. This will typically be non-informative, sceptical or some kind of consensus. Design prior distribution. This should generally be based on all information available to the company.

15. Department Author Example Modified from Briggs and Tambour (1998). Bayesian formulation, analysis and design priors informative and identical, prior means μ 1 = 5, μ 2 = 6.5; prior variance = 4; prior co-variance = 3, ω = 0.975 (1-sided, equates to 5% 2-sided test). Profit : £5bn Cost per patient : £1000; £10,000; £100,000; £1,000,000 (Ratio : £5,000,000; £500,000; £50,000, £5,000)

16. Department Author

17. Department Author

18. Department Author Example - Conclusions We have a prior probability of 85.55% that Treatment 2 is more efficacious than Treatment 1, which we approach rapidly at sample sizes above 2000 per treatment group. The maximum utility (return on investment) depends on the ratio of the profit to the cost per patient. Unnecessarily large sample sizes reduce the return on investment with no major increase in the probability that the regulator will be convinced to approve the registration of the treatment.

19. Department Author Discussion The Bayesian approach allows considerable flexibility to represent the (real world) problem. The Bayesian approach encourages the assessment of the genuine beliefs regarding the true treatment means. Unequal sample sizes could be used is there is a prior belief that the variances are different between treatment groups. The design objective allows the incorporation of prior information in determining the optimal sample size to maximise the return on investment. Using the decision-based approach could provide an informed basis for management to use when allocating limited clinical resources to different clinical projects.