1Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting Futility stopping Carl-Fredrik Burman, PhD Statistical Science Director AstraZeneca R&D

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 2 Stakeholder perspectives  The patient  A pharmaceutical company  The public (MRC, NIHR)

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 3 The fundamental design requirement: Ethics ”My old mother – principle” The trial is ethical if (and only if) I would recommend my mother to take part in the trial, given that she would be eligible.

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 4 Interim stopping  Stop the trial as soon as I would not include my mother, e.g. if  One (publicly available) treatment is clearly better  A “new” treatment fails to show sufficient effect, when it has known safety disadvantages  No ethical obligation to stop  If two treatments with similar safety have no clear difference in effect

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 5 (Genuine) informed consent  The patient should get  Full information regarding the trial treatments (and procedures), including previous data, potential risks, etc.  Help to understand the information and  Apply it to his/her specific situation (health status, preferences)  When would a fully informed, fully competent patient give consent?  If and only if it is better (not worse) for him/her to take part in the trial, as compared to receiving standard therapy.  Cf. “my old mother” principle

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 6 Easy-going clinical equipose is not enough  Clinical equipose  If there is uncertainty about which treatment is better  (Alternatively, compelling evidence of one treatment being better)  (Alternatively, medical experts disagree)  It’s far too easy to say that we are uncertain  I expect my doctor to say what he believes is best

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 7 Our old Mother  Scientific equipose  Not every expert agree on CO 2 -induced global warming  Do you suggest a randomised N-of-1 trial?  Of course not — choose the treatment we believe is best Earth

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 8 What is ”best” for the patient? May depend on e.g.  Effect (best guess + uncertainty)  Safety  Better care in the trial?  Economic compensation (but beware of exploitation)  Altruism Likely effect will differ between individuals (covariates) Preferences are different Decision theory may help decide (at least in theory …)

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 9 Decision analysis (DA) Patient perspective  Utility function U(effect, safety, QoL, cost, …)  Model for effect, safety, etc., based on best information (data, expert knowledge, …). Often Bayesian prior.  Choose decision (volunteer to participate in trial, or not) to maximise expected utility The DA approach can also be used by a trial sponsor

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 10 A pharmaceutical company perspective (simplified)  A new drug will be licensed if and only if the (next) phase III trial has a statistically significant effect (p<5%)  If licensed, the company will make a profit of V (unit: £)  The trial cost is k·N, where N is the sample size  The assumed (believed) treatment effect is .  Maximise V · Power(N) – k · N Of course, this model is wrong (as all models are). Should e.g. have V=V(T)=V(T(N)), where T is time.

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 11 Gain Net gain = Gain – Cost Cost Optimal sample size N opt = 1010

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 12 The interim decision (continue vs. stop for futility)  Value V if significant  Conditional power CP if trial is continued  C additional trial cost if continued (compared to if stopped)  Continue iffV · CP > C, that is, iff CP > C / V

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 13 DA vs. ”least clinically relevant” effect  DA approach:  Maximise expected utility based on ”best guess” effect (or prior)  Traditional approach:  90% power at ”least clinically relevant” effect  What is the least clinically relevant effect?  If no adverse effects, no cost  And the outcome is death  One single saved life is clinically relevant … at least to the one saved  What is a relevant effect depends on safety, cost etc.

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 14 Conditional power at interim  Final estimate  is N( , 1/N).  Stage i has sample size N i and estimate  i. Then  = (N 1 ·  1 +N 2 ·  2 )/N  Statistical significance if  > C /  N (where C=1.96 say)  CP = P(  > C /  N ) =  (  ·  N 2 +  1 ·N 1 /  N 2 -C  (N/N 2 ) )  But which  to use when calculating CP?  Original alternative  Alternative ?  Interim estimate  1 ?  Linear combination of  1 and  Alternative ?  Bayesian posterior based on interim data ?

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 15 Stop, continue, or something else?  Run a new trial?  Sample size reestimation, based on interim estimates  Flexible design methodology (Bauer & Köhne –94)  Predefined weights for the different stages (generally, weight not proportional to information)  May change the sample size for stage 2 after viewing interim results  Discussion on CP  Somewhat controversial  May be better than design with only futility stopping  Group-sequential designs should often be preferred

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 16 Publicly funded trial: Treatments with similar safety  Assume  Whole patient population will receive one of these treatments  Efficacy is the only unknown  Same safety, cost, etc.  The closer the interim effect is to zero, the more value in continuing  Thus, no reason to stop for futility

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 17 Example 1: Value of information  Compare 2 treatments with probabilities p A, p B for death.  Assume total future population size is T (10,000 say)  If we knew that , we would choose treatment A  T·  lives would be spared as compared to using B  Similarily, choose B if  <0  Net value T·Abs(  ) or T·Abs(  )/2 if compared to using random treatment

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 18 Maximal value of information  Before trial,  p 2  p 1 has approximately normal prior with mean=0, SD=  (say 10%)  What would the value be if we could learn the exact value of  ?  Take the Bayesian expectation of the value T·Abs(  )/2, E prior [T·Abs(  )/2] = T·  /  (2  )  With T=10,000 and  =10%, about 400 lives would be spared Example cont’d

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 19

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 20 Publicly funded trial: Intervention vs. no treatment (placebo)  Assume  Intervention is associated with some cost, safety risks  Not clear whether intervention has a positive effect  If effect, then the size of the effect will determine the size of the patient population which will get a positive net benefit  First objective: is there any effect?  Reasonable to stop for futility if interim estimate is low  Expected value by continuing study is then small

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 21 Information leakage  In regulatory setting, large discussion on who should see interim data  Does the DMC have to be independent from the sponsor  What are the risks of potential information leakage?  Problems may be over-emphasised?  The ethical aspect

Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting 22 Summary  Futility stopping may be an ethical requirement  Industry funded trials: Tradeoff cost and expected value  Publicly funded trials (examples)  Don’t stop for futility if two active treatments differ only in effect  May stop for futility if “active” treatment unlikely to have sufficient effect (tradeoff cost and value)  (If basic science objective …)