The DMC’s role in monitoring futility Janet Wittes SCT Liverpool, England 4-May-2017

Outline What is futility? What kinds of futility “bounds” should one consider? What if the study has no futility “bound” but DSMB recommends stopping? a futility “bound” but DSMB doesn’t recommend stopping How do you deal with subgroups, secondary outcomes, other? General conclusions

What is futility?

“Futility”, aka “lack of efficacy” Study is x% of the way through The probability of the trial’s showing benefit is low Most relevant in testing new drug or strategy Idea: Expensive (and not ethical?) to continue Not relevant if study compares two treatments in common use Goal: to get a good estimate of the difference E.g., CATT (Lucentis vs. Avastin for AMD)

Types of futility “bounds”

Several choices for recommending stopping If power, given data thus far, is low Conditional power (CP) – look at a variety of alternatives Predictive P – CP integrated over distribution of alternatives

Conditional power

Predictive probability

Some other definitions of futility Current observed effect is essentially zero If data rule out alternatives of interest Emerson and Fleming – Biometrics 45: 905-923

But don’t just give a “rule” Show how it is likely to play out in practice

DMC members are in for the long haul In talking to DMCs, we often focus on early stopping Harms or overwhelming benefit or “futility” But DMCs don’t want to stop too early They want the data to be convincing Once a trial stops, restarting is difficult On the other hand… they don’t want to continue if balance of risk & benefit unfavorable

Study has no futility “bound” but the DMC recommends stopping

Example: chronic, uniformly fatal disease Outcome: lung function Last patient to be followed for 48 weeks Stopping guidelines No stated guidelines for safety No interim analysis for efficacy or futility DSMB was looking at data every three months

The DMC recommended stopping for futility Fully enrolled: ~500 randomized – 250 active; 250 placebo 3 months more to go in trial No difference in lung function Active group had more non-serious GI adverse events DSMB: recommend stopping – why make these people feel bad? Sponsor was angry at the decision

Study has a futility “bound” but the DSMB doesn’t recommend stopping

VA HIT trial (HIT=HDL Intervention Trial) Gemfibrozil vs. placebo in those with low HDL, normal LDL Informal futility boundary If conditional power is very low, recommend stopping No difference for two years

What the DSMB saw for the first two years was essentially no difference. But the committee thought that the effect might take a while to become evident.

At the end of the trial, the effect was strong At the end of the trial, the effect was strong. An early “futility” stop would not have identified this.

Dealing with subgroups, secondaries, and items of interest

Subgroups and secondary outcomes What if overall the study is “futile” but A subgroup of interest is showing benefit? An important secondary is showing benefit E.g., time to first exacerbation “futile” But, treatment group has many fewer exacerbations Often DMC will want to continue in such cases Recommendation: talk to the Sponsor/Exec Comm up front

What if there is something “interesting” Often the DMC will see something of interest A small subgroup A putative mechanism Should it continue? My general view is “no” That’s asking for a different experiment

Conclusions – what you should plan It is usually good to have a discussion of futility up front It is often good to have a guideline for futility But make sure the guideline is not abstract Show examples of what would constitute a “stop”

In my experience, there is often less agreement in the DMC about futility than about safety or efficacy DMC may recommend stopping for futility without a guideline DMC may not stop even with a guideline It sees promising results in an important subgroup It feels that continuing likely to gain important knowledge It is insecure about the data It suspects there may be a lag in effect Ultimately, the DMC (and everyone) wants a study to have a clear result which affects the decision to pull or not to pull the plug