1. The DMC’s role in monitoring futility
Janet Wittes SCT Liverpool, England 4-May-2017

2. Outline What is futility?
What kinds of futility “bounds” should one consider?
What if the study has
no futility “bound” but DSMB recommends stopping?
a futility “bound” but DSMB doesn’t recommend stopping
How do you deal with subgroups, secondary outcomes, other?
General conclusions

3. What is futility?

4. “Futility”, aka “lack of efficacy”
Study is x% of the way through
The probability of the trial’s showing benefit is low
Most relevant in testing new drug or strategy
Idea: Expensive (and not ethical?) to continue
Not relevant if study compares two treatments in common use
Goal: to get a good estimate of the difference
E.g., CATT (Lucentis vs. Avastin for AMD)

5. Types of futility “bounds”

6. Several choices for recommending stopping
If power, given data thus far, is low
Conditional power (CP) – look at a variety of alternatives
Predictive P – CP integrated over distribution of alternatives

7. Conditional power

8. Predictive probability

9. Some other definitions of futility
Current observed effect is essentially zero
If data rule out alternatives of interest
Emerson and Fleming – Biometrics 45:

10. But don’t just give a “rule”
Show how it is likely to play out in practice

11. DMC members are in for the long haul
In talking to DMCs, we often focus on early stopping
Harms or overwhelming benefit or “futility”
But DMCs don’t want to stop too early
They want the data to be convincing
Once a trial stops, restarting is difficult
On the other hand…
they don’t want to continue if balance of risk & benefit unfavorable

12. Study has no futility “bound” but the DMC recommends stopping

13. Example: chronic, uniformly fatal disease
Outcome: lung function
Last patient to be followed for 48 weeks
Stopping guidelines
No stated guidelines for safety
No interim analysis for efficacy or futility
DSMB was looking at data every three months

14. The DMC recommended stopping for futility
Fully enrolled: ~500 randomized – 250 active; 250 placebo
3 months more to go in trial
No difference in lung function
Active group had more non-serious GI adverse events
DSMB: recommend stopping – why make these people feel bad?
Sponsor was angry at the decision

15. Study has a futility “bound” but the DSMB doesn’t recommend stopping

16. VA HIT trial (HIT=HDL Intervention Trial)
Gemfibrozil vs. placebo in those with low HDL, normal LDL
Informal futility boundary
If conditional power is very low, recommend stopping
No difference for two years

17. What the DSMB saw for the first two years was essentially no difference. But the committee thought that the effect might take a while to become evident.

18. At the end of the trial, the effect was strong
At the end of the trial, the effect was strong. An early “futility” stop would not have identified this.

19. Dealing with subgroups, secondaries, and items of interest

20. Subgroups and secondary outcomes
What if overall the study is “futile” but
A subgroup of interest is showing benefit?
An important secondary is showing benefit
E.g., time to first exacerbation “futile”
But, treatment group has many fewer exacerbations
Often DMC will want to continue in such cases
Recommendation: talk to the Sponsor/Exec Comm up front

21. What if there is something “interesting”
Often the DMC will see something of interest
A small subgroup
A putative mechanism
Should it continue?
My general view is “no”
That’s asking for a different experiment

22. Conclusions – what you should plan
It is usually good to have a discussion of futility up front
It is often good to have a guideline for futility
But make sure the guideline is not abstract
Show examples of what would constitute a “stop”

23. In my experience, there is often less agreement in the DMC about futility than about safety or efficacy
DMC may recommend stopping for futility without a guideline
DMC may not stop even with a guideline
It sees promising results in an important subgroup
It feels that continuing likely to gain important knowledge
It is insecure about the data
It suspects there may be a lag in effect
Ultimately, the DMC (and everyone) wants a study to have a clear result which affects the decision to pull or not to pull the plug