1. HOW FAR SHOULD A DMC STRAY FROM THE PLANNED MONITORING APPROACH?
Susan S. Ellenberg, Ph.D.
University of Pennsylvania
SCT/ICTMC
Liverpool, UK
May 9, 2017

2. STATISTICAL MONITORING APPROACHES
Will focus on sequential designs with possibility of early stopping
Adaptive designs allowing mid-course sample size increases based on interim comparisons raise different issues

3. BASIS FOR BOUNDARIES Primary reason for boundaries
Conclude study if one treatment is clearly superior to comparator
Not ethical to continue
Desire to make results widely known
Often no specific boundary for safety outcomes
Specific safety concerns not always known
When efficacy outcome is mortality, safety and efficacy outcome is the same

4. GUIDELINES, NOT RULES
Sequential designs were never intended to serve as rigid rules
They are tools to aid decision-making
If a boundary is crossed but the study is not terminated that does not inflate Type 1 error—no statistical penalty
Nevertheless, designs should reflect agreement among trial investigators, sponsors and DMC as to reasonable criteria for early termination

5. BALANCING CURRENT VS FUTURE PATIENTS
If trials are stopped too early they may not be sufficiently persuasive, so may fail to accomplish purpose; is it ethical to enter patients in such a trial?
Is it ethical to continue to randomize to a trial long after a difference has been clearly established?
This balance is ultimately a judgment call— and judgments will differ
Drug Information Association

6. ONE PERSPECTIVE
Wheatley and Clayton described an interesting DMC decision process*
Interim data grazed boundary at an early analysis, crossed boundary at next analysis
Interim effect estimate seemed implausible to DMC, who recommended continuation
By the end of the study, all evidence of a treatment effect had disappeared
Wheatley and Clayton described this experience as exemplifying the good judgment of this DMC
*Wheatley and Clayton, Clinical Trials, 2003

7. RESPONSE TO ARTICLE
Some questioned whether a DMC should ignore crossing of a monitoring boundary on this basis*
No emerging safety issue, or inconsistency with interim results of secondary outcomes
Maybe boundaries should have been more conservative to begin with—interim values that would cross boundary could have been calculated
What if the trend observed at the interim analysis had continued, with the value crossing the boundary by a lot at the next look? (Note: primary outcome was survival)
Was the DMC wise, or lucky, in this case?
*Whitehead; Korn, Friedlin, George; Clinical Trials 2004

8. ANOTHER EXAMPLE
NIAID-sponsored trial of treatment for infants with HIV conducted in southern Africa
Trial question: when to start antiretroviral therapy, and how long to treat
Primary endpoint: death or failure of therapy (based on CD4 levels, clinical outcomes or treatment toxicity)
DSMB was geographically diverse, with representation from US, UK, South Africa, Botswana, Ghana, Zimbabwe

9. STUDY DESIGN Three-arm design
Begin treatment immediately following diagnosis and continue for 40 weeks
Begin treatment immediately following diagnosis and continue for 96 weeks
Defer treatment until child qualified according to national guidelines (low CD4 count or emergence of symptoms)
Current standard of care

10. STANDARD OF CARE: SOUTH AFRICA
Control arm reflected international guidelines at that time
In developed countries, movement toward immediate treatment for all infected infants
Modifying guideline to begin treatment earlier would have huge economic implications
DSMB was told that extremely strong findings would be required to persuade national authorities to revise guidelines because of drug costs
Investigators proposed very stringent monitoring plan

11. EARLY STOPPING PLAN
The study protocol specified a stringent Haybittle- Peto early stopping plan to the DSMB
Require a treatment effect for either active arm significant at the level at any interim analysis
DSMB was not fully comfortable with this plan
Acknowledged need for more data to have impact on national treatment policies
Shared emerging concerns about leaving HIV-infants untreated
DSMB proposed O’Brien-Fleming design
Difficult to stop very early, but easier as more data accumulate
At any point after the first or second look, would be more permissive of early stopping

13. TRIAL INITIATED DSMB did not insist on any change in proposed design
Accepted Haybittle-Peto monitoring plan (with some reluctance), recognizing that if results showed need for immediate treatment, they would have to be very strong to lead to change in practice
Became increasingly uncomfortable as early interim results trended strongly toward improved mortality with early treatment

14. ONE YEAR LATER Third full board review
At this meeting the level was not yet reached
Deferred vs 40 weeks: p=0.008
Deferred vs 92 weeks: p=0.01
DSMB requested comparison of the delayed treatment arm with the two immediate treatment arms combined
This was not a pre-specified analysis
Comparison of immediate vs delayed treatment: p<0.001

15. RECOMMENDATION Pre-specified stopping boundary not crossed
Nevertheless, DSMB was not comfortable recommending that the study continue
DSMB recommendation
Terminate untreated arm
Refer all infants in that arm to clinic for re- evaluation and consideration of treatment
Continue study of 2 immediate treatment arms, comparing duration of treatment
Recommendation accepted by study team and by NIAID

16. RESULTS NEJM 359:2233-44, 2008 Outcome Early ARV (N=252)
Deferred ARV (N=125)
Total (N=377)
Hazard Ratio
Death
10 (4%)
20 (16%)
30 (8%)
0.24 (0.11, 0.51)
Non-fatal treatment failure 1 2
Total endpoints 11 21 32
0.25 (0.12, 0.51)

17. CONSEQUENCES
Within a year of recommendation, international treatment guidelines were changed
Strong recommendation for initiating antiretroviral treatment for all HIV-infected children under 12 months of age regardless of clinical symptoms or immune status
Was the DSMB right in this case?
What if study team had rejected recommendation?
What if data had not persuaded policy makers to change guidelines?

18. RECENT ARGUMENTS
A group of epidemiologists has published a series of papers complaining about early stopping in clinical trials*
Concern that trials that stopped early could substantially overestimate treatment effect and skew meta-analyses
Most recent paper argues that since many trials stopped early are followed by additional trials of same question, we need more restrictive early stopping criteria**
*Bassler et al, J Clin Epid 2008; Bassler et al, JAMA 2010; Guyatt et al, BMJ 2012
**Murad et al, J Clin Epid 2017

19. COUNTER-ARGUMENTS Others have shown that Ethical arguments
The magnitude of overestimation is minimal in most cases*
There are statistical procedures that can adjust the estimate for this problem (although no one uses them)**
Ethical arguments
when outcome is serious, precision of estimation of the effect size is not what is most important
Delay in making community aware of superior treatment is problematic
*Friedlin and Korn, Clin Trials, 2009; Wang et al, Clin Trials 2016
**Hughes and Pocock, Stat Med, 1988; Emerson and Fleming, Biometrika 1990; Pinheiro and DeMets, Biometrika 1997; Fan et al, J Biopharm Stat 2004

20. BOTTOM LINE
Critical for DMC and study team to be in agreement about criteria for early termination based on interim efficacy findings
DMC must carefully consider monitoring plan proposed by study team; should engage team in discussion if concerns
Ultimately, DSMB will have to make its own best judgment