Application of adaptive designs in clinical trials research Munya Dimairo Research Fellow in Medical Statistics University of Sheffield, UK m.dimairo@sheffield.ac.uk / mdimairo@gmail.com Twitter: @mdimairo CREDO Ethiopia 12-13 July 2017

Declarations Employed by the University of Sheffield Lead investigator of an Adaptive designs CONSORT Extension (ACE) project Funded by NIHR and MRC HTMR No conflict of Interest to declare

Objectives Highlight some limitations of fixed sample size RCTs How we can overcome some of the limitations concept of adaptive designs some trial adaptation opportunities with case studies (prospective and retrospective) Highlight some considerations – nothing is for free Questions and discussion (small group exercise)

Some limitations of fixed sample size RCTs Implications of poor ‘success’ rates of investigative treatments research waste (time and resources) ethics Inefficient in some cases inaccurate design assumptions (Charles et al., 2009; Clark et al., 2013; Vickers, 2003) multiple competing treatments (grant application review experiences) addressing some research questions robustly (population enrichment example) evaluation of new treatments is time consuming Ethical issues/ urgency in decision-making process outbreaks or emergency care cases exposure to ineffective treatments (interests of the patient vs society)

How can we do things differently? Adaptive designs … ‘… it’s like taking insurance against unforeseeable events’ “… provides pre-planned opportunities to use accumulating trial data to modify aspects of an ongoing trial while preserving its validity and integrity” what is validity? (robust inference) what is integrity? (convincing stakeholders) why pre-planning and what does it mean? (credibility and adequate design evaluation)

Adaptation: sample size re-estimation (1) Implications Median (IQR) of 2271 (2190 to 2298) vs 3130 planned Median (IQR) overestimation of 861 (835 to 942) Funding ran out after 2243 participants Funders declined funding extension request for reasons Study had already addressed questions Unnecessary recruitment could have happened Sample size re-estimation could have provided safeguards No need to go back to the funder Often inaccurate design assumptions either over estimate or under estimate the required sample size Need to validate design assumptions and react accordingly RATPAC retrospective case study (Goodacre et al., 2011) evaluation of a point-of-care cardiac marker panel in patients presenting to the ED with suspected MI design assumptions 50% standard care hospital discharge rate 5% absolute increase to declare superiority 80% power and 5% type I error planned sample size N=3130 (1565 per arm)

Adaptation: sample size re-estimation (2) CARISA prospective case study (Chaitman et al., 2004) Investigated two doses of ranolazine (750mg or 1000mg) against placebo on exercising capacity of patients with severe chronic angina Primary endpoint: treadmill exercise duration at trough (12 hrs after dosage) Design assumptions yielded 462 patients to achieve a 90% power increased to 577 to account for potential dropouts What happened during the trial Blinded sample size re-estimation after 231 patients (~50% of the planned sample size) Variability found to be markedly higher than assumed Sample size increased by 40% to 810 to preserve 90% power 823 were recruited Statistical analysis used for fixed sample size designed RCT appropriate following a blinded sample size re-estimation significant clinical improvements in exercise duration were found for both ranolazine doses

Adaptation: early trial stopping – group sequential (1) Desire to stop the trial early as soon as we have enough evidence about investigative treatment regarding: Beneficial effects (effectiveness/efficacy or futility) Safety Why bother? Accelerates decision-making process, especially in critical care or outbreaks Accelerates evaluation of new therapies Effective treatment are made accessible to patients quicker Fewer patients are exposed to potentially unsafe treatments Potential savings in research time and resources Back to RATPAC retrospective case study Emergency care, primary endpoint observed early, and huge sample size planned Redesign features 2 interim analysis at 50% and 70% of the planned sample size Options to stop early for efficacy or futility Stringent/aggressive stopping rule (stop when there is overwhelming evidence at interim) Results Could have stopped early at 50% of planned sample size in favour of PoC treatment Stagewise adjusted results to account for interim analysis and early stopping are consistent with the observed Implications Patients, time and resources Scenario Sample size Proportion of sample size used Patients savings Reduction in recruitment duration (months) Planned fixed sample size 3130 53.5% 1456 10.3 Planned GSD 3348 50.0% 1674 11.8 Achieved recruitment 2263 74.0% 589 4.2

Adaptation: early trial stopping – group sequential (2) 3CPO retrospective case study (Gray et al., 2008) To determine whether noninvasive ventilation (CPAP or NIPPV) against standard of care reduces mortality (within 7 days) in the treatment of patients with acute cardiogenic pulmonary edema Redesign features 80% power, 15% mortality rate in SOC, 6% reduction, 5% type 1 error 2 interim analyses at 50% and 65% Stop for futility only at the 1st interim analysis Stop for either futility or efficacy at the 2nd interim analysis Stringent LD (OBF) stopping rules Results Could have stopped for futility at 65% After 100 000 trial simulations, the: chances of failing to reject H0 if trend continued this way = 99.7% chances of showing superiority in favour of CPAP or NPPV assuming an overwhelming benefits of 6% difference for the remaining 35% of the data = 15.4% Stagewise adjusted results to account for interim analyses and early stopping are consistent with the observed

Adaptation: multi-arm multi-stage (1) Rationale To investigate multiple competing interventions in a single trial To save patients, resources and time in the long run compared to a sequence of two-arm parallel group trials Accelerate decision-making process Number of variations of the designs Either phase 2 or 3 only Phase 2/3 combined in one trials (aka seamless adaptive design – see next slide) TAILoR prospective case study (Pushpakom et al., 2015) Investigates 3 doses of telmisartan (20mg, 40mg, and 80mg) against control in reducing insulin resistance in HIV patients on combination ART Primary endpoint – change in 24 weeks insulin resistance from baseline Planned with one interim analysis Phase 2 dose-selection (treatment selection) Interim results and implemented trial adaptations Interim analysis conducted at 50% of the planned 336 patients 20mg and 40mg were dropped for futility Malawi ongoing case study (Mr Augustine Choko) Welcome Trust Funded A Phase II adaptive multi-arm multi-stage cluster randomised trial randomising antenatal clinic days to six different trial arms. Pregnant women accessing ANC in urban Malawi for the first time will be recruited into either the standard of care arm (invitation letter to the male partner offering HIV testing) or one of five intervention arms offering oral HIV self-test kits.

Adaptation: multi-arm multi-stage (2) Seamless 2/3 adaptive design Either operational or inferential in nature Comparator is phase depended (new comparator introduced) Options to drop arms in phase 2 if unsafe or ineffective Options to drop arms in phase 3 if ineffective Hypothetical scheme

Adaptation: response-adaptive randomisation Rationale Allocate more patients to promising treatment during the trial Balancing the interests of patients within the trial and society Most appealing for critical care/outbreaks (such as oncology, ebola, etc) RAR prospective case study (Giles et al., 2003) To assess troxacitabine-based regimes as induction therapy in patients aged ≥50 years with untreated, adverse karyotype, acute myeloid leukemia Bayesian response-adaptive randomisation Started with equal randomisation; IA (1): TA(1): TI (1) Allocation probability to IA (usual care) was held constant (2/3) if 3 arms are in the trial Primary endpoint: complete remission that occurred within 49 days of starting treatment P(randomising to TI) ~ 7% after 24 patients (TI dropped) P(randomising to TA) <4% after 34 patients (TA stopped) Study stopped after 34 patients (<50% of the planned 75) Final success (complete remission) rates: 10/18 (55%) for the IA arm 3/11 (27%) for the TA arm 0/5 (0%) for the TI arm 70% probability that TA was inferior to IA Only 5% probability that TA would have a 20% higher complete remission rate than IA

Conclusions Well conducted adaptive designs can address shortcomings of fixed sample designed RCTs Nothing is for free so there are challenges depending on trial adaptations considered – adequate planning and careful consideration is required Not all potential trial adaptations have been considered here .. . It’s often a good idea to take insurance against unforeseeable events when conducting trials …

References Chaitman et al. (2004), “Effects of Ranolazine With Atenolol, Amlodipine, or Diltiazem on Exercise Tolerance and Angina Frequency in Patients With Severe Chronic Angina: A Randomized Controlled Trial”, JAMA, American Medical Association, Vol. 291 No. 3, p. 309. Charles et al (2009), “Reporting of sample size calculation in randomised controlled trials: review.”, BMJ (Clinical Research Ed.), Vol. 338 No. may12_1, p. b1732. Clark et al (2013), “Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review.”, BMJ (Clinical Research Ed.), Vol. 346 No. mar21_1, p. f1135. Giles et al. (2003), “Adaptive Randomized Study of Idarubicin and Cytarabine Versus Troxacitabine and Cytarabine Versus Troxacitabine and Idarubicin in Untreated Patients 50 Years or Older With Adverse Karyotype Acute Myeloid Leukemia”, Journal of Clinical Oncology, Vol. 21 No. 9, pp. 1722–1727. Goodacre et al (2011), “The Randomised Assessment of Treatment using Panel Assay of Cardiac Markers (RATPAC) trial: a randomised controlled trial of point-of-care cardiac markers in the emergency department.”, Heart (British Cardiac Society), Vol. 97 No. 3, pp. 190– 6. Gray et al (2008), “Noninvasive ventilation in acute cardiogenic pulmonary edema.”, The New England Journal of Medicine, Vol. 359 No. 2, pp. 142–51. Pushpakom et al. (2015), “Telmisartan and Insulin Resistance in HIV (TAILoR): protocol for a dose-ranging phase II randomised open- labelled trial of telmisartan as a strategy for the reduction of insulin resistance in HIV-positive individuals on combination antiretroviral therapy.”, BMJ Open, BMJ Publishing Group, Vol. 5 No. 10, p. e009566. Vickers (2003), “Underpowering in randomized trials reporting a sample size calculation”, Journal of Clinical Epidemiology, Vol. 56 No. 8, pp. 717–720.

Acknowledgement Philip Pallmann on behalf of MRC HTMR ADWG

Questions Discussion task …. OR Think about a research problem you may have which could be addressed by an adaptive trial and discuss the following within your group: What trial adaptations could be considered or you are interested in considering? What motivates you to consider such adaptations? What is the primary endpoint and is it suitable for an adaptive design? What are the other practical considerations? OR  A research problem you have conducted using a different trial design but given what you know now it could have been adaptive and discuss: What trial adaptations could have been considered and motivations behind those adaptations?