1. The Use of Bayesian Basket Design in Early Phase Trials
Shiling Ruan, Matt Whiley
Novartis
JSM 2019

2. Single Investigational drug or drug combination (T)*
Basket Design
Description
Basket design is a type of master protocols designed to test a single investigational drug or drug combination in different populations defined by disease indication, stage, histology, genetic or biomarkers or other characteristics.
Schematic representation of a master protocol with a basket trial design
Single Investigational drug or drug combination (T)* D1 D2 D3 D4 D5
T= investigational drug; D = protocol defined subpopulation in multiple disease subtypes
[1] Master Protocols: Efficient Clinical Trial Design Strategies to Expedite Development of Cancer Drugs and Biologics, FDA Guidance for Industry 2018

3. A quick glimpse on the current usage
A search of “basket trial or basket design” in ClinicalTrials.gov returns 34 entries* and all of them are oncology phase I/II and phase II trials.
* search date: 6/11/2019
The search may be significantly underestimate the basket trials currently in use due to key word selection; in addition, basket trails are also used in non- oncology trials, such as the infectious disease and autoimmune disease
Basket trials are mostly used for exploratory purpose, but can also be used to support market application, for example:
Keytruda expanded indications to treat  microsatellite instability-high cancer based only on genetic abnormality (approval: May 2017)
Vitrakvi by Loxo Oncology for solid tumor patients with TRK+ genetic mutation, regardless of where in the body the tumors originated; this is the first new drug approval supported by a basket design (approval: Nov 2018)
Number of trials
(Recruiting/Active but not recruiting/and not yet recruiting)
Phase I/II, II 34
Phase III and IV
Keytruda: accelerated approval for the treatment of adult and pediatric patients with unresectable or metastatic solid tumors that have been identified as having MSI-H or dMMR (deficient DNA mismatch repair). This is a historical approval as the first time a cancer treatment is approved based on a common biomarker rather than the anatomic location in the body where the tumor originated. Phase II, N=58 patients. Patients from four other single arm open labels studies were pooled into the overall efficacy assessment.
Vitrakvi: three multi center, single arm open label study, rare pediatric drug with accelerated approval - efficacy assessment N=55, N of tumor type 9, simon’s two stage design was used.

4. A typical phase I/II Study incorporated basket design in early phase
Dose Escalation
(Phase I/Ib)
Identification of recommended phase II dose (RP2D)
Dose Expansion
(Phase II) D1 D2 D3
... Dn
Dose 1
Dose 2
.....
MTD/RP2D
Basket designs
Basket design is used after the recommended phase II dose for the investigational drug(s) has been established
Under special scenarios a dose-finding phase may be incorporated within the basket designs, i.e., pediatric population when sufficient adult data are available to inform a starting dose [1]

5. Motivation
Strong interest to explore multiple subpopulation simultaneously in a single study in early phase trials (especially in IO or target therapy)
The need to use the data more efficiently since sample size are often quite limited
Ethical consideration: allow interim decisions to stop for futility if no or very low efficacy signals observed for a particular subpopulation
Flexible designs - Bayesian, or adaptive, or both.

6. Bayesian adaptive basket trials
Bayesian Adaptive basket trials are flexible and efficient: range of design elements can be implemented
There is a trade-off between efficiency and complexity
The complexity brings challenges; many questions to be considered in terms of:
study design
statistical modeling and analysis
Operational considerations

7. The design questions
Is the study population heterogeneous or are there different baseline prognostic factors for which we need to allow for differences in treatment effects?
Is it the drug targeting a specific mutation across multiple disease types?
Is it a therapy with more general mode of action such as immunotherapy, which might be efficacious across several different disease types?
What is the study objective?
Want to know if the drug works in any disease type?
Does the drug effect differs by disease type?
What other information are available? For example,
Patient prevalence may differ for the different subpopulations. Do we require equal recruitment, or do we skew in favor of more common groups?
Maximum sample size allowed?

8. Bayesian adaptive design
Set up: Let 𝑌 𝑖𝑗 be the response for patient i in tumor type 𝑇 𝑗 , and 𝜋 𝑗 =Pr( 𝑌 𝑗𝑖 =1| 𝑇 𝑗 ), be the underlying probability of response in tumor type 𝑗. Let 𝑌 𝑗 be the number of responders in in tumor type 𝑇 𝑗 with 𝑛 𝑗 patients. Then 𝑌 𝑗 follows a binomial distribution:
𝑌 𝑗 ~𝐵𝑖𝑛( 𝑛 𝑗 , 𝜋 𝑗 )
The response rate 𝜋 𝑗 for each basket could be modeled
independently if the goal of the study is to determine the effect of the drug separately
via hierarchical models, such as mixture models or Dirichlet process
via aggregation (pooling)
Cunanan K. et. al. (2018) simulated the statistical property of various Bayesian models in terms of power, sample size, family wise error rate (FWER) and recommend simple mixture model as compared to pooling and complicated hierarchical models

9. Exchangeability Assumptions
Common mean μ
Between strata variability 2 π π1 π2 π3 π1 π2 π3
Exchangeable basket
‘Similar’ response rate
Hierarchical model allows sharing of information
Independent basket
Each basket has independent response rate 𝜋 𝑗
‘Pooled’
Common response rate

10. Modeling of treatment effects
The exchangeability of the treatment effects are often defined by
Assuming the response rate of the basket 𝜋 𝑗 are exchangeable with that of other baskets
Appropriate if we expect to see similar response rates in some or all indications
Assuming log odd ratios of the response rate of the basket are exchangeable with that of other baskets
More suitable for examples where indications have different historical rates
Hypothesis is to see similar improvement over historical rates

11. Timing of the interim analysis
Indication A
Indication B
Indication C
Indication D
Indication E
Indication F
Indication G
Indication H
1. Trigger an interim as each strata reaches N=n1
With flexibility to shift decision time points based on projected enrolment
2. Trigger initial interim when first strata reaches N=n1
Subsequent interims held periodically (e.g. quarterly) and could be adjusted based on enrollment
Business Use Only

12. Decision rules Interim decision rules: Final analysis
What is the threshold (C1) for each basket such as that if p(ORR> C1) < pf this is not considered interesting for further clinical investigation?
Once a cohort met futility rule at one interim analysis, will it still have opportunity to continue?
Will early stopping for efficacy be considered?
Is the futility rule binding or non-binding?
Final analysis
What is the threshold (C2) for each basket such that the p(ORR>C2) > ps so that it might be a good candidate for further development, i.e., support of POC

13. Assess the operating Characteristics
Simulation is often necessary to assess the operating characteristics of the Bayesian design
What are the marginal type I error and power for each basket?
What is the expected trial size?
What is the family wise error rate?
Often only the nominal false positive rate are controlled within each basket; the overall false positive rate may not be so strictly controlled but needs to be understood
For early phase trial, the trade off between the benefit of finding an active new drug quickly and cost-effectively and the risk of wrong tumor selection should be evaluated carefully (Chen et. al. 2017)

14. A Case Example: A Bayesian Adaptive Basket Design
Paused enrollment if N>=8 and met futility criterion
Stop if met futility rule after enrollment pause D1
At least one cohort reaches N=8 Y D2 Y
Interim Analysis:
Met futility rule?
Subsequent IAs: met futility rule? D3 N D4 N
𝒑 𝑶𝑹𝑹>𝑪𝟏|𝒅𝒂𝒕𝒂 <𝟒𝟎%
Continue Accrual
Continue Accrual
Delayed D5 D6
Final Analysis at max sample size
Additional baskets may not all enroll at the same time
maximum sample size=80; max per basket = 20
𝒑 𝑶𝑹𝑹>𝑪𝟏|𝒅𝒂𝒕𝒂 ≥𝟗𝟎%∗ 𝑳𝒊𝒌𝒆𝒍𝒚 𝑪𝒍𝒊𝒏𝒊𝒄𝒂𝒍𝒍𝒚 𝒊𝒏𝒕𝒆𝒓𝒆𝒔𝒕𝒊𝒏𝒈
𝒑 𝑶𝑹𝑹≥𝑪𝟐|𝒅𝒂𝒕𝒂 ≥𝟓𝟎%∗ 𝑴𝒐𝒅𝒆𝒓𝒂𝒕𝒆 𝒆𝒗𝒊𝒅𝒆𝒏𝒄𝒆 𝒐𝒇 𝒄𝒍𝒊𝒏𝒊𝒄𝒂𝒍𝒍𝒚 𝒎𝒆𝒂𝒏𝒊𝒏𝒈𝒇𝒖𝒍 𝒆𝒇𝒇𝒆𝒄𝒕
𝒑 𝑶𝑹𝑹≥𝑪𝟐|𝒅𝒂𝒕𝒂 ≥𝟗𝟎% ∗𝑺𝒕𝒓𝒐𝒏𝒈 𝒆𝒗𝒊𝒅𝒆𝒏𝒄𝒆 𝒐𝒇 𝒄𝒍𝒊𝒏𝒄𝒂𝒍𝒍𝒚 𝒎𝒆𝒂𝒏𝒊𝒏𝒈𝒇𝒖𝒍 𝒆𝒇𝒇𝒆𝒄𝒕
Response rates:
C1: Clinically not interesting response rate
C2: Historical control rate

15. EXNEX model
Exchangeable non-Exchangeable model (Neuenschwander et. al. 2016)
A hierarchical mixture model with three components – a basket will have probably of 𝑝 𝑘 belong to a certain clusters of baskets (EXNEX)
With probability 𝑝 1
With probability 𝑝 2
With probability 1- 𝑝 1 −𝑝 2

16. Model Specification
Define the log odds ratio be as 𝜃 𝑗 = log 𝜋 𝑗 1− 𝜋 𝑗 − log 𝐶 1𝑗 1− 𝐶 1𝑗 , then
𝜃 𝑗 ~ 𝑝 1 𝑁( 𝜇 1 , 𝜏 1 2 ) + 𝑝 2 𝑁 𝜇 2 , 𝜏 𝑝 3 𝑁 𝑚 𝑤 , 𝑣 𝑤 2 , 𝑝 3 =1− 𝑝 1 − 𝑝 2
The two exchangeable groups have means
𝜇 1 ~ N(0, 32), 𝜇 2 ~ N(log(2), 32)
The heterogeneity between strata within group is modeled through 𝜏
𝜏 1 2 ~ Gamma(3, 5), 𝜏 2 2 ~ Gamma(3, 5),
- moderate between strata heterogeneity (median=0.73, 95% interval= (0.35,1.20)) (Friede et al,2015).
For the non-exchangeable group, weakly informative prior are used:
𝑚 𝑤 , 𝑣 𝑤 2 ~ (log(2), 32)

17. Operating Characteristics
Scenario 1: the true ORR for all tumor types are clinically uninteresting: log (odds ratio) =0
FWR: 21.4%, false positive rate for each disease type is ≤ 0.05, > 50% prob to pause for futility for all disease types
High prob. of failing to show evidence of likely or strong clinically meaningful effect
*Based on 1000 simulations, limit N to 80 max and 20 for each disease type)

18. Scenario 2: : the true ORR for all tumor types are likely clinically interesting
Log(odd ratio) with respect to strong clinically meaningful effect for D1
The prob to pause for futility for all disease types < 10%
The prob of declaring evidence for likely clinical interesting effect is >70% for most disease types (lower for the disease type with small sample size)
The prob of declaring strong evidence of clinically meaningful effect is small

19. Scenario 3: One tumor type (D1) has strong evidence to be clinically meaningful and D2 has likely clinical meaningful effect. Others are not interesting.
The prob to pause for futility for D1 and D2 are smaller (< 10%) than that for others
The prob of declaring evidence for likely clinical interesting effect for D1 and D2 are higher than that for others
The prob of declaring strong evidence of clinically meaningful effect for D1 is higher than that for D2 and others

20. Concluding Remarks
Basket designs are very efficient in exploring multiple tumor type/disease type with or without targeted mutation in one study, which are very useful in some early phase studies
These designs can be very flexible and adaptive; it is possible to incorporate many design features and may be used to address multiple questions of interest
They are challenging designs. There is a trade off between complexity and efficiency, which should be evaluated prospectively in the planning stage.
It is important to understand the potential issues of the basket designs, especially take cautions to interpret the results

21. References
FDA Guidance for Industry (2018) Master Protocols: Efficient Clinical Trial Design Strategies to Expedite Development of Cancer Drugs and Biologics
Chugh R, Wathen JK, et al (2009). Phase II multicenter trial of imatinib in 10 histologic subtypes of sarcoma using a bayesian hierarchical statistical model. J Clin Oncol. 1:27(19):
Friede T, Röver C, Wandel S, Neuenschwander B (2017). Meta-analysis of few small studies in orphan diseases. Research Synthesis Methods. 8,
Thall PF, Wathen JK, et al, (2003). Hierarchical Bayesian approaches to phase II trials in diseases with multiple subtypes. Statist. Med. 2003; 22:763–780.
Neuenschwander B, Roychoudhury S, Schmidli H (2016). On the Use of Co- Data in Clinical Trials. Statistics in Biopharmaceutical Research. 8:3,
Woodcock J, LaVange, L. (2017) Master Protocols to Study Multiple Therapies, Multiple Diseases, or Both. The New England Journal of Medicine, 2017, 337:62-70.
Yan L, Zhang W. (2018) Precision medicine becomes reality—tumor type- agnostic therapy, Cancer Communications, 38:6
Neuenschwander B, Wandel S., Roychoudhury S, Bailey S (2016). Robust exchangeability designs for early phase clinical trials with multiple strata. Pharmaceutical Statistics,15:2 ( )

22. References
Cunanan K., Lasonos A, Shen R, Begg C, Gönen M, (2018) Evaluating the Statistical Properties of Bayesian Basket Trial Designs, ICSA presentation
Chen C, Deng Q, He L, Mehrotra D, Rubin EH, Beckman RA. (2017) How many tumor indications should be initially studied in clinical development of next generation immunotherapies? Contemporary Clinical Trials 59:
Cunanan K, Iasonos A, Shen R, et al. (2017)  An efficient basket trial design. Statistics in Medicine, 36:10,
Leblanc M, Rankin C, Crowley J. (2009) Multiple histology phase II trials. Clin Cancer Res. 15:4256–4262. 
Berry SM, Broglio KR, Groshen S, et al. (2013) Bayesian hierarchical modeling of patient subpopulations: Efficient designs of phase II oncology clinical trials. Clin Trials. 10:720– 734. 
Simon R, Geyer S, Subramanian J, et al. (2016) The Bayesian basket design for genomic variant-driven phase II trials. Semin Oncol. 2016;43:13–18. 
Cunanan KM, Gonen M, Shen R, et al. (2017) Basket Trials in Oncology: A Trade-Off Between Complexity and Efficiency. Journal of Clinical Oncology, Vol 35,,
Cunanan K, Iasonos A, Shen R, et al. (2017) Specifying the True- and False-Positive Rates in Basket Trials. Commentary, JCO precison oncology, published online November 3, 2017