Alexia Iasonos and John O’Quigley Phase I designs that allow for uncertainty in the attribution of adverse events Alexia Iasonos and John O’Quigley Phase I trials primary endpoint: rate of Dose Limiting Toxicities (DLT) and estimating the Maximum Tolerated Dose (MTD) Background: The consequences of Errors in toxicity attribution have not been studied in the statistical literature Assumption is that the DLT outcome is measured without error Type A error: leads to overestimation of MTD Type B error: leads to underestimation of MTD Type B has greater influence than Type A error because we can recover from Type A errors 3+3 is sensitive to Type B errors because of the stopping rule of 2 DLTs Observed True DLT yes True DLT no DLT Type B Flag non DLT Non DLT Type A Miss DLTs ORAL PRESENTATION Your oral presentation is scheduled for Session #215802 on 7/30/2018, beginning at 2:00:00 PM. The session will consist of up to 20 talks of 4 minutes each. Please know you should be prepared with 5 or fewer slides for this portion of the presentation. To ease the transition between talks, speed session presenters are required to upload the slides for the oral portion of their presentations in advance or onsite in the Speaker Management Room or at a kiosk. More information about the speaker management system will be sent to you closer to the conference.

How does the Continual Reassessment Method recover from errors? True DLT rates are scenario 1: 0.22 and 0.3 at d3 and d4 Iasonos et al Clin Cancer Research 2012

Design: use scores as opposed to binary outcome When attribution is certain, Y=Z, the assigned score is 0 or 1; sj When attribution is uncertain, clinicians assign a score between 0-1 quantifying the probability that the observed toxicity is drug related For an observed DLT, the clinician makes a determination whether this is a true DLT (score=1) or what is the likelihood that this is indeed drug related ( score 0-1) Scores can be point or intervals Data are dose level xj, and score sj for patient j. Estimating equation uses (xj, sj), zj We want to infer Pr(Y=1| Z=1), Y: true DLT; Z: observed DLT when there is a true DLT we assume Z=1, ie Pr(Z=0|Y=1)=0).

Model parameter estimation Assume for simplicity that A) P(Z=0|Y=1) =0 or P(Z=1|Y=1) =1 ie when there is a true DLT we see it. We only allow Type B errors ie not sure whether an observed DLT is a true DLT We observe the dose levels xj and the outcomes; either zj =0, in which case we shall take it that the unobserved yj =0; ie type A error =0 or zj =1, in which case the clinician is asked to provide further data in the form of sj, where 0<sj <1 is the clinician’s assessment of the probability that the toxicity is drug related. Our observations now look like a collection of pairs .xj, sj/ where for each patient xj is the dose level and 0<sj<1 is the probability that is assigned by the clinician to represent the uncertainty in toxicity attribution for patient j. In equation (6), we could make a notational economy by allowing sj to assume the value 0 whenever zj takes the value 0. psi is a model working model. Integral is in region R,asking the clinician to provide it for interval score

Results Iasonos A, J O’Quigley J Royal Stat Society Ser C 2017 Large sample properties: Well calibrated scores ensure convergence of the dose toxicity parameter to the true value under the conditions in Shen and O’Quigley 1996 and the condition that scores are on average correct. Small sample behavior: If scores are right on average, the method provides large improvements over a method that ignores errors. Simulation study: If they are wrong, how wrong can they be while the method works acceptably well in terms of finding the true MTD? Evaluated: point, interval scores, dose dependent and dose independent; systematically biased scores

Current attribution description Current designs Proposed Design Hypothetical Scores Unrelated The AE is clearly not related to the intervention No Unlikely doubtfully (0, 0.30) Possible may be related No or Yes (most often) [0.30, 0.65) Probable likely related Yes [0.65, 1) Definite The AE is clearly related to the intervention 1 Logistical Aspects This novel conceptual framework incorporates the subjectivity involved in toxicity attribution into Phase I designs, such that the final recommended dose reflects this uncertainty. The scores can describe numerically existing verbal descriptors. These descriptors will map onto separate scores or a group of scores. It also allows the investigators to strictly adhere to the protocol by recording all DLTs and, yet, still allow expert opinion to prevent Adverse events that are most likely not drug related from seriously compromising the exercise of identifying the correct MTD. Common Terminology Criteria for Adverse Events (CTCAE) version 4.0

REFERENCES – RESEARCH PROGRAM Conclusions Significant improvements in the accuracy of finding the MTD compared to ignoring errors Include knowledge from earlier studies together with the clinical investigators’ expertise. Quantifying the uncertainty in attribution: In some cases, the investigators may be close to certain on correct attribution In other cases, they will be less certain and we explicitly incorporate this The investigators' input tailored to the individual patients can enable more accurate dose finding and greater flexibility in escalation while still maintaining a rigorous adhesion to protocol definitions. REFERENCES – RESEARCH PROGRAM 1. Iasonos A, J O’Quigley J R Stat Soc Ser C 2017 2. Iasonos et al Clin Cancer Research 2012 3. Hyman DM JCO 2014 (3104 pts, 127 trials); Ballman K, JCO. Editorial. Funding: The Translational and Integrative Medicine Research Fund at Memorial Sloan Kettering Cancer Center. R Code available at: https://rss.onlinelibrary.wiley.com/doi/10.1111/rssc.12195

iasonosa@mskcc.org john.oquigley@upmc.fr “I think that the paper will make an excellent contribution to the journal” iasonosa@mskcc.org john.oquigley@upmc.fr

Sensitivity Analysis: how wrong can the assigned scores be? Robustness If the scores are well calibrated, on average we expect patients with high probabilities of DLTs to correlate with an actual presence of DLT and similarly low scores to correlate with absence of DLT. If the clinicians' scores have no systematic variability and on average agree with the presence of DLTs, then the proposed method is more accurate than using a binary outcome with misclassification errors. Systematic biases however, as expected, will lead to under estimation or overestimation of the MTD when bias here is defined as systematically assigning low scores to true DLT or very high scores to non DLT's throughout the trial. The scores can be inexact for every patient but the method's accuracy will not be unduly affected as long as the scores are well calibrated and not systematically biased.