Use of Piecewise Weighted Log-Rank Test for Trials with Delayed Effect Boguang Zhen, Ph.D. Therapeutics Evaluation Branch Division of Biostatistics CBER/FDA Disclaimer: My contributions are an informal communication and represent my own best judgment. These comments do not bind or obligate FDA

Outline Background Concerns Evaluations/Discussions Concluding Remark

Delayed Effect (DE) In chemotherapy, study drugs act directly on cancer In therapeutic cancer vaccine therapy considerable time might be needed for cancer vaccines to induce immunity similar to many other immunotherapies, The impact of the so delayed effect (or lag time) should be considered

Sipuleucel-T case: 6 months~ DE NEJM: 363 (5): 418, 2010

Issues on Delayed Effect For time-to-event endpoint Regular log-rank test commonly used Under non-proportional hazards setting Ignore it or increase sample size Lose power or increase cost/duration Piecewise weighted log-rank test (PWLRT) Allocate different weight to a subset of events Reduce sample size Increase power time-to-event endpoint – motivated by therapeutic vaccine cancer trials with OS endpoint, but the concern could be in any trial that deals with time-to-endpoint with delayed effect, , e.g. cardio-vascular trials. the regular log-rank test with equal weight across all time points has been commonly used to compare the difference of the endpoint between the experimental group and the control group

PWLRT not widely used Challenging and Opportunity! Why not free lunch? Lack of awareness Issues Allocate different weight to a subset of events Could specify zero weight for some events - implies that some subjects would not contribute ‘meaningful’ information to the test result May not be straightforward to interpret the analysis results based on the weighted methods … Challenging and Opportunity!

Concerns Type I probability Intention-to-treat principle Study results interpretation Changing point specification Crossover effect Others? -- evaluate and address the above 5 concerns

Evaluations/Discussions Use the PWLRT method in Xu’s paper (Stat in Med 2017; 36(4):592-605) for evaluation for simplicity t* denote the hazard ratio changing point For simplicity, I only use the PWLRT method as proposed by Xu et al. for the evaluation, but the results can also be generalized to other weighted methods that deal with delayed effect. Inject the product to tumor, melanoma, and break tumor cells so there is an effect. After breaking the tumor cells, they release a lot of antigens that will take some time to stimulate the immune system response, and then you will have a full effect. Extreme case that hazard ratio=1 or w1 = 0. IF we don’t have issue for w1=0, not effect at all before t*, it should not have the problem for the case that w1>0.

Type I probability Inflated? Evaluation Some subsets may behave differently from others By allocating zero weight for other subsets, it may inflate Type I probability Evaluation No DE under the null of no difference between the two groups – use of different subset should not inflate Type I probability Simulation

Simulation study shows no Type I probability inflation   Sample Size Regular log-rank PWLRT t*= 3 months t*= 6 months t*= 9 months 300 600 900 0.0502 0.0503 0.0514 0.0510 0.0503 0.0520 0.0513 0.0511 0.0495 0.0496 0.0501 -- 100,000 replications testing at the alpha level of 0.05

Intention-to-treat (ITT) principle Violated? less weight for some events could mean they contribute less information to the test result 0 weight for some events may imply excluding some subjects so violating ITT principle Evaluation What is behind the ITT principle? Any potential bias?

Behind ITT principle ITT principle? Behind ITT principle? Conduct analysis according to the assigned treatment instead of the actual treatment Include all randomized subjects Behind ITT principle? Preserve the initial randomization in order to prevent bias and provide a secure foundation for statistical tests Bias with PWLRT? Should conduct analysis according to the assigned treatment instead of the actual treatment – same as for other methods Include all randomized subjects with different weights It’s not difficult to show the initial randomization is preserved even if zero weight is used (in a manuscript to be submitted - not show here in the interest of time) we have showed that the proposed PWLRT can still preserve the initial randomization with w1=0 as long as subjects are analyzed according to the treatment assignments they received in a manuscript to be submitted.

Study results interpretation -- a hypothetical example Consider a new product: no survival benefit until 7 months after treatment Regular log-rank test vs. PWLRT? Regular log-rank test (p = 0.04) KM plot for median survival: 25-month (new), 20-month (control) Show 5-month survival benefit PWLRT (p = 0.004) w*1 =0 for subjects died before 7-month, w*2 =1 for others KM plot for survival - same as above KM plot for subset of subjects who live > 7-month Median survival: 35-month (new) vs. 27-month (control) Show 8-month survival benefit KM shows the observed median survival 25 vs 20 month….

K-M plot with all subjects   K-M plot with subjects who live after 7 months

Study results interpretation (2) -- a hypothetical example Both regular log-rank test and PWLRT can be used Regular log-rank test: less power and may fail to detect a statistically significant Interpretations with PWLRT: 5-month survival benefit unconditional on the time point when the new product starts to work – same as for regular log-rank test No survival benefit for the new product before 7-month 8-month survival benefit instead of 5 35-month median survival instead of 25 for the treated group -- if live to the time when the new product kicks in its effect Which one is more informative or we need both? Study results based on PWLRT should be interpretable.

Changing point specification Challenging in estimating the changing point (t*) in practice If miss the changing point Under powered study? Worse than using the conventional method?

Estimate the changing point in practice Obtain a close estimate for the changing point Mechanism of action for the product Animal/other pre-clinical study data Literature Early phase trial data -- Join adventure from all disciplines

Missing the changing point Xu et al.: Designing therapeutic cancer vaccine trials with delayed treatment effect (Stat in Med 2017; 36(4):592-605) Assuming fixed changing point Address the robustness to the mis-specification of t* under various scenarios (Evaluation #4) Still better than the regular log-rank

Xu et al.: Evaluation #4 Table 1. The impact of mis-specifying t* on power calculation -- assume t*=6-month with 80% power for sample size (see paper for details)

Missing the changing point (2) “Designing cancer immunotherapy trials with random treatment time-log effect” -- Xu et al. accepted by SIM Assuming a subject-specific lag duration: t* ~ Dist(T1, T2) Incorporating individual heterogeneity in lag time to response to treatment Further reducing risk of mis-specification

Crossover effect Survival curves for a hypothetical crossover effect trial

Crossover effect Problematic for using PWLRT in trials with crossover effect Same problem by using other conventional methods such as the regular log-rank test Conventionally, a clinical trial should be designed with the assumption that there will be no crossover effect Review issues and sensitivity analyses

Summary Can the concerns be addressed? Type I probability Intention-to-treat principle Study results interpretation Changing point specification Crossover effect

Concluding Remark With careful consideration of all kind of potential biases and appropriate planning, the results based on PWLRT should be valid and interpretable

Thank you! My email: boguang.zhen@fda.hhs.gov