1. Challenges of Non-Inferiority Trial Designs R. Sridhara, Ph.D.

2. 12/6/06Pediatric Advisory Committe2 Terminology Superiority = Drug T better than Placebo (P) or T better than Active Control (C) C may or may not have known efficacy Non-inferiority (NI) = T not much less effective than C  ‘Was Not Different’ or ‘Similar’ C must have known efficacy

3. 12/6/06Pediatric Advisory Committe3 Clinical Trials to Demonstrate Efficacy Superiority Claim First choice Direct evidence Non-inferiority Claim Last choice Indirect evidence Interpretation could be misleading

4. 12/6/06Pediatric Advisory Committe4 Non-inferiority Claim Implies: 1.The effect of T and C are close 2.T and C may be both equally beneficial or T and C may be both equally not beneficial

5. 12/6/06Pediatric Advisory Committe5 Assumptions for NI consideration C has an effect (compared to placebo, P) Therefore, C can not be another experimental therapy in NI trial Can reliably estimate C effect size * C effect size compared to P Control (C) effect in the future study population will be same as in the historical population.

6. 12/6/06Pediatric Advisory Committe6 Non-inferiority Trial Design Considerations Endpoint - Overall Survival - Progression-free Survival - Response Rate Control Effect  % Retention

7. 12/6/06Pediatric Advisory Committe7 Non-inferiority or Superiority Challenges with Non-inferiority Trial How well we know the effect of the control How much of the control effect can we afford to give up At least ‘  ’% retention of the control effect necessary to make sure that the new treatment is significantly better than placebo

8. 12/6/06Pediatric Advisory Committe8 Superiority trial designs with OS as the primary endpoint Null Hypothesis, H 0 : HR(T/C) = 1 vs. Alternative, H 1 : HR (T/C) < 1 if T is better than C

9. 12/6/06Pediatric Advisory Committe9 Non-inferiority trial designs with OS as the primary endpoint H 0 : HR(T/C) ≥ M vs. H 1 : HR (T/C) = 1 if the two are expected to be similar, Or H 1 : HR(T/C) = 0.95, for example, if the new treatment (T) is expected to be slightly better M is the margin determined based on the active control (C) effect size estimated from historical trials and percentage of the effect to be retained HR > 1 implies, T is worse than C

10. 12/6/06Pediatric Advisory Committe10 Percent Retention and Estimated Active Control Effect Size Non-inferiority ≈ “not much less effective” ‘X’ is the effect size of the active control; example: Point estimate of HR (C/P) = 0.5 (or HR(P/C) = 2.0) implies an estimate of active control effect size was a 50% reduction in the risk of event (ex: death) Percent retention is a percentage of ‘X’ that is retained; example: a 50% retention of the 50% effect size is 25% effect size, i.e., the putative HR(T/P) = 0.75 (or HR(P/T) = 1.33)

11. 12/6/06Pediatric Advisory Committe11 Estimates of True Control Effect Lower 95% C.L. (  << 0.025) Lower  % C.L. (  = 0.025) Point Estimate (  >> 0.025)

12. 12/6/06Pediatric Advisory Committe12 Methods for estimating active control effect size: No method is ideal and no particular method is endorsed by the Agency All methods assume that the control effect has not changed over time Some methods do not consider the variation between and within studies Other methods incorporate variation between and within studies

13. 12/6/06Pediatric Advisory Committe13 Hypothetical Example of a NI Trial with Overall Survival Endpoint Suppose HR(P/C) = 2.0 (95% CI: 1.9, 2.1) Then, Lower 70% Confidence limit = 1.97

14. 12/6/06Pediatric Advisory Committe14 Hypothetical example: Estimates of True Control Effect Lower 95% C.L. of HR = 1.9 Lower 70% C.L. of HR: 1.97 Point Estimate HR(P/C) = 2.0

15. 12/6/06Pediatric Advisory Committe15 Hypothetical Example of a NI Trial with Overall Survival Endpoint Suppose we want to retain 50% of 1.97, i.e., HR(P/T) = 1.49, and we can accrue 100 patients per unit time If H 1 : HR(T/C) = 1, then N = 407 events or 1045 patients will be required If H 1 : HR(T/C) = 0.95, then N = 290 events or 836 patients

16. 12/6/06Pediatric Advisory Committe16 Sample Size Depends On: How good the historical data is: One randomized study – large confidence interval – more uncertainty about the effect size Many randomized studies – smaller confidence interval – unless there were fundamental differences in the conduct of the studies: population, dose, etc.

17. 12/6/06Pediatric Advisory Committe17 Summary - 1 Superiority trials provide direct evidence New drug can be compared to placebo or control (established or not) Non-inferiority trials provide indirect evidence New drug must be compared to established control Interpretation can be misleading

18. 12/6/06Pediatric Advisory Committe18 Summary - 2 For NI trial consideration, active control effect must be well characterized Able to estimate effect size Control effect is same now as before NI trials are generally large Sample sizes for a NI trial is dependent on the estimate of control effect, population, % retention and the alternative hypothesis

19. 12/6/06Pediatric Advisory Committe19 Summary - 3 In considering NI trial, any potential loss of efficacy must be weighed against risk-benefit ratio Failed superiority does not imply non- inferiority