1. Survival Analysis, Type I and Type II Error, Sample Size and Positive Predictive Value Larry Rubinstein, PhD Biometric Research Branch, NCI International Clinical Trial Workshop ASCO, FLASCA, NCI, ONS Cordoba, Argentina September 12, 2014

2. Financial Disclosure  I have nothing to disclose.

3. Survival Analysis: The Hazard Ratio  The basic problem in clinical trials is measuring and comparing time-to-death (OS) or time-to-progression (PFS) for experimental vs. control treatments.  We can describe the event (death or progression) rates, over time, by means of the hazard functions, λ e (t) and λ c (t), for the experimental and control treatments.  We assume that the experimental treatment decreases the event rate by a constant proportion Δ: λ e (t) = λ c (t) /Δ.

4. The Hazard Ratio Estimator  Δ = λ c (t) / λ e (t) = Median (E) / Median (C), the ratio of the experimental to control median times-to-event.  Δ ranges from 0 to ∞, and under the null hypothesis of no treatment effect (H 0 ), Δ = 1.  ln Δ ranges from -∞ to ∞, and under H 0, ln Δ = 0.  The estimator ln δ of ln Δ is normal with variance 1/D e + 1/D c ≈ 4/D (where D e, D c, and D are the numbers of observed deaths).  The statistic Z = ln δ / √(4/D) is asymptotically normal with variance ≈ 1.

5. Type I Error (α, Significance Level)  We want to limit the type I error (α) – the probability of calling an experimental treatment useful when H 0 is true (Δ = 1).  Under H 0, Z = ln δ / √(4/D) is normal (0,1).  If the test statistic Z > 1.645, we are at least 95% confident that Δ > 1 (H 0 is false; ln Δ > 0), by the properties of the standard normal distribution, and we can reject H 0.  In general, if Z > Z α, the upper α-percentile of the standard normal, we have (100 - α) % confidence that Δ > 1 and can reject H 0.

6. Type II Error (β, Power = 1 - β)  We want to limit the type II error – the probability of calling an experimental treatment useless when H 1 is true (Δ = 2, for example). We want 90% power, for example, to reject H 0 in this case.  We want Z > Z α with 90% likelihood. By the properties of the normal distribution, we want the expectation of Z to be Z α + 1.282 (Z has variance 1 under H 1, also).  In general, to have power 1 – β, we want the expectation of Z = (ln Δ * √D) / 2 = Z α + Z β.

7. Sample Size: Required Number of Events D  For type I error α and type II error β, we want the expectation of Z = (ln Δ * √D) / 2 = Z α + Z β.  For type I error α and type II error β, the required D = 4 * (Z α + Z β ) 2 / (ln Δ) 2.  Required D increases as α and β decrease: (Z.1 + Z.1 ) 2 = 6.6; (Z.025 + Z.1 ) 2 = 10.5 (60% larger)  Phase 3 trials must be larger than randomized phase 2 trials, in part, because of the smaller type I error required.

8. Sample Size: Phase 2 vs. Phase 3 Trials  Generally, phase 3 trials have an OS primary endpoint, while randomized phase 2 trials have a PFS primary endpoint, since it is generally much shorter and the targeted Δ can be larger.  The required D drops as the target hazard ratio Δ increases. For example, for α =.025, β =.1: Target Hazard Ratio ΔRequired Number of Events D 1.4372 1.6191 1.8122 2.088

9. Phase 2 and 3: Positive Predictive Value  Positive predictive value: the likelihood that a treatment is effective when the trial is positive  PPV = pr(H 1 ) * (1–β) / (pr(H 1 ) * (1–β) + pr(H 0 ) * α)  For a likely phase 2 scenario, the pr(H 1 ) =.1, and α = β =.1, we can calculate that PPV =.5.  For a likely phase 3 scenario, pr(H 1 ) =.5, α =.025, β =.1, we can calculate that PPV =.97.  Randomized phase 2 trials with PFS endpoints efficiently screen out most of the ineffective treatments and enable reliable phase 3 trials with OS endpoints (and equipoise at initiation).

10. References 1. Simon: Optimal two-stage designs for phase II clinical trials. Control Clin Trials 10:1, 1989. 2. Rubinstein, Korn, Freidlin, Hunsberger, Ivy, Smith: Design issues of randomized phase II trials and a proposal for phase II screening trials. J Clin Oncol 23:7199, 2005. 3. Rubinstein: Therapeutic studies. Hematology/Oncology Clinics of North America 14:849, 2000.