1 BLA Sipuleucel-T (APC-8015) FDA Statistical Review and Findings Bo-Guang Zhen, PhD Statistical Reviewer, OBE, CBER March 29, 2007 Cellular, Tissue and Gene Therapies Advisory Committee Meeting

2 Outline of Presentation Review of Efficacy Results Review of Efficacy Results Issues in Survival Analysis Issues in Survival Analysis Limitations of Post-Hoc Analysis Limitations of Post-Hoc Analysis Challenges in Survival Analysis Challenges in Survival Analysis

3 Review of Efficacy Results Two Phase III studies as main efficacy evidence to support licensing application (Studies 1 and 2) Both studies failed to meet the primary endpoint of time to disease progression (TTP) demonstrate statistical significance for other pre- specified endpoints Key efficacy evidence was based on the difference in overall survival (OS) between two arms

4 Review of Overall Survival Analysis * CI = confidence interval, unit for survival is in month Study 1 (N = 127) Study 2 (N = 98) Studies 1 & 2 (N = 225) Median Surv. (95% CI) * Sipuleucel-T Sipuleucel-T Placebo Placebo Diff. in Median Surv. Diff. in Median Surv (20.0, 32.4) 21.4 (12.3, 25.8) (13.6, 31.9) 15.7 (12.8, 25.4) (19.0, 31.0) 18.9 (13.5, 25.3) 4.3 Hazard Ratio (95% CI) 1.71 (1.13, 2.58) 1.27 (0.78, 2.07) 1.50 (1.10, 2.05) Log-rank p-value

5 Survival Sensitivity Analysis for Study 1 Covariate Adjustment in Cox model (N=127) * The sponsor’s analysis ~ HR = Hazard ratio Testing Method HR ~ P- value Covariate Adjustment Missing Log-rank* Cox Model (I)* Cox Model (II) Cox Model (III) None PSA(ln), LDH(ln), Weight, Localization of disease, #Bone metastases ( ≤5,6-10,>10 ) Localization of disease, Gleason Score ( ≤6, 7, ≥8 ), PSA (<20, 20 - <100, ≥100 ) Gleason Score ( ≤6, 7, ≥8 ), PSA (<20, 20 - <100, ≥100 ) 01051

6 Survival Sensitivity Analysis for Study 1 Covariate Adjustment in Cox model (Cont.) Impact of missing covariate data in Cox Model (I): Excluding from the Model? Sipuleucel-T Median Median N #Deaths Surv. (mons) N #Deaths Surv. (mons) Placebo Placebo Median Median N #Deaths Surv. (mons) N #Deaths Surv. (mons) YesNo

7 Survival Sensitivity Analysis for Study 1 Covariate Adjustment in Cox model (Cont.) Exclusion of patients due to missing covariate data could lead to biased estimates Exclusion of patients due to missing covariate data could lead to biased estimates Although p-values for treatment effect were greater than 0.05 in a few sensitivity analyses, the majority of the sensitivity analyses resulted in a p-value of <0.05 Although p-values for treatment effect were greater than 0.05 in a few sensitivity analyses, the majority of the sensitivity analyses resulted in a p-value of <0.05 Sensitivity analyses supported the “statistically significant finding” for OS Sensitivity analyses supported the “statistically significant finding” for OS

8 Survival Analysis for Study 2 p= based on log-rank test p= based on log-rank test Some patients may be excluded in sensitivity analyses using Cox model which could lead to biased estimates Some patients may be excluded in sensitivity analyses using Cox model which could lead to biased estimates Hypothesis test for treatment effect in Cox model resulted in: Hypothesis test for treatment effect in Cox model resulted in: op-values from to op-values from to op > 0.05 in most analyses op > 0.05 in most analyses Sensitivity analyses did not support the “statistically significant finding” for OS Sensitivity analyses did not support the “statistically significant finding” for OS

9 Review of Overall Survival Analysis --- Sensitivity analyses support the “statistically significant finding” for Study 1

10 Issues in Survival Analysis Overall survival (OS) as an endpoint was not defined in either study protocols Overall survival (OS) as an endpoint was not defined in either study protocols A statistical analysis method for the primary comparison in OS was not pre-specified A statistical analysis method for the primary comparison in OS was not pre-specified The alpha level (probability of making a false positive claim for treatment effect) was not allocated to the primary test for OS The alpha level (probability of making a false positive claim for treatment effect) was not allocated to the primary test for OS The ‘post-hoc’ analyses make it difficult to interpret the hypothesis test results for OS The ‘post-hoc’ analyses make it difficult to interpret the hypothesis test results for OS

11 Limitations of Post-Hoc Analysis Pre-specified vs. post-hoc analysis For designing a confirmative trial, it is essential to: Define endpoint(s) clearly Define endpoint(s) clearly Describe statistical analysis method(s) and state which one would be used for the primary comparison Describe statistical analysis method(s) and state which one would be used for the primary comparison Set alpha level. e.g.: α = 0.05 Set alpha level. e.g.: α = 0.05 Allocate alpha level to each test if multiplicity adjustment is needed Allocate alpha level to each test if multiplicity adjustment is needed -- Then, one is able to say: the difference is statistically significant or not based on the p-value from the primary comparison. Otherwise, it is difficult to interpret the p-value

12 Hypothetical cases: Interpretation of p-value in studies with pre-specified analysis * NS: non-statistically significant at the level of 0.05 Primary endpoint(s) Number of comparisons Alpha (0.05) Allocation Hypothetical p-value Trial A: TTP Interim analysis I Interim analysis I Interim analysis II Interim analysis II Final analysis Final analysis (NS)* 0.40 (NS)* 0.01 (NS) 0.09 (NS) Trial B: TTP and OS TTP – final analysis TTP – final analysis OS – final analysis OS – final analysis (NS) 0.01 (NS) Trial C: TTP and OS TTP – final analysis TTP – final analysis OS – final analysis OS – final analysis (NS) 0.50 (NS)

13 Limitations of Post-Hoc Analysis -- Difficulty in interpreting p-value Obtaining a p-value of 0.01 (or < 0.05) may not always be considered statistically significant in a well pre-specified analysis. Obtaining a p-value of 0.01 (or < 0.05) may not always be considered statistically significant in a well pre-specified analysis. When a study fails to meet its primary endpoint(s), there is no alpha left for other endpoint analyses. So literally, the difference in other endpoints should not be considered statistically significant. When a study fails to meet its primary endpoint(s), there is no alpha left for other endpoint analyses. So literally, the difference in other endpoints should not be considered statistically significant. Therefore, it is difficult to interpret the hypothesis test result for OS in Study 1 (p=0.01) Therefore, it is difficult to interpret the hypothesis test result for OS in Study 1 (p=0.01)

14 Limitations of Post-Hoc Analysis In post-hoc analyses one could In post-hoc analyses one could o keep conducting hypothesis tests for treatment effect on different endpoints and/or on the same endpoint using different analysis method o then easily obtain a so called “statistically significant result” (p< 0.05) even when there is no treatment effect If OS is one of the many un-specified endpoints under the testing, it is possible that a p-value of 0.01 was observed just by chance If OS is one of the many un-specified endpoints under the testing, it is possible that a p-value of 0.01 was observed just by chance However, OS is a preferred endpoint for cancer trial However, OS is a preferred endpoint for cancer trial

15 Challenges in Survival Analysis  p-value (0.01)  Difficulty in interpreting the p-value (0.01)  “Statistical significance” only demonstrated in Study 1.  The lower bound of 95% CI for hazard ratio (1.13) close to one