1. ISCB Vaccines Sub-Committee Web Seminar Series November 7, 2012 Assessing Immune Correlates of Protection Via Estimation of the Vaccine Efficacy Curve Peter Gilbert Fred Hutchinson Cancer Research Center and University of Washington, Department of Biostatistics

2. Outline 1.Introduction: Concepts and definitions of immune correlates/surrogate endpoints 2.Evaluating an immune correlate of protection via the vaccine efficacy curve 3.Statistical methods 2

3. Context: Preventive Vaccine Efficacy Trial Primary Objective – Assess VE: Vaccine Efficacy to prevent pathogen-specific disease Secondary Objective – Assess vaccine-induced immune responses as correlates of protection Randomize Vaccine Measure immune response Follow for clinical endpoint (pathogen-specific disease) Receive inoculations Placebo 3

4. Importance of an Immune Correlate Finding an immune correlate is a central goal of vaccine research – One of the 14 ‘Grand Challenges of Global Health’ of the NIH & Gates Foundation (for HIV, TB, Malaria) Immune correlates useful for: – Shortening trials and reducing costs – Guiding iterative development of vaccines between basic and clinical research – Guiding regulatory decisions – Guiding immunization policy – Bridging efficacy of a vaccine observed in a trial to a new setting Pearl (2011, International Journal of Biostatistics) suggests that bridging is the reason for a surrogate endpoint 4

5. Two Major Concepts/Paradigms for Surrogate Endpoints  Causal agent paradigm (e.g., Plotkin, 2008, Clin Infect Dis) Causal agent of protection = marker that mechanistically causes vaccine efficacy against the clinical endpoint  Prediction paradigm (e.g., Qin et al., 2007, J Infect Dis) Predictor of protection = marker that reliably predicts the level of vaccine efficacy against the clinical endpoint  Both are extremely useful for vaccine development, but are assessed using different approaches  For the goal of statistical assessment of surrogate endpoint validity in an efficacy trial, the prediction paradigm is used  As in the statistical literature, a good surrogate endpoint allows predicting VE from the vaccine effect on the surrogate 5

6. Immune Correlates Terminology: Contradictions Qin et al. (2007) Correlate (of risk) = measured immune response that predicts infection in the vaccine group Surrogate = measured immune response that can be used to reliably predict VE (may or may not be a mechanism of protection) Plotkin (2008) Correlate (of protection) = measured immune response that actually causes protection (mechanism of protection) Surrogate = measured immune response that can be used to reliably predict VE (is definitely not a mechanism of protection) Qin et al. correlate  Plotkin correlate [very different] Qin et al. surrogate  Plotkin surrogate 6

7. Reconciliation of Terminology: Plotkin and Gilbert (2012, Clin Inf Dis) 7 TermSynonymsDefinition CoPCorrelate of Protection Predictor of Protection; Good Surrogate Endpoint An immune marker statistically correlated with vaccine efficacy (equivalently predictive of vaccine efficacy)* that may or may not be a mechanistic causal agent of protection mCoPMechanistic Correlate of Protection Causal Agent of Protection; Protective Immune Function A CoP that is mechanistically causally responsible for protection nCoPNon-Mechanistic Correlate of Protection Correlate of Protection Not Causal; Predictor of Protection Not Causal A CoP that is not a mechanistic causal agent of protection *A CoP can be used to accurately predict the level of vaccine efficacy conferred to vaccine recipients (individuals or subgroups defined by the immune marker level). Thus a CoP is a surrogate endpoint in the statistical literature, and may be assessed with the Prentice framework or the principal stratification framework.

8. A Predictive Surrogate/CoP May or May Not be a Mechanism of Protection*  Definition of a CoP: An endpoint that can be used to reliably predict the vaccine effect on the clinical endpoint Plotkin and Gilbert (2012, Clin Inf Dis) Figure 1. A correlate of protection (CoP) may either be a mechanism of protection, termed mCoP, or a non-mechanism of protection, termed nCoP, which predicts vaccine efficacy through its (partial) correlation with another immune response(s) that mechanistically protects. 8

9. Many Ways for a CoR to Fail to be a CoP “A Correlate Does Not a Surrogate Make” –Tom Fleming 1.The biomarker is not in the pathway of the intervention's effect, or is insensitive to its effect –E.g., the immunological assay is noisy 2.The biomarker is not in the causal pathway of the exposure/infection/disease process –E.g., the antibody response neutralizes serotypes of the pathogen that rarely expose trial participants but fails to predominantly exposing serotypes 3.The intervention has mechanisms of action independent of the disease process –E.g., other immunological functions not measured by the assay are needed for protection 9

10. Catastrophic Failure of a CoR to be a CoP: the ‘Surrogate Paradox’ Surrogate Paradox: The vaccine induces an immune response, the immune response is inversely correlated with disease risk in vaccinees, but VE < 0% Three Causes of the Surrogate Paradox* 1.Confounding of the association between the potential surrogate and the clinical endpoint 2.The vaccine positively affects both the surrogate and the clinical endpoint, but for different sets of subjects 3.The vaccine may have a negative clinical effect in ways not involving the potential surrogate 10 *From Tyler VanderWeele

11. “There is a plague on Man, the opinion that he knows something.” − Michel de Montaigne (1580, Essays)

12. Outline 1.Introduction: Concepts and definitions of immune correlates/surrogate endpoints 2.Evaluating an immune correlate of protection via the vaccine efficacy curve* 3.Statistical methods *Gilbert, Hudgens, Wolfson (2011, J Inter Biostatistics) discussed the scientific value of the vaccine efficacy curve for vaccine development 12

13. Two Frameworks for Assessing a CoP from a Single Vaccine Efficacy Trial: Prentice & Principal Stratification (PS) Key Issue: Do trial participants have prior exposure to the pathogen under study?  If Yes, immune responses vary for both vaccine and placebo recipients  In this case, the Prentice and PS frameworks both apply  If No, immune responses vary for vaccinees only, and the Prentice framework does not apply (Chan et al., 2002, Stats Med)  In this case, only the PS framework applies  In this talk we consider the PS approach in both settings 13

14. Concept of PS Framework: Assess Association of Individual- Level Vaccine Effects on the Surrogate and Clinical Endpoint Vaccine Effect on Immune Response Marker for an Individual Vaccine Effect on Clinical Endpoint 14 Probability an individual is protected

15. Definition of a Principal Surrogate/Principal CoP Define the vaccine efficacy surface as VE(s1, s0) = 1 – Interpretation: Percent reduction in clinical risk for a vaccinated subject with markers (s1, s0) compared to if s/he had not been vaccinated Definition: A principal CoP is a marker with large variability of VE(s1, s0) in (s1, s0) Another useful property is VE(s1 = s0) = 0 –This property is Average Causal Necessity: No vaccine effect on the marker implies no vaccine efficacy Risk of clinical endpoint for vaccinees for subgroup with marker effect (s1, s0) Risk of clinical endpoint for placebos for subgroup with marker effect (s1, s0) 15

16. Marker Useless as a CoP CEP R (v 1, v 0 ) VE(s1, s0) Marker if vaccinated s1 Marker if placebo s0 16

17. CEP R (v 1, v 0 ) VE(s1, s0) Marker if vaccinated s1 Marker if placebo s0 Marker that is an Excellent CoP 17

18. Simplest Way to Think About the PS Framework for Assessing a CoP: It’s Simply Subgroup Analysis Conceptually the analysis assesses VE in subgroups defined by the vaccine effect on the marker –Evaluate if and how VE varies with ‘baseline’ subgroups defined by (S1, S0) –Principal stratification makes (S1, S0) equivalent to a baseline covariate A useful CoP will have strong effect modification, i.e., VE(s1, s0) varies widely in (s1, s0) It would be even more valuable to identify actual baseline covariates that well-predict VE, but it’s much more likely that a response to vaccination well-predicts VE 18

19. Simplified Definition of a Principal Surrogate/Principal CoP: Ignore the Immune Response under Placebo, S0 Define the vaccine efficacy curve as VE(s1) = 1 – Interpretation: Percent reduction in clinical risk for a vaccinated subject with markers s1 compared to if s/he had not been vaccinated Definition: A principal CoP is a marker with large variability of VE(s1) in s1 The vaccine efficacy curve is useful in both settings that participants have prior exposure to the pathogen or not If no prior exposure, then VE(s1, s0) = VE(s1), such that the vaccine efficacy surface simplifies to the vaccine efficacy curve Risk of clinical endpoint for vaccinees for subgroup with marker s1 Risk of clinical endpoint for placebos for subgroup with marker s1 19

20. Vaccine Efficacy Curve: Assess How VE Varies in the Marker Under Vaccination Marker level s1 VE(s1) Black marker: worthless as surrogate Green and blue markers satisfy causal necessity Blue marker: very good surrogate 20

21. Excellent CoP: Sets the Target for Improving the Vaccine Marker level s1 Black marker: worthless as surrogate Green and blue markers satisfy causal necessity Blue marker: very good surrogate VE(s1) Target: Improve the vaccine regimen by increasing the percentage of vaccinees with high immune responses 21

22. Knowledge of a CoP Guides Future Research to Develop Improved Vaccines  Identification of a good CoP in an efficacy trial is the ideal primary endpoint in follow-up Phase I/II trials of refined vaccines  It also generates a bridging hypothesis: If a future vaccine is identified that generates higher marker levels in more vaccinated subjects, then it will have improved overall VE 22

23. Using the CoP for Improving the Vaccine Regimen Original VaccineNew Vaccine 1New Vaccine 2 Marker levels 23

24. Using the CoP for Improving the Vaccine Regimen  Suppose each new vaccine is tested in an efficacy trial  Under the bridging hypothesis we expect the following efficacy results:  This is the idealized model for using a CoP to iteratively improve a vaccine regimen Original Vaccine New Vaccine 1 New Vaccine 2 Marker level Marker level Marker level Estimated VE Overall TE = 75% Overall TE = 50% Overall TE = 31% 24

25. Outline 1.Introduction: Concepts and definitions of immune correlates/surrogate endpoints 2.Evaluating an immune correlate of protection via the vaccine efficacy curve 3.Statistical methods 25

26. Challenge to Evaluating a Principal CoP: The Immune Responses to Vaccine are Missing for Subjects Assigned Placebo Accurately filling in the unknown immune responses is needed to evaluate a principal CoP Two approaches to filling in the missing data (Follmann, 2006, Biometrics): – BIP (Baseline immunogenicity predictor): At baseline, measure a predictor(s) of the immune response in both vaccinees and placebos – CPV (Close-out placebo vaccination): At study closeout, vaccinate disease-free placebo recipients and measure the immune response 26

27. Example of a Good BIP: Antibody Responses to Hepatitis A and B Vaccines* *Czeschinski et al. (2000, Vaccine) 18:1074-1080 Spearman rank r =.85 No cross- Reactivity N=75 subjects 27

28. Baseline Immunogenicity Predictor 28

29. Schematic of Baseline Immunogenicity Predictor (BIP) & Closeout Placebo Vaccination (CPV) Trial Designs* W W - - S=S(1) ScSc *Proposed by Follmann (2006, Biometrics)    1  1 +   1  1 +  S(1) CPV Approach BIP Approach Vx 29

30. Literature on Statistical Methods for Estimating the Vaccine Efficacy Curve via BIP and/or CPV ArticleComment 1. Follmann (2006, Biometrics) Binary outcome; BIP&CPV; Estimated likelihood 2. Gilbert and Hudgens (2008, Biometrics) Binary outcome; BIP; Estimated likelihood; 2-phase sampling 3. Qin, Gilbert, Follmann, Li (2008, Ann Appl Stats) Time-to-event outcome (Cox model); BIP&CPV; Estimated likelihood; 2- phase sampling 4. Wolfson and Gilbert (2010, Biometrics) Binary outcome; BIP&CPV; Estimated likelihood; 2-phase sampling; relaxed assumptions 5. Huang and Gilbert (2011, Biometrics) Binary outcome; BIP&CPV; Estimated likelihood; 2-phase sampling; relaxed assumptions; compare markers 6. Huang, Gilbert, Wolfson (2012, under revision) Binary outcome; BIP&CPV; Pseudolikelihood; 2-phase sampling; relaxed assumptions; marker sampling design 7. Miao, Li, Gilbert, Chan (2012, under revision) Time-to-event outcome (Cox model); BIP; Estimated likelihood with multiple imputation; 2-phase sampling 8. Gabriel and Gilbert (2012, submitted) Time-to-event outcome (Weibull model); BIP+CPV; Estimated likelihood and pseudolikelihood; 2-phase sampling; threshold models 30

31. Summary of One of the Principal Stratification Methods: Gilbert and Hudgens (2008, Biometrics) [GH] 31

32. Notation (Observed and Potential Outcomes) Z = vaccination assignment (0 or 1; placebo or vaccine) W = baseline immunogenicity predictor of S S = candidate surrogate endpoint/immune CoP measured at time  after randomization Y = clinical endpoint (0 or 1; 1 = experience event during follow-up) S(Z) = potential surrogate endpoint under assignment Z, for Z=0,1 Y(Z) = potential clinical endpoint under assignment Z, for Z=0,1 32

33. Assumptions A1 Stable Unit Treatment Value Assumption (SUTVA): (S i (1), S i (0), Y i (1), Y i (0)) is independent of the treatment assignments Z j of other subjects − A1 implies “consistency”: (S i (Z i ), Y i (Z i )) = (S i, Y i ) A2 Ignorable Treatment Assignment: Z i is independent of (S i (1), S i (0), Y i (1), Y i (0)) − A2 holds for randomized blinded trials A3 Equal individual clinical risk up to time  that S is measured (zero vaccine efficacy for any individual up to time  ) 33

34. Definition of a Principal Surrogate/Principal CoP (Frangakis and Rubin, 2002; Gilbert and Hudgens, 2008) Define risk (1) (s 1, s 0 ) = Pr(Y(1) = 1|S(1) = s 1, S(0) = s 0 ) risk (0) (s 1, s 0 ) = Pr(Y(0) = 1|S(1) = s 1, S(0) = s 0 ) A contrast in risk (1) (s 1, s 0 ) and risk (0) (s 1, s 0 ) is a causal effect on Y for the population {S(1) = s 1, S(0) = s 0 } VE(s 1, s 0 ) = 1 - risk (1) (s 1, s 0 ) / risk (0) (s 1, s 0 ) A good CoP has VE(s 1, s 0 ) varying widely in (s 1, s 0 ) [i.e., a large amount of effect modification] Also, with VE(s 1 ) = 1 - risk (1) (s 1 ) / risk (0) (s 1 ), a good CoP has VE(s 1 ) varying widely in s 1 These definitions allow for a spectrum of principal CoPs, some more useful than others, depending on the degree of effect modification 34

35. Statistical Methods: Build on Two-Phase Sampling Methods Case-cohort or case-control sampling (Ignore S0) −(W, S(1)) measured in All infected vaccines Sample of uninfected vaccines −W measured in o All infected placebos o Sample of uninfected placebos 2-Phase designs (E.g., Prentice, 1986, Biometrika; Kulich and and Lin, 2004, JASA; Breslow et al., 2009, AJE, Stat Biosciences) −Phase 1: Measure inexpensive covariates in all subjects −Phase 2: Measure expensive covariates X in a sample of subjects Our application −Vaccine Group: Exactly like 2-phase design with X = (W, S(1)) −Placebo Group: Like 2-phase design with X = (W, S(1)) and S(1) missing 35

36. IPW Case-Cohort Methods Do Not Apply: Hence we use a Full Likelihood-Based Method Most of the published 2-phase sampling/case-cohort failure time methods cannot be extended to estimate the VE curve – This is because they are inverse probability weighted (IPW) methods, using partial likelihood score equations that sum over subjects with phase-2 data only, which assume that every subject has a positive probability that S(1) is observed – However, all placebo subjects have zero-probability that S(1) is observed To deal with this problem, the published methods all use full likelihood, using score equations that sum over all subjects 36

37. Maximum Estimated Likelihood* with BIP Posit models for risk (1) (s 1,0;  ) and risk (0) (s 1,0;  ) Vaccine arm: −(W i, S i (1)) measured: Likld contribn risk (1) (S i (1), 0;  ) −(W i, S i (1)) not measured:  risk (1) (s 1, 0;  ) dF(s 1 ) Placebo arm: −W i measured: Likld contribn  risk (0) (s 1, 0;  ) dF S|W (s 1 | W i ) −W i not measured:  risk (0) (s 1, 0;  ) dF(s 1 ) L( , F S|W, F) =  i  risk (1) (S i (1),0;  ) Yi (1 - risk (1) (S i (1),0;  )) 1-Yi ] Zi }  i [Vx subcohort]   risk (0) (s i,0;  )dF S|W (s 1 |W i ) Yi (1 -  risk (0) (s 1,0;  )) dF S|W (s 1 |W i ) 1-Yi ] 1-Zi }  i [Plc subcohort]   risk (1) (s i,0;  )dF(s 1 ) Yi (1 -  risk (1) (s 1,0;  )) dF(s 1 ) 1-Yi ] Zi } 1-  i [Vx not subcohort]   risk (0) (s i,0;  )dF(s 1 ) Yi (1 -  risk (0) (s 1,0;  )) dF(s 1 ) 1-Yi ] 1-Zi } 1-  i [Plc not subcohort] *Pepe and Fleming (1991) an early article on estimated likelihood 37

38. Maximum Estimated Likelihood Estimation (MELE) Likelihood L( , F S|W, F) −  is parameter of interest [VE curve depends only on  ] −F S|W and F are nuisance parameters Step 1: Choose models for F S|W and F and estimate them based on vaccine arm data Step 2:Plug the consistent estimates of F S|W and F into the likelihood, and maximize it in  −e.g., EM algorithm Step 3:Estimate the variance of the MELE of , accounting for the uncertainty in the estimates of F S|W and F −Bootstrap 38

39. Example: Nonparametric Categorical Models Assume: − S and W categorical with J and K levels; S i (0)=1 for all i [No prior exposure scenario: category 1 = negative response] − Nonparametric models for P(S(1)=j, W=k) − A4-NP: Structural models for risk (z) (for z=0, 1) risk (z) (j, 1, k;  ) =  zj +  ’ k for j=1, …, J; k=1, …, K Constraint: 0 ≤  zj +  ’ k ≤ 1 and  k  ’ k = 0 for identifiability A4-NP asserts no interaction: W has the same effect on risk for the 2 study groups (untestable) 39

40. Vaccine Efficacy Curve for Categorical Marker CEP risk (j, 1) = log (avg-risk (1) (j, 1) / avg-risk (0) (j, 1)) where avg-risk (z) (j, 1) = (1/K)  k risk (z) (j, 1, k;  ) VE(j, 1) = 1 – exp{CEP risk (j, 1)} The vaccine efficacy curve is VE(j, 1) at each level j of S(1) 40

41. Tests for the Vaccine Efficacy Curve VE(j, 1) Varying in j Wald tests for whether a biomarker has any surrogate value − Under the null, PAE(w) = 0.5 and AS = 0 − Z = (Est. PAE(w) – 0.5)/ s.e.(Est. PAE(w)) − Z = Est. AS/ s.e.(Est. AS) o Estimates obtained by MELE; bootstrap standard errors For nonparametric case A4-NP, test H0: CEP risk (j, 1) = 0 vs H1: CEP risk (j, 1) increases in j (like Breslow-Day trend test) T =  j>1 (j-1) {Est.  0j – (Est.  0j + Est.  1j )(Est.  z0 /(Est.  z0 + Est.  z1 ))} divided by bootstrap s.e. Est.  z = (1/J)  j  zj 41

42. Simulation Study: Vax004 HIV Vaccine Efficacy Trial* Step 1: For all N=5403 subjects, generate (W i, S i (1)) from a bivariate normal with means (0.41, 0.41), sds (0.55, 0.55), correlation  = 0.5, 0.7, or 0.9  sd of 0.55 chosen to achieve the observed 23% rate of left-censoring  Values of W i, S i (1) 1 set to 1 Step 2: Bin W i and S i (1) into quartiles  Under model A4-NP generate Y i (Z) from a Bernoulli(  zj +  ’ k ) with the parameters set to achieve: o P(Y(1) = 1) = 0.067 and P(Y(0) = 1) = 0.134 (overall VE = 50%) o The biomarker has either (i) no or (ii) high surrogate value *Flynn et al. (2005, JID), Gilbert et al. (2005, JID) 42

43. Simulation Plan Scenario (i) (no surrogate value) −CEP risk (j, 1) = -0.69 for j = 1, 2, 3, 4 −i.e., VE(j, 1) = 0.50 for j = 1, 2, 3, 4 Scenario (ii) (high surrogate value) −CEP risk (j, 1) = -0.22, -0.51, -0.92, -1.61 for j = 1, 2, 3, 4 −i.e., VE(j, 1) = 0.2, 0.4, 0.6, 0.8 for j = 1, 2, 3, 4 43

44. Simulation Plan Step 3: Create case-cohort sampling (3:1 control: case) − Vaccine group: (W, S(1)) measured in all infected (n=241) and a random sample of 3 x 241 uninfected − Placebo group: W measured in all infected (n=127) and a random sample of 3 x 127 uninfected The data were simulated to match the real VaxGen trial as closely as possible 44

45. Model A4-NP Simulation Results: Bias and Coverage Probabilities [Table 1 of GH] 45

46. Model A4-NP Simulation Results: Power to Detect the VE(j,1) curve varying in j [Table 2 of GH]  = 0.5, 0.7, 0.9 Trend tests for VE(j, 1) increasing in j: Power 0.83, 0.99, > 0.99 for  = 0.5, 0.7, 0.9 46

47. Conclusions of Simulation Study The MELE method of Gilbert and Hudgens performs well for realistically-sized Phase 3 vaccine efficacy trials, with accuracy, precision, and power improving sharply with the strength of the BIP (desire high  ) This shows the importance of developing good BIPs R code for the nonparametric method available at the Biometrics website and at http://faculty.washington.edu/peterg/SISMID2011.html http://faculty.washington.edu/peterg/SISMID2011.html 47

48. Crossing over more placebo subjects improves power of CPV and BIP + CPV designs There is no point of diminishing returns− steady improvement with more crossed over, out to complete cross-over If the BIP is high quality (e.g.,  > 0.50), then the BIP design is quite powerful with only modest incremental gain by adding CPV However, CPV has additional value beyond efficiency improvement: – Helps in diagnostic tests of structural modeling assumptions (A4) – May help accrual and enhance ethics – May adaptively initiate crossover, e.g., as soon as the lower 95% confidence limit for VE exceeds 30% Pseudoscore method superior to estimated likelihood method (Huang, Gilbert, Wolfson, 2012, under revision); recommend this method in practice – Happy to provide the code for this method (for BIP, CPV, BIP+CPV ) Remarks on Power for Evaluating a Principal Surrogate Endpoint (For All Methods- Beyond GH) 48

49. Concluding Remarks Opportunity to improve assessment of immune CoPs by increasing research into developing BIPs – The better the BIP, the greater the accuracy and precision for estimating the vaccine efficacy curve 49

50. Some Avenues for Identifying Good BIPs Demographic factors – E.g., age, gender, BMI, immune status Host immune genetics – E.g., HLA type and MHC binding prediction machine learning methods for predicting T cell responses Add beneficial licensed vaccines to efficacy trials and use known correlates of protection as BIPs (Follmann’s [2006] original proposal) – The HVTN is exploring this strategy in a Phase 1 trial in preparation for efficacy trials Develop ‘pathogen exposure history’ chip In efficacy trials where participants have prior exposure to the pathogen, measure the potential CoP at baseline and use it as the baseline predictor – E.g., Varicella Zoster vaccine trials: baseline gpELISA titers strongly predict post-immunization titers – Miao, Li, Gilbert, Chan (2012, under review) and Gabriel and Gilbert (2012, in preparation) estimate the Zoster vaccine efficacy curve using this excellent BIP (will constitute an excellent example when published) 50