1. Use of Prognostic & Predictive Genomic Biomarkers in Clinical Trial Design Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. BRB Website brb.nci.nih.gov Powerpoint presentations Powerpoint presentations Reprints Reprints BRB-ArrayTools software BRB-ArrayTools software Data archive Data archive Q/A message board Q/A message board Web based Sample Size Planning Web based Sample Size Planning Clinical Trials Clinical Trials Optimal 2-stage phase II designs Optimal 2-stage phase II designs Phase III designs using predictive biomarkers Phase III designs using predictive biomarkers Phase II/III designs Phase II/III designs Development of gene expression based predictive classifiers Development of gene expression based predictive classifiers

3. Prognostic & Predictive Biomarkers Most cancer treatments benefit only a minority of patients to whom they are administered Most cancer treatments benefit only a minority of patients to whom they are administered Being able to predict which patients are likely to benefit would Being able to predict which patients are likely to benefit would Save patients from unnecessary toxicity, and enhance their chance of receiving a drug that helps them Save patients from unnecessary toxicity, and enhance their chance of receiving a drug that helps them Control medical costs Control medical costs Improve the success rate of clinical drug development Improve the success rate of clinical drug development

5. Different Kinds of Biomarkers Endpoint Endpoint Measured before, during and after treatment to monitor treatment effect Measured before, during and after treatment to monitor treatment effect Pharmacodynamic Pharmacodynamic Intermediate Intermediate Phase II Phase II Futility analysis in phase III Futility analysis in phase III Patient management Patient management Surrogate for clinical outcome Surrogate for clinical outcome

6. Surrogate Endpoints It is extremely difficult to properly validate a biomarker as a surrogate for clinical outcome. It requires a series of randomized trials with both the candidate biomarker and clinical outcome measured It is extremely difficult to properly validate a biomarker as a surrogate for clinical outcome. It requires a series of randomized trials with both the candidate biomarker and clinical outcome measured

7. Intermediate Endpoints in Phase I and II Trials Biomarkers used as endpoints in phase I or phase II studies need not be validated surrogates of clinical outcome Biomarkers used as endpoints in phase I or phase II studies need not be validated surrogates of clinical outcome The purposes of phase I and phase II trials are to determine whether to perform a phase III trial, and if so, with what dose, schedule, regimen and on what population of patients The purposes of phase I and phase II trials are to determine whether to perform a phase III trial, and if so, with what dose, schedule, regimen and on what population of patients Claims of treatment effectiveness should be based on phase III results Claims of treatment effectiveness should be based on phase III results

8. Different Kinds of Biomarkers Predictive biomarkers Predictive biomarkers Measured before treatment to identify who will or will not benefit from a particular treatment Measured before treatment to identify who will or will not benefit from a particular treatment Prognostic biomarkers Prognostic biomarkers Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment

9. Prognostic and Predictive Biomarkers in Oncology Single gene or protein measurement Single gene or protein measurement Expression of drug target Expression of drug target Activation of pathway Activation of pathway Scalar index or classifier that summarizes expression levels of multiple genes Scalar index or classifier that summarizes expression levels of multiple genes Disease classification Disease classification

10. Types of Validation for Prognostic and Predictive Biomarkers Analytical validation Analytical validation Accuracy, reproducibility, robustness Accuracy, reproducibility, robustness Clinical validation Clinical validation Does the biomarker predict a clinical endpoint or phenotype Does the biomarker predict a clinical endpoint or phenotype Clinical utility Clinical utility Does use of the biomarker result in patient benefit Does use of the biomarker result in patient benefit By informing treatment decisions By informing treatment decisions Is it actionable Is it actionable

11. Pusztai et al. The Oncologist 8:252-8, 2003 939 articles on “prognostic markers” or “prognostic factors” in breast cancer in past 20 years 939 articles on “prognostic markers” or “prognostic factors” in breast cancer in past 20 years ASCO guidelines only recommend routine testing for ER, PR and HER-2 in breast cancer ASCO guidelines only recommend routine testing for ER, PR and HER-2 in breast cancer

12. Most prognostic markers or prognostic models are not used because although they correlate with a clinical endpoint, they do not facilitate therapeutic decision making; i.e. they have no demonstrated medical utility Most prognostic markers or prognostic models are not used because although they correlate with a clinical endpoint, they do not facilitate therapeutic decision making; i.e. they have no demonstrated medical utility Most prognostic marker studies are based on a “convenience sample” of heterogeneous patients, often not limited by stage or treatment. Most prognostic marker studies are based on a “convenience sample” of heterogeneous patients, often not limited by stage or treatment. The studies are not planned or analyzed with clear focus on an intended use of the marker The studies are not planned or analyzed with clear focus on an intended use of the marker Retrospective studies of prognostic markers should be planned and analyzed with specific focus on intended use of the marker Retrospective studies of prognostic markers should be planned and analyzed with specific focus on intended use of the marker Design of prospective studies depends on context of use of the biomarker Design of prospective studies depends on context of use of the biomarker Treatment options and practice guidelines Treatment options and practice guidelines Other prognostic factors Other prognostic factors

13. OncotypeDx as a Model for Development of a Therapeutically Relevant Gene Expression Signature <10% of node negative ER+ breast cancer patients require or benefit from the cytotoxic chemotherapy that they receive <10% of node negative ER+ breast cancer patients require or benefit from the cytotoxic chemotherapy that they receive Identify patients with node negative ER+ breast cancer who have low risk of recurrence on tamoxifen alone Identify patients with node negative ER+ breast cancer who have low risk of recurrence on tamoxifen alone

14. B-14 Results—Relapse-Free Survival 338 pts 149 pts 181 pts p<0.0001 Paik et al, SABCS 2003

15. Key Features of OncotypeDx Development Focus on important therapeutic decision context Focus on important therapeutic decision context Staged development and validation Staged development and validation Separation of data used for test development from data used for test validation Separation of data used for test development from data used for test validation Development of robust analytically validated assay Development of robust analytically validated assay

16. Potential Uses of a Prognostic Biomarker Identify patients who have very good prognosis on standard treatment and do not require more intensive regimens Identify patients who have very good prognosis on standard treatment and do not require more intensive regimens Identify patients who have poor prognosis on standard chemotherapy who are good candidates for experimental regimens Identify patients who have poor prognosis on standard chemotherapy who are good candidates for experimental regimens

17. Predictive Biomarkers

20. In the past often studied as exploratory post-hoc subset analyses of RCTs. In the past often studied as exploratory post-hoc subset analyses of RCTs. Numerous subsets examined Numerous subsets examined No pre-specified hypotheses No pre-specified hypotheses No control of type I error No control of type I error Led to conventional wisdom Led to conventional wisdom Only hypothesis generation Only hypothesis generation Only valid if overall treatment difference is significant Only valid if overall treatment difference is significant

21. Prospective Co-Development of Drugs and Companion Diagnostics 1. Develop a completely specified genomic classifier of the patients likely to benefit from a new drug 2. Establish analytical validity of the classifier 3. Use the completely specified classifier in the primary analysis plan of a phase III trial of the new drug

22. Guiding Principle The data used to develop the classifier should be distinct from the data used to test hypotheses about treatment effect in subsets determined by the classifier The data used to develop the classifier should be distinct from the data used to test hypotheses about treatment effect in subsets determined by the classifier Developmental studies can be exploratory Developmental studies can be exploratory Studies on which treatment effectiveness claims are to be based should not be exploratory Studies on which treatment effectiveness claims are to be based should not be exploratory

23. Using phase II data, develop predictor of response to new drug Develop Predictor of Response to New Drug Patient Predicted Responsive New Drug Control Patient Predicted Non-Responsive Off Study

24. BRB-ArrayTools Architect – R Simon Architect – R Simon Developer – Emmes Corporation Developer – Emmes Corporation Contains wide range of analysis tools that I have selected Contains wide range of analysis tools that I have selected Designed for use by biomedical scientists Designed for use by biomedical scientists Imports data from all gene expression and copy-number platforms Imports data from all gene expression and copy-number platforms Automated import of data from NCBI Gene Express Omnibus Automated import of data from NCBI Gene Express Omnibus Highly computationally efficient Highly computationally efficient Extensive annotations for identified genes Extensive annotations for identified genes Integrated analysis of expression data, copy number data, pathway data and data other biological data Integrated analysis of expression data, copy number data, pathway data and data other biological data

25. Predictive Classifiers in BRB-ArrayTools Classifiers Classifiers Diagonal linear discriminant Diagonal linear discriminant Compound covariate Compound covariate Bayesian compound covariate Bayesian compound covariate Support vector machine with inner product kernel Support vector machine with inner product kernel K-nearest neighbor K-nearest neighbor Nearest centroid Nearest centroid Shrunken centroid (PAM) Shrunken centroid (PAM) Random forrest Random forrest Tree of binary classifiers for k-classes Tree of binary classifiers for k-classes Survival risk-group Survival risk-group Supervised pc’s Supervised pc’s With clinical covariates With clinical covariates Cross-validated K-M curves Cross-validated K-M curves Predict quantitative trait Predict quantitative trait LARS, LASSO LARS, LASSO Feature selection options Univariate t/F statistic Hierarchical random variance model Restricted by fold effect Univariate classification power Recursive feature elimination Top-scoring pairs Validation methods Split-sample LOOCV Repeated k-fold CV.632+ bootstrap Permutational statistical significance

26. BRB-ArrayTools June 2009 10,000+ Registered users 10,000+ Registered users 68 Countries 68 Countries 1000+ Citations 1000+ Citations

27. Acknowledgements NCI Biometric Research Branch NCI Biometric Research Branch Kevin Dobbin Kevin Dobbin Alain Dupuy Alain Dupuy Boris Freidlin Boris Freidlin Wenyu Jiang Wenyu Jiang Aboubakar Maitournam Aboubakar Maitournam Michael Radmacher Michael Radmacher Jyothi Subramarian Jyothi Subramarian George Wright George Wright Yingdong Zhao Yingdong Zhao BRB-ArrayTools Development Team BRB-ArrayTools Development Team Soon Paik, NSABP Soon Paik, NSABP Daniel Hayes, U. Michigan Daniel Hayes, U. Michigan