1. Use of Archived Tissue in Evaluating the Medical Utility of Prognostic & Predictive Biomarkers Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. Prognostic & Predictive Biomarkers Most cancer treatments benefit only a minority of patients to whom they are administered Most cancer treatments benefit only a minority of patients to whom they are administered Being able to predict which patients are likely to benefit would Being able to predict which patients are likely to benefit would Save patients from unnecessary toxicity, and enhance their chance of receiving a drug that helps them Save patients from unnecessary toxicity, and enhance their chance of receiving a drug that helps them Control medical costs Control medical costs Improve the success rate of clinical drug development Improve the success rate of clinical drug development

4. Predictive & Prognostic Biomarkers Predictive biomarkers Predictive biomarkers Measured before treatment to identify who will or will not benefit from a particular treatment Measured before treatment to identify who will or will not benefit from a particular treatment ER, HER2, KRAS ER, HER2, KRAS Prognostic biomarkers Prognostic biomarkers Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment Used to identify who does not require more intensive treatment Used to identify who does not require more intensive treatment OncotypeDx OncotypeDx

5. Prognostic and Predictive Biomarkers in Oncology Single gene or protein measurement Single gene or protein measurement ER protein expression ER protein expression HER2 amplification HER2 amplification KRAS mutation KRAS mutation Index or classifier that summarizes expression levels of multiple genes Index or classifier that summarizes expression levels of multiple genes OncotypeDx recurrence score OncotypeDx recurrence score

6. Types of Validation for Prognostic and Predictive Biomarkers Analytical validation Analytical validation Accuracy compared to gold-standard assay Accuracy compared to gold-standard assay Robust and reproducible if there is no gold-standard Robust and reproducible if there is no gold-standard Pre-analytical robustness to tissue handling and preparation Pre-analytical robustness to tissue handling and preparation Clinical validation/correlation Clinical validation/correlation Does the biomarker predict what it’s supposed to predict for independent data Does the biomarker predict what it’s supposed to predict for independent data Clinical/Medical utility Clinical/Medical utility Does use of the biomarker result in patient benefit Does use of the biomarker result in patient benefit Generally by improving treatment decisions Generally by improving treatment decisions Is it “actionable” Is it “actionable”

7. Prognostic Factors in Oncology Most prognostic factors are not used because although they correlate with a clinical endpoint, they have no demonstrated medical utility Most prognostic factors are not used because although they correlate with a clinical endpoint, they have no demonstrated medical utility They are developed in unfocused studies that use convenience samples of heterogeneous patients for whom tissue is available They are developed in unfocused studies that use convenience samples of heterogeneous patients for whom tissue is available They are not reliable because they are exploratory and not prospectively focused on a single factor They are not reliable because they are exploratory and not prospectively focused on a single factor

8. Predictive Biomarkers In the past often studied as un-focused post-hoc subset analyses of RCTs. In the past often studied as un-focused post-hoc subset analyses of RCTs. Numerous subsets examined Numerous subsets examined No focused pre-specified hypothesis No focused pre-specified hypothesis No control of type I error No control of type I error

9. Prospective Validation of the Medical Utility of a Prognostic Biomarker Develop biomarker predictive of outcome for low stage patients without chemotherapy. Then Develop biomarker predictive of outcome for low stage patients without chemotherapy. Then Select low stage patients who are considered good prognosis by biomarker Select low stage patients who are considered good prognosis by biomarker Randomize them to standard chemotherapy or to no chemotherapy Randomize them to standard chemotherapy or to no chemotherapy e.g. MINDACT e.g. MINDACT

10. If predicted risk of recurrence without chemotherapy is sufficiently low, then randomization might be omitted omited and chemotherapy withheld If predicted risk of recurrence without chemotherapy is sufficiently low, then randomization might be omitted omited and chemotherapy withheld If their outcome is sufficiently good without chemotherapy, absolute benefit of chemotherapy would be very small If their outcome is sufficiently good without chemotherapy, absolute benefit of chemotherapy would be very small TAILORx TAILORx

11. Prospective Evaluation of a Predictive Biomarker Perform Test with Predictive Biomarker Predicted Non- responsive to New Rx Predicted Responsive To New Rx Control New RXControl New RX

12. Prospective Evaluation of Predictive Biomarker The RCT The RCT focused on a single pre-specified biomarker focused on a single pre-specified biomarker sized with sufficient marker + and marker – patients for adequately powered separate analysis of T vs C differences sized with sufficient marker + and marker – patients for adequately powered separate analysis of T vs C differences Evaluating a predictive biomarker does not involve comparison of outcome of marker + vs marker – patient Evaluating a predictive biomarker does not involve comparison of outcome of marker + vs marker – patient The purpose of the RCT is not to re-derive or refine the definition of the biomarker The purpose of the RCT is not to re-derive or refine the definition of the biomarker

13. R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-93, 2008 R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-93, 2008 R Simon and A Maitournim. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004. R Simon and A Maitournim. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004. R Simon. Roadmap for developing and validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology 23:7332-41, 2005. R Simon. Roadmap for developing and validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology 23:7332-41, 2005.

14. It may be infeasible or unethical to conduct a new prospective trial to test a hypothesis about a prognostic or predictive biomarker For tests relating to the use of approved therapeutics For tests relating to the use of approved therapeutics KRAS for anti-EGFR antibodies in colorectal cancer KRAS for anti-EGFR antibodies in colorectal cancer HER2 for doxorubicin HER2 for doxorubicin TAILORx and MINDACT trials require many thousands of patients TAILORx and MINDACT trials require many thousands of patients Genomic test strategy designs which randomize whether to perform or not perform the test often require tens of thousands of patients Genomic test strategy designs which randomize whether to perform or not perform the test often require tens of thousands of patients

15. In some cases the benefits of a prospective trial can be closely achieved by the carefully planned use of archived tissue from a previously conducted randomized clinical trial In some cases the benefits of a prospective trial can be closely achieved by the carefully planned use of archived tissue from a previously conducted randomized clinical trial

16. Use of Archived Specimens in Evaluation of Prognostic and Predictive Biomarkers Richard M. Simon, Soonmyung Paik and Daniel F. Hayes Claims of medical utility for prognostic and predictive biomarkers based on analysis of archived tissues can be considered to have either a high or low level of evidence depending on several key factors. Claims of medical utility for prognostic and predictive biomarkers based on analysis of archived tissues can be considered to have either a high or low level of evidence depending on several key factors. Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence. Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence. Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a convenience sample of tissues and conducted in an exploratory and unfocused manner provide a very low level of evidence for clinical utility. Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a convenience sample of tissues and conducted in an exploratory and unfocused manner provide a very low level of evidence for clinical utility.

17. For Level I Evidence Archived tissue adequate for a successful assay must be available on a sufficiently large number of patients from a phase III trial with a design that enables the appropriate analyses Archived tissue adequate for a successful assay must be available on a sufficiently large number of patients from a phase III trial with a design that enables the appropriate analyses e.g. For predictive marker, RCT comparing T to Control e.g. For predictive marker, RCT comparing T to Control Adequate statistical power Adequate statistical power The patients included in the evaluation are clearly representative of the patients in the trial. The patients included in the evaluation are clearly representative of the patients in the trial. The test should be analytically and pre-analytically validated for use with archived tissue. The test should be analytically and pre-analytically validated for use with archived tissue. An analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier. An analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier. The results of the analysis should be validated using specimens from a similar, but separate, study The results of the analysis should be validated using specimens from a similar, but separate, study

18. Does It Matter If the Randomization in the RCT Was Not “Stratified” By the Test? No No Stratification improves balance of stratification factors in overall comparisons Stratification improves balance of stratification factors in overall comparisons Stratification does not improve comparability of treatment (T) and control (C) groups within test positive patients or within test negative patients Stratification does not improve comparability of treatment (T) and control (C) groups within test positive patients or within test negative patients

19. What Proportion of the Patients Should Have Adequate Archived Tissue Available for a Valid Analysis? No minimum percentage can be adequately defended No minimum percentage can be adequately defended For treatment versus control comparisons within test + patients (or within test – patients), the analysis is internally valid regardless of the proportion with available tissue or the selection factors affecting tissue availability For treatment versus control comparisons within test + patients (or within test – patients), the analysis is internally valid regardless of the proportion with available tissue or the selection factors affecting tissue availability External validity (generalization of results to the broad population of patients) depends on representativeness of the full study group as well as the proportion with tissue available External validity (generalization of results to the broad population of patients) depends on representativeness of the full study group as well as the proportion with tissue available

20. Does the RCT From Which the Archived Tissues are Taken Need to Be Significant Overall for the T vs C Treatment Comparison? No No In a fully prospective trial of T vs C with a companion test it is incorrect to require that the overall T vs C comparison be significant to claim that T is better than C for test + patients but not for test – patients In a fully prospective trial of T vs C with a companion test it is incorrect to require that the overall T vs C comparison be significant to claim that T is better than C for test + patients but not for test – patients That requirement has been traditionally used to protect against data dredging. It is inappropriate for focused trials of a treatment with a companion test. That requirement has been traditionally used to protect against data dredging. It is inappropriate for focused trials of a treatment with a companion test. Since the requirement is inappropriate for a fully prospective trial, it is also inappropriate for a properly designed prospective- retrospective trial Since the requirement is inappropriate for a fully prospective trial, it is also inappropriate for a properly designed prospective- retrospective trial

21. Factor ABCD Clinical trial RCT* designed to address tumor marker RCT* not designed to address tumor marker, but design accommodates tumor marker utility. Prospective observational registry, treatment and followup not dictated No prospective aspect to study Patients and patient data Prospectively enrolled, treated, and followed in RCT* Prospectively enrolled in registry, but treatment and followup standard of care No prospective stipulation of treatment or followup; patient data collected by retrospective chart review Specimen collection, processing, and archival Specimens collected, processed and assayed for specific marker in real time Specimens collected, processed, and archived prospectively using generic SOPs. Assayed after trial completed with analytically validated test Specimens collected, processed, and archived prospectively using generic SOPs. Assayed after trial completed Specimens collected, processed and archived with no prospective SOPs Statistical Design and analysis Study powered to address tumor marker question. Study powered to address therapeutic question; underpowered to address tumor marker question. Focused analysis plan for marker question developed prior to doing assays Study not prospectively powered at all. Retrospective study design confounded by selection of specimens for study. Focused analysis plan for marker question developed prior to doing assays Study not prospectively powered at all. Retrospective study design confounded by selection of specimens for study. No focused analysis plan for marker question developed prior to doing assays Validation Although preferred, validation not required Requires one or more additional validation studies Result very likely to be play of chance. Requires subsequent validation studies Result very likely to be play of chance. Requires subsequent validation TerminologyProspective Prospective using archived samples Prospective /observational Retrospective/observational * RCT required for predictive biomarker

22. Revised Levels of Evidence for Tumor Marker Studies Level of Evidence Category from Table 1 Validation Studies Available IA None required IB One or more with consistent results IIBNoneor Inconsistent results IIC 2 or more with consistent results IIICNoneor 1 with consistent results or Inconsistent results Inconsistent results IV-VDNA

23. Barriers to the Development of Prognostic and Predictive Biomarkers for Clinical Use Increased complexity and cost of developing new drugs with predictive biomarkers Increased complexity and cost of developing new drugs with predictive biomarkers Difficulty of identifying the appropriate predictive biomarker prior to conduct of phase III clinical trials of new drugs Difficulty of identifying the appropriate predictive biomarker prior to conduct of phase III clinical trials of new drugs Inadequate appreciation of the distinction between establishing clinical correlation and medical utility for prognostic markers Inadequate appreciation of the distinction between establishing clinical correlation and medical utility for prognostic markers Inadequate appreciation of the importance of focused testing of pre-specified marker hypotheses Inadequate appreciation of the importance of focused testing of pre-specified marker hypotheses High costs and time required for prospective validation of prognostic factors for withholding chemotherapy in low-risk populations High costs and time required for prospective validation of prognostic factors for withholding chemotherapy in low-risk populations Inadequate focus on analytically validating tests Inadequate focus on analytically validating tests Unavailability of adequate archived tissue from patients on key RCT’s Unavailability of adequate archived tissue from patients on key RCT’s

24. Barriers to the Development of Prognostic and Predictive Biomarkers for Clinical Use Lack of regulatory guidance for approval of in-vitro diagnostics and of treatments for restricted populations that Lack of regulatory guidance for approval of in-vitro diagnostics and of treatments for restricted populations that is appropriate to the inherent heterogeneity of human cancers is appropriate to the inherent heterogeneity of human cancers encourages the development of predictive biomarkers encourages the development of predictive biomarkers is cognizant of the infeasibility of always addressing treatment by subset questions prospectively is cognizant of the infeasibility of always addressing treatment by subset questions prospectively e.g. Difficulty of approving even restriction of a labeling indication based on a compelling prospective-retrospective analyses of KRAS in multiple randomized colorectal cancer trials e.g. Difficulty of approving even restriction of a labeling indication based on a compelling prospective-retrospective analyses of KRAS in multiple randomized colorectal cancer trials

25. Conclusions New biotechnology and knowledge of tumor biology provide important opportunities to improve therapeutic decision-making New biotechnology and knowledge of tumor biology provide important opportunities to improve therapeutic decision-making Treatment of broad populations with regimens that do not benefit most patients is no longer necessary nor economically sustainable Treatment of broad populations with regimens that do not benefit most patients is no longer necessary nor economically sustainable The established molecular heterogeneity of tumors requires new approaches to the development of cancer treatments The established molecular heterogeneity of tumors requires new approaches to the development of cancer treatments The traditional assumption underlying most clinical trial design that treatment by subset interactions are unlikely is often inappropriate today The traditional assumption underlying most clinical trial design that treatment by subset interactions are unlikely is often inappropriate today

26. For establishing medical utility of prognostic and predictive tests, fully prospective trial designs are desirable For establishing medical utility of prognostic and predictive tests, fully prospective trial designs are desirable It is often not possible, however, to identify the appropriate predictive marker for a drug prior to the conduct of the pivotal trials of that drug It is often not possible, however, to identify the appropriate predictive marker for a drug prior to the conduct of the pivotal trials of that drug It is often not feasible or ethical to conduct prospective RCT’s for validating predictive markers of approved drugs It is often not feasible or ethical to conduct prospective RCT’s for validating predictive markers of approved drugs It is very expensive and time consuming to conduct prospective trials to evaluate prognostic markers that indicate withholding standard chemotherapy regimens from good risk patients It is very expensive and time consuming to conduct prospective trials to evaluate prognostic markers that indicate withholding standard chemotherapy regimens from good risk patients

27. Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence Barriers to the development, approval and use of effective prognostic and predictive biomarkers that benefit patients and society are substantial Barriers to the development, approval and use of effective prognostic and predictive biomarkers that benefit patients and society are substantial

28. Acknowledgements Soonmyung Paik, NSABP Soonmyung Paik, NSABP Daniel Hayes, U. Michigan Daniel Hayes, U. Michigan