Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics October 29, 2007

Outline General guidelines Objectives of screening studies Phase I: Pre-clinical exploratory studies Phase II: Clinical assay development for clinical disease Phase III: Retrospective longitudinal repository studies Phase IV: Prospective screening studies Phase V: Prevention/control studies

General Guidelines Biomarker / Marker indicator of a particular disease state of a patient individual marker, panel of markers, signature, etc. Clinical development process series of well-defined steps from identification of a potentially useful biomarker through systematic evaluation of its clinical utility

General Guidelines (2) What is the intended final use of biomarker? clinical versus population screen stand-alone biomarker versus panel asymptomatic versus symptomatic normals diagnosis versus prognosis versus prediction Need discrete decision points: pursue or not Criteria identify markers that have promise to be clinically useful assess the best methodology for clinical evaluation of markers in question confirm or validate that additional clinical utility is gained by using marker compared to standard practice

Screening Study Objectives Non-invasive Inexpensive Secreted by disease tissue only Highly sensitive Highly specific Most likely requires the use of multiple markers to obtain high sensitivity and specificity.

Pre-clinical Exploratory Studies Comparison of disease tissue versus non- disease tissue Identify unique disease characteristics that might lead to ideas for clinical assays immunohistochemistry western blot gene expression profiles protein expression profiles levels of circulating antibodies

Pre-clinical Exploratory Studies (2) Primary objectives identify leads for potential useful markers prioritize identified leads Specimen selection (case) disease tissue before treatment (control) non-disease tissue matched to case samples

Pre-clinical Exploratory Studies (3) Primary outcome measure biomarker value assay reliability / reproducibility Analysis binary TPR: true positive rate FPR: false positive rate continuous sensitivity (TPR) specificity (1 – FPR) ROC curve

Pre-clinical Exploratory Studies (4) Analysis (2) selection of candidate markers find all that are statistically significant rank based on summary statistic confirmatory analysis training / test samples cross-validation

Pre-clinical Exploratory Studies (5) Sample size considerations number and relative prevalence of disease subtypes ability of markers to discriminate among different disease subtypes number of candidate markers under study number of case / control samples statistical methodology being used Best to select sample sizes based on simulation studies.

Clinical Assay Development Develop (non-invasive) clinical assay Primary objective estimate the TPR and FPR (or ROC curve) of the clinical biomarker assay Other objectives optimize assay performance determine relationship between assay levels on disease tissue and clinical specimen assess patient/subject characteristics associated with biomarker status (level) in control subjects assess disease characteristics associated with biomarker status (level) in case subjects

Clinical Assay Development (2) Specimen selection case samples before treatment control samples matched to case samples Primary outcome measure result of clinical marker assay

Clinical assay development (3) Analysis estimate of TPR and FPR (or ROC curve) test of TPR is too low and/or FPR is too high select minimally acceptable FPR and determine whether TPR is about the acceptable threshold

Clinical assay development (4) Sample Size depends on the precision wanted for TPR and FPR choose size for adequate power to determine whether TPR and FPR are acceptable

Retrospective (Longitudinal) Repository Studies Idea: compare the assay values of case samples collected before their diagnosis to control samples Primary objectives evaluate, as a function of time before clinical diagnosis, the capacity of the biomarker to detect preclinical disease define criteria for a positive screening test

Retrospective (Longitudinal) Repository Studies (2) Other objectives explore the impact of covariates (demographics, disease-related characteristics, etc.) on the discriminatory abilities of the biomarker before clinical diagnosis compare markers with a view to selecting those that are most promising to develop algorithms for screen positivity based on combinations of markers determine a screening interval if repeated screening is of interest

Retrospective (Longitudinal) Repository Studies (3) Specimen selection should be protocol driven cases/controls should be obtained from target population controls are those that develop disease should match on all variables, including follow-up

Retrospective (Longitudinal) Repository Studies (4) Primary outcome result of clinical marker assay Analysis comparison TPR and FDR (or ROC curves) consider restricting analysis to TPR at the (maximally) acceptable FPR rate ROC curves should be time-dependent (to account for time from test to disease presentation)

Retrospective (Longitudinal) Repository Studies (5) Sample size number of case subjects number of control subjects number of clinical specimens per subject The sample sizes should ensure that, for each preclinical time lag of interest (e.g., 1 year, 2 years, 4 years), there are enough specimens from control subjects and from case subjects taken close to those intervals so that biomarker accuracy can be estimated with sufficient precision.

Prospective Screening Studies Idea: the screen is applied to individuals and definitive diagnostic procedures are applied at that time to those screening positive the number and nature of cases detected with the screening tool are determined as are the numbers of subjects falsely screening positive and referred for work- up

Prospective Screening Studies (2) Primary objective determine the operating characteristics (TPR and FPR) of the biomarker based screening test in a relevant population Other objectives describe the characteristics of disease detected by screening test assess practicability of applying the screening program make preliminary assessments of effects of screening on costs/mortality/morbidity monitor disease that occurs but is not detected by the screening protocol

Prospective Screening Studies (3) Subject selection target population inclusion/exclusion criteria also consider inclusion of unscreened control arm Primary outcome measure screening test positive and disease confirmed screening test positive and disease not confirmed screening test negative

Prospective Screening Studies (4) Analysis estimate of detection rate: those screened positive who are positive estimate of false-referral rate: those screened positive but do not have disease multivariable analysis to adjust for covariates comparison of multiple screening tests Sample size depends on desired precision, or depends on relative performance if comparing different screening assays

Disease Control Studies Idea: determine whether screening reduces the burden of disease on the population Primary objective estimate the reductions in disease mortality afforded by the screening test

Disease Control Studies (2) Other objectives obtain information about the costs of screening and treatment and the cost per life saved evaluate compliance with screening and work-up in a diverse range of settings compare different screening protocols and/or to compare different approaches to treating screen-detected subjects in regard to effects on mortality and costs

Disease Control Studies (3) Subject selection randomly selected from populations in which the screening program is likely to be implemented Ideal: standard parallel-arm randomized clinical trial, with one arm consisting of subjects undergoing the screening protocol and the other arm consisting of unscreened subjects

Disease Control Studies (4) Primary outcome time from entry into the study until death Analysis survival analysis methods are used to compare the study arms with regard to overall mortality methods for comparing costs and quality of life for randomized trials

Disease Control Studies (5) Sample size To detect a 20% reduction in cause- specific mortality with 80% power at the.05 two-sided significance level, standard calculations indicate that 650 deaths would need to be observed

Discussion Not all studies need to undergo all the described phases Need for formal guidelines For Phase III studies (retrospective repository), need criteria to allocate scarce resources in sensible/fair fashion Choices of cases and controls in all phases is complex, requires thought Need new statistical methodology