1. Metrological Experiments in Biomarker Development (Mass Spectrometry—Statistical Issues) Walter Liggett Statistical Engineering Division Peter Barker Biotechnology Division National Institute of Standards and Technology

2. Biomarker (Clinical Pharmacology & Therapeutics, 2001) A characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention. Two parts of a biomarker –Execution of measurement protocol –Interpretation of measured response

3. Metrology Development and evaluation of a measurement protocol, the first part of a biomarker Diverse lessons learned from varied applications Focus on general purpose protocols which may be adequate for a particular purpose The use of metrology in biomarker development is the subject of this talk

4. Metrological Experiments Experimental units (specimens) –Knowledge of their characteristics –Relation to unknowns of future interest Response –Univariate—interval-scale variable –Multivariate/Functional Protocol parameters—parameter design Cost of experimental runs—high throughput?

5. Outline Alternative statistical formulations –Classification based on cases and controls –Measurement of an interval-scale variable Aspects of protocol development –Property of interest –Realization of protocol Multivariate and functional measurements

6. Statistics for Classification Assume gold standard for disease status Evaluate marker on training data –Sensitivity—true positive rate –Specificity—1 – false positive rate Continuous test result—ROC curves Multivariate test result—classification, discriminant analysis

7. Pepe, et al., J. National Cancer Institute, 2001 Specimen Selection 1.Wide spectrum of tumor and non-tumor tissue 2.Serum from cases and controls in a target screening population 3.Apparently healthy subjects monitored for development of cancer 4.Cohort from a population that might be targeted 5.Subjects randomly selected from populations in which the screening program is likely

8. Thinking Outside the Box Bottom line is prediction of disease status Definitive gold standard may not be available Including laboratory sources of error in training data is a problem There are metrological experiments that do not require a gold standard

9. The Role of Science Given valid training data, statisticians can proceed without scientific knowledge In the classification approach, scientific thought must go into specimen selection In the metrological approach, focus is on a property to be measured Scientific thought must go into the relation of the metrological property to biomarker goals

10. Statistics for Metrology Focus (as best one can) on the property to be measured, an interval- or ratio-scale variable Specify a baseline measurement protocol Experiment with realizations of alternative protocols Optimize repeatability (at least) and then ask if the measurement protocol is adequate for the purpose

11. Framework of Metrology Relation between property and protocol obtained scientifically or through realization Metrology explores faithfulness of realization before adequacy for the purpose Property RealizationProtocol

12. Some Metrological Experiments Protocol development through classes of units known to differ in the property of interest Protocols linked to a scientific definition of the property of interest in such a way that all sources of error can be assessed (definitive methods) Sets of protocols that measure the same property but are based on different scientific principles (independent methods)

13. Aspects of Performance Repeatability All manner of reproducibility –Operator, equipment –Inter-laboratory Noise factors, effect of sample matrix Calibration Measurement assurance Uncertainty components, type A and type B uncertainties

14. Experimental Units (Reference Materials) Homogeneity (solution versus particles) Quantity (cost) Adaptable to high-throughput experiments Known value of the property of interest Classes with different values of the property of interest

15. From Univariate to Functional Carryover has been done for classification Extending measurement performance concepts to multivariate and functional responses is still a challenge Chemometrics is the key word for much of the literature in this area

16. Functional Principal Components Analysis (Ramsay and Silverman) Metrologists like to look at the spread of a batch of measurements (outliers, more than one mode) For functional measurements, functional PCA provides a way to look at the spread Consider results of functional PCA on Petricoin’s Lancet…/Normal Healthy (SPLUS, Ramsay’s software) Main purpose is to illustrate metrological thinking

21. Conclusion Producing large data sets has become easier except perhaps for selecting individuals with a particular disease status With scientific and statistical reasoning, the advances in experimentation technology can be used to speed biomarker development Statisticians have a role in formulating overall experimental strategy, allocating effort among different approaches