Personalized Medicine Detection Diagnosis Treatment Survival

Prediction is very difficult, especially about the future. Niels Bohr Danish physicist ( ) Niels Bohr

Biomarkers TestInformationDecision Outcome 1. Discrimination (sensitivity, specificity, predictive value, ROC analysis) 2. Utility (disease free survival, recurrence rates, survival etc)

Diagnostic tests Describing test performance Test Result DiseaseNo disease Total Positiveaba+b Negativecdc+d Totala+cc+da+b+c+d Properties of a test Sensitivity: – a/a+c Specificity: – d/c+d Positive predictive value: – a/a+b Negative predictive value: – d/c+d

The importance of disease prevalence Test result Breast cancer No breast cancer Total Positive3604,9805,340 Negative4094,62094,660 Total40099,600100,000 Screening mammography Properties of the test Sensitivity: 90% a/a+c = 360/400 Specificity: 95% d/c+d = 94,620/99,600 Positive predictive value: a/a+b = 360/5340 = 7% Negative predictive value: d/c+d =94,620/94,660 = 100%

Desiderata for studies of diagnostic tests. “Gold” standard Test result before outcome known “Blind” reading Pre-determined cut-off Sensitivity and specificity. Predictive value. Receiver operating. characteristic curves (ROC).

Diagnostic tests and the spectrum of disease. Spectrum of patients. Clinical spectrum Co-morbid spectrum Pathologic spectrum Potential biases in test evaluation. Exclusion of equivocal cases Work up bias Test review bias Incorporation bias

Clinical value of tests Test Information Decision Outcome

PRINCIPAL AGENT COMPARATIVE AGENT INITIAL STATE, RECIPIENTS OF PRINCIPAL AGENT INITIAL STATE, RECIPIENTS OF COMPARATIVE AGENT SUBSEQUENT EVENTS, RECIPIENTS OF PRINCIPAL AGENT SUBSEQUENT EVENTS, RECIPIENTS OF COMPARATIVE AGENT

Research Designs-General Structure Purpose of research (initial states) Prevention. Prediction of risk in healthy. Treatment response or toxicity in those with disease. Identify factors that influence outcome (prognosis). Types of manoeuver Inherited (eg genetic variant). Acquired –Self selected (smoking, alcohol) –Other (treatment). Imposed (atomic irradiation).

Principal research designs Disease PresentAbsent Presentab Exposure Absentcd Passage of time Relative risk = a/a+b ÷ c/c+d Cohort study

Nested case control studies Screening programs: NBSS, SMPBC, OBSP case 6-8 years follow-up case control Baseline mammogram Risk factors

How many subjects (or samples) do you need? Number of events (eg deaths). Willingness to risk a false positive (Type I) error. Willingness to risk a false negative result (Type II) error. Magnitude of difference worthwhile to detect. Time for accrual and follow-up.

Sample size to detect an improvement in survival (alpha=0.05; 1-beta=0.90) P2-P1 P

Sample size for genetic studies

SUBSEQUENT EVENTS R } PRINCIPAL AGENT COMPARATIVE AGENT { INITIAL STATE

A trial to change diet Vancouver + Surrey Windsor London + Sarnia Hamilton + KW Toronto Funding: Ontario Ministry of Health, Medical Research Council, Canadian Breast Cancer Research Alliance, National Institutes of Health, American Institute for Cancer Research Screening Randomization 4,693 Low-fat diet Usual diet >8 years counseling and follow-up

Association or causation? Not all associations are causal All causal factors show association May be due to bias or confounding Genetic associations –Causal –In linkage disequilibrium with the causal variant –Population stratification

Population stratification Type of confounding Ethnicity –associated with disease –associated with genotype –gives spurious association between genotype and disease Can be controlled in analysis (if recognized) Dispute about importance

Analysis P<0.05 What does this mean?

The meaning of p-values. If the TRUE difference between the compared groups is zero (the null hypothesis), the PROBABILITY of obtaining a difference as large or larger than the one observed by CHANCE is p.

Multiple comparisons The problem. If alpha = comparisons can be expected to generate one p<0.05. (1-(1-alpha) k, where alpha is the level for significance and k=number of tests. What protection? Few, a priori hypotheses Correction for number of tests eg Bonferroni –Alpha/number of tests Stringent alpha eg E Replication/validation

Francis Galton’s ox and the “Winner’s curse”. Country fair in bought tickets and predicted the weight of an ox. Actual weight was 1,198 lbs. None were close to the actual weight. Mean predicted weight (N=787) was 1,197 lbs. At auction, most bids cluster around the “true” value of the object. The winning bid is always higher than the “true” value.

Replication -validation “leave one out” – Applied to “learning set” – Not an independent sample – May help avoid overfitting Independent data set – Preferably also an independent investigator

How to get a statistically significant result. Count or ignore differences in follow-up time. Censor at different time points. Exclude specific causes of death. Exploit sub-group analysis. Use different cut-offs for gene expression (or other test result). Note: all of the above increases the number of statistical tests you can do!

Can you believe the literature? Publication bias (author and editor bias). Multiple statistical testing. The “Winner’s curse”. Bias in the sampling, measurement or analysis of the data. Most published reports are never replicated.

The “ Winners Curse ” False positives more likely : Small studies Small effects Early, hypothesis generating studies Financial interest “ Hot ” field Ioannidis PLos Medicine 2005

How to stay out of trouble Define target population. Standardize sample collection. Collect samples at zero time. Define outcomes at the outset. Random selection of cases and controls. Analyze samples without knowledge of case/control status. Replicate.