1. 

20. Ruling in or out a disease Tests to rule out a disease  You want very few false negatives  High sensitivity  Thus, if you get a negative test, it is likely a true negative  Mnemonic: SnOUT (High SeNsitivity rules OUT disease)  Example: D-dimer for DVT/PE Test to rule in a disease  You want very few false positives  High specificity  A positive test is likely to be a true positive  Mnemonic: SpIN (High Specificty rules IN disease)  Example: Pathology for malignancy

21.  Prior and posterior disease probabilities

22. Prior and Posterior Probabilities  What you thought before + New Information = What you think now  “What you thought before” = Prior (pre-test) probability  Disease prevalence  Probability of disease given patient’s presentation  “New Information” = Test result  “What you think now” = Posterior (post-test) probability  For positive dichotomous test, positive predictive value  For negative dichotomous test, 1 – negative predictive value  For multilevel and continuous tests, see chapter 4  Serial tests

23.  2x2 table method for updating prior probabilities How to populate a 2 x 2 table Example: Serological testing for TB

24. How to populate a 2 x 2 table  Given sensitivity, specificity, prevalence (prior probability)  Calculate positive predictive value, negative predictive value, etc.  Scenarios  Applying test characteristics derived in one population to another  Applying test characteristics derived from a case / control study  Applying independent tests serially*  Assume sensitivity and specificity are intrinsic to test and independent of population * Coming in chapter 8, “Multiple tests and multivariable decision rules”

25. How to populate a 2 x 2 table  Use prevalence to calculate D + and D - totals  Use sensitivity and specificity to calculate A, B, C, and D  Use A, B, C, and D to calculate positive and negative predictive values

26. Example: Serological Test for TB  Anda-TB IgG  ELISA test for anti-A60 antibodies  Sensitivity 76%  Specificity 92%  Prevalence  Uganda: 30%  SFGH: 5% TBNo-TBTotal Positive Negative Total3007001000  Have TB  1000 x 30% = 300  Don’t have TB  1000 – 300 = 700

27. Example: Serological Test for TB  Anda-TB IgG  ELISA test for anti-A60 antibodies  Sensitivity 76%  Specificity 92%  Prevalence  Uganda: 30%  SFGH: 5% TBNo-TBTotal Positive228 Negative72 Total3007001000  Test positive if they have TB  300 x 76% = 228  Test negative if they have TB  300 – 228 = 72

28. Example: Serological Test for TB  Anda-TB IgG  ELISA test for anti-A60 antibodies  Sensitivity 76%  Specificity 92%  Prevalence  Uganda: 30%  SFGH: 5% TBNo-TBTotal Positive22856 Negative72644 Total3007001000  Test negative if healthy  700 x 92% = 644  Test positive if healthy  700 – 644 = 56

29. Example: Serological Test for TB  Anda-TB IgG  ELISA test for anti-A60 antibodies  Sensitivity 76%  Specificity 92%  Prevalence  Uganda: 30%  SFGH: 5% TBNo-TBTotal Positive22856284 Negative72644716 Total3007001000  Positive Predictive Value  228/284 = 80.3%  Negative Predictive Value  644/716 = 89.9%

30.  Positive Predictive Value  228/284 = 80.3%  Negative Predictive Value  644/716 = 89.9% Example: Serological Test for TB Uganda (prevalence 30%)SFGH (prevalence 5%) TBNo-TBTotal Positive3876114 Negative12874886 Total509501000 TBNo-TBTotal Positive22856284 Negative72644716 Total3007001000  Positive Predictive Value  38/114 = 33.3%  Negative Predictive Value  12/886 = 98.6%

31. Sampling Scheme Matters Cross sectional studyCase control study  Positive Predictive Value  228/284 = 80.3%  Negative Predictive Value  644/716 = 89.9% TBNo-TBTotal Positive38040420 Negative120460580 Total500 1000 TBNo-TBTotal Positive22856284 Negative72644716 Total3007001000  Positive Predictive Value  380/420 = 90.5%  Negative Predictive Value  460/580 = 79.3%

32.  Odds and Probabilities Mmmm… pizza Converting probabilities to odds and vice versa

33. Mmmm… pizza  Imagine that you want to divide a pizza evenly between you and a friend  1 to 1 ratio of pizza between the two of you  Each gets 50% of the pizza

34. Mmmm… pizza  3 to 2 ratio of pepperonis between the two slices  One person gets 3/5 (60%) of the pepperonis  One person gets 2/5 (40%) of the pepperonis

35. Odds and Probabilities Odds  ‘Odds’ refers to the ratios of the two portions  Odds of 1 to 1 can be expressed as 1:1 or 1  Odds of 3 to 2 can be expressed as 3:2 or 1.5  Odds of 2 to 3 can be expressed as 2:3 or 0.67 Probabilities  ‘Probabilities’ refers to the proportion of each portion to the whole  Probability of ½ is 0.5  Probability of 3/5 is 0.6  Probability of 2/5 is 0.4

36. Odds and Probabilities Odds  Odds can range from 0 to ∞  To convert from probabilities:  Odds = p / (1 – p)  Odds > probability  As p → 0  Odds ≈ probabilities Probabilities  Probabilities can range from 0 to 1  To convert from odds:  p = odds / (1 + odds)

37. Odds at low probabilities

38. Converting odds to Probability and vice versa  Odds of pizza is 0.33 (1:3)  P = odds / (1 + odds)  P = 0.33 / (1 + 0.33)  P = 0.25 (¼)  Probability of pepperoni is 0.2 ( ⅕ )  Odds = p / (1 – p)  Odds = 0.2 / (1 – 0.2)  Odds = 0.25 (1:4)

39.  Likelihood ratios for dichotomous tests Likelihood ratios Example: Fetal fibronectin and preterm labor

40. Likelihood ratios  Converts prior odds to posterior odds given a test result  Prior odds  (A + C) / (B + D)  Posterior odds  Positive test: A / B  Negative test: C / D  Likelihood ratio is multiplier to go from prior odds to posterior odds D+D+ D-D- Total PositiveABA + B Negativ e CDA + C Total A + CB + D N LR + = sens/(1 – spec) LR - = (1 – sens) / spec

41. Generalizing likelihood ratios  It is possible to have more than two results in a test  Multilevel and continuous tests (chapter 4)  You can use likelihood ratios for these  LR + and LR - don’t make sense  LR result = P(result|D + ) / P(result|D - )

42. LR method for updating prior probabilities  Step 1: Convert pretest probability to pretest odds  Step 2: Calculate appropriate likelihood ratio  Step 3: Multiply pretest odds by appropriate likelihood ratio to get post-test odds  Step 4: Convert post-test odds back to probability

43. Example: FFN for Preterm Labor  Fetal fibronectin: extracellular glycoprotein produced in the decidua and chorion  Presence in vaginal secretions between 24 and 34 weeks gestation associated with preterm delivery  ACOG recommends against screening asymptomatic women  “May be useful in patients at risk for preterm birth” (within 7 days)

44. Example: FFN for Preterm Labor  Clinical scenario: 35 year old woman at 30 weeks gestation and history of preterm births, complains of “uterine tightening”  Should she be given tocolytics and steroids for fetal lung maturation  Fetal Fibronectin  Sensitivity: 76%  Specificity: 82%  Prior probability of preterm birth within 7 days: 8%  Sanchez-Ramos et al. Obstet Gynecol 2009;114:631–40

45. Example: FFN for Preterm Labor  Step 1: Convert prevalence to odds  Odds = p/(1 – p)  0.08/(1 – 0.08) =.087  Remember when probability is small, odds ~ probability

46. Example: FFN for Preterm Labor  Step 2: Calculate likelihood ratios  LR+ = sensitivity / (1 – specificity)  LR- = (1 – sensitivity) / specificity  LR+ = 0.76 / (1 – 0.82) = 0.76 / 0.18 = 4.2  LR- = (1 – 0.76) / 0.82 = 0.24 / 0.82 = 0.29

47. Example: FFN for Preterm Labor  Step 3: Multiply pretest odds by appropriate likelihood ratio  Post-test odds = pretest-odds x LR  For positive test:  Post-test odds = 0.087 x 4.2  Post-test odds = 0.37  For negative test  Post-test odds = 0.087 x 0.29  Post-test odds = 0.025

48. Example: FFN for Preterm Labor  Step 4: Convert post-test odds back to probability  p = odds / (1 + odds)  For positive test:  Post-test probability = 0.37 / (1 + 0.37)  Post-test probability = 27%  For negative test  Post-test probability = 0.025 / (1 + 0.025)  Post-test probability = 2.48%

49. Example: FFN for Preterm Labor  Positive predictive value  61/227 = 26.9%  Negative predictive value  754/773 = 97.5% preter m not preter m Tota l Positive61166227 Negativ e 19754773 Total809201000 Sensitivity: 76% Specificity: 82%

50.  Treatment and Testing Thresholds Quantifying costs and benefits Testing thresholds for a perfect but risky or expensive test

51. Quantifying costs and benefits  A “wrong” clinical decision carries cost  Treatment of individuals without disease  Cost of therapy  Discomfort, side effects, etc.  Failure to treat individuals with disease  Pain and suffering  Lost productivity  Additional cases

52. Quantifying costs and benefits  Benefits of tests  Increases probability of making the “right” decision  Reduces costs from “wrong” decision  Costs of tests  Cost of the test itself  Discomfort and complications from performing the test  May still lead to “wrong” decision (imperfect sensitivity, specificity)

53. Quantifying costs and benefits  C = Cost of unnecessary treatment  B = Benefit forgone by failure to treat  T = Testing cost  Cost of test itself, pain and discomfort associated with procedure  Does not include cost of errors

54. Treatment Thresholds  Balance costs  Cost of treatment x probability of unnecessary treatment (no disease)  Net benefit forgone x probability of failure to offer treatment (probability of disease)  Cost of test

55. Treatment Thresholds  Treat:  Probability of disease is high and/or  Cost of treatment is low  Don’t treat:  Probability of disease is low and/or  Cost of failure to treat is low and/or  Cost of unnecessary treatment is high  Cancer chemotherapy  Test:  Test offers substantial improvement in diagnosis and/or cost of test is low

56. Cost of treatment  Cost of treatment x probability of unnecessary treatment (no disease)  C x (1 – P)

57. Cost of no treatment  Net benefit forgone x probability of failure to offer treatment (probability of disease)  B x P

58. Lowest Cost Option  Black line is lowest cost option  Cost of treatment > Cost of no treatment  Don’t treat (red)  Cost of treatment < Cost of no treatment  Treat (green)  Cost of treatment = Cost of no treatment  Treatment threshold (P TT )

59. Extending to dichotomous tests Why you would do a test  Cost of treatment and/or failure to treat is high  Test will reduce misdiagnosis  Test has good performance characteristics  Test is used when diagnosis is ambiguous  Pre-test probability is in the middle Why you wouldn’t do a test  The test is expensive  The test is unlikely to help  The test’s performance characteristics are limited  You are already sure or nearly sure of the diagnosis  Pre-test probability is very high or very low

60. Cost of a perfect test  If the test is perfect, the only cost to consider is the cost of the test  T  T is constant at all probabilities

61. Lowest Cost Option  Black line is lowest cost option  Cost of no treatment < Cost of test  Don’t treat (red)  Cost of treatment < Cost of test  Treat (green)  Cost of test < Cost of empiric treatment/no treatment  Test (yellow)

62. Lowest Cost Option  Treat/No Treat threshold  P = C / (C + B)  Test/Treat threshold  P = 1 – T/C  No Treat/Test threshold  P = T/B

63. Example: EGFR in NSCLC  EGFR by PCR and fragment analysis for non-small cell lung cancer (NSCLC)  Cost of test: $400  Assume sensitivity and specificity = 100%  Prevalence of EGFR mutation (19% in males to 26% in females)  C = Cost of erlotinib  $1300/mo x 3 months + risk of rash, diarrhea, occasional interstitial pneumonitis ~ $4,000  B = Benefits of erlotinib in EGFR positive patients  3.2 months of progression free survival – cost of drug ~ $11,000

64. Example: EGFR in NSCLC

65. Imperfect tests  Must consider residual probability of being wrong  Lots of complex algebra (covered in the book)  Can also use Excel spreadsheet  Available on course website  http://rds.epi- ucsf.org/ticr/syllabus/courses/4/2011/10/06/Lecture/tools/Treat ment_Testing_Thresholds_Galanter.xls  Example Example

66. Summary  A dichotomous test is a test with two possible outcomes  We covered the definitions of sensitivity, specificity, prevalence, PPV, NPV, & accuracy  What you thought before + New information = What you think now  Probabilities can be updated using a 2 x 2 table  Odds are the ratio of two portions, Probabilities the proportion from the whole  Likelihood ratios convert prior odds to posterior odds given a test result  Treatment and testing thresholds allow you to estimate the lowest cost option