Introduction to Biomedical Statistics

Signal Detection Theory What do we actually “detect” when we say we’ve detected something?

Signal Detection Theory What do we actually “detect” when we say we’ve detected something? We say we’ve “detected” when a criterion value exceeds a threshold

Signal Detection Theory examples: – the onset of a light or sound –the presence of an abnormality on x-ray

Signal Detection Theory There are 4 possible situations Target is: You Respond: PresentAbsent Present Absent

Signal Detection Theory There are 4 possible situations Hit Target is: You Respond: PresentAbsent Present Absent

Signal Detection Theory There are 4 possible situations Hit Miss Target is: You Respond: PresentAbsent Present Absent

Signal Detection Theory There are 4 possible situations HitFalse Alarm Miss Target is: You Respond: PresentAbsent Present Absent

Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent

Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent This is the total # of Target Present Trials

Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent This is the total # of Target Present Trials

Signal Detection Theory Hit Rate (H) is the proportion of target present trials on which you respond “present”

Signal Detection Theory Notice that H is a proportion, so 1 - H gives you the “miss” rate or…

Signal Detection Theory False-Alarm Rate (FA) is the proportion of target absent trials on which you respond “present”

Signal Detection Theory Notice that FA is a proportion. 1 minus FA gives you the correct rejections or …

Signal Detection Theory Signal Detection can be modeled as signal + noise with some detection threshold

Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Absent

Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Present trials contain a little bit extra intensity contributed by the signal Target Absent Target Present

Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Present trials contain a little bit extra intensity contributed by the signal Target Absent This is the signal’s contribution Target Present

Signal Detection Theory We can imagine a static criterion above which we’ll respond “target is present” Stimulus Intensity Frequency Target Absent Target Present Criterion

Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Hits Criterion

Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Misses Criterion

Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion False Alarms Criterion

Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Correct Rejections Criterion

Signal Detection Theory Notice that as H increases, FA also increases Stimulus Intensity Frequency H Criterion

Signal Detection Theory Notice that as H increases, FA also increases Stimulus Intensity Frequency FA Criterion

Signal Detection Theory d’ (pronounced d prime) is a measure of sensitivity to detect a signal from noise and does not depend on criterion - it is the distance between the peaks of the signal present and signal absent curves Stimulus Intensity Frequency d’ is computed by converting from H and FA proportions into their corresponding Z scores and subtracting Zinv(FA) from Zinv(H)

Some Common Biomedical Statistics Sensitivity Specificity Positive Predictive Value Negative Predictive Value Likelihood Ratio Relative Risk Absolute Risk Number needed to treat/harm

Sensitivity and Specificity Consider a test for a condition –e.g. Pregnancy test –e.g. Prostate-specific Antigen (Prostate Cancer) –e.g. Ultrasound (Breast Cancer) These are all signal detection problems

Sensitivity and Specificity Four possible situations: Condition is: Test Result: PresentAbsent Present Absent

Sensitivity and Specificity Four possible situations: True Positive Condition is: Test Result: PresentAbsent Present Absent

Sensitivity and Specificity Four possible situations: True Positive False Negative Condition is: Test Result: PresentAbsent Present Absent

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative Condition is: Test Result: PresentAbsent Present Absent

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of Condition Present Cases

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of Condition Absent Cases

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of “positive” tests

Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of “negative” tests

Sensitivity and Specificity Sensitivity is the proportion of condition present cases on which the test returned “positive” Analogous to the hit rate (H) in Signal Detection Theory

Sensitivity and Specificity Specificity is the proportion of condition absent cases on which the test returned “negative” Analogous to the Correct Rejection rate in Signal Detection Theory

Sensitivity and Specificity Notice that 1 minus the Sensitivity is analogous to the FA of Signal Detection Theory

Sensitivity and Specificity Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased

Sensitivity and Specificity Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased Sensitivity and Specificity have a similar relationship: as a cut-off value for a test becomes more stringent the sensitivity goes down and the specificity goes up…and vice versa

Sensitivity and Specificity “For detecting any prostate cancer, PSA cutoff values of 1.1, 2.1, 3.1, and 4.1 ng/mL yielded sensitivities of 83.4%, 52.6%, 32.2%, and 20.5%, and specificities of 38.9%, 72.5%, 86.7%, and 93.8%, respectively.” JAMA Jul 6;294(1):66-70.

Sensitivity and Specificity Likelihood Ratio is the ratio of True Positive rate to False Positive rate Loosely corresponds to d’ in that Likelihood ratio is insensitive to changes in criterion

Sensitivity and Specificity If a test is positive, how likely is it that the condition is present? Positive Predictive Value is the proportion of “positive” test results that are correct

Sensitivity and Specificity Negative Predictive Value is the proportion of “negative” test results that are correct

Sensitivity and Specificity Consider the influence of exposure to some substance or treatment on the presence or absence of a condition e.g. smoking and cancer e.g. aspirin and heart disease

Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo Yes

Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo This is total # exposure No Yes

Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo This is total non-exposure No Yes

Sensitivity and Specificity We can think in terms of “Event Rates” AB CD Disease: Exposure: YesNo Yes No e.g. the proportion of non-smokers who get lung cancer e.g. the proportion of smokers who get lung cancer

Sensitivity and Specificity Relative Risk is the ratio of Exposure Events to Non-Exposure Events Often encountered in regard to rate of adverse reactions to drugs AB CD Disease: Exposure: YesNo Yes No

Sensitivity and Specificity Often we are interested in whether the chance of an event changes with exposure Relative Risk Reduction is the difference between event rates in the exposure and non-exposure groups, expressed as a fraction of the non-exposure event rate

Sensitivity and Specificity Notice that Relative Risk Reduction can be positive or negative: that is, exposure could reduce the risk of some event (e.g. exposure to wine reduces risk of heart disease) or increase the risk (e.g. exposure to cigarette smoke increases risk of heart disease)

Sensitivity and Specificity Notice also that these figures do not take into account the absolute numbers e.g. control event rate =.264 and exposure event rate =.198 e.g. control event rate = and exposure event rate = Both yield the same relative risk reduction of -25%

Sensitivity and Specificity Notice also that these figures do not take into account the absolute numbers e.g. control event rate =.264 and exposure event rate =.198 e.g. control event rate = and exposure event rate = Both yield the same relative risk reduction of -25% Doesn’t discriminate between large and small effects

Sensitivity and Specificity The absolute risk reduction conveys effect size

Sensitivity and Specificity The absolute risk reduction conveys effect size An intuitive version is to consider the reciprocal - the “number needed to treat or harm” Indicates the number of individuals that would have to be exposed to the treatment in order to cause one to have the outcome of interest