1. Project Management Risk and Uncertainty Making decisions using ROC (receiver operating characteristic) curves Graham Collins University College London (UCL)

2. Diagnostic Decisions
A doctor examines a breast x-ray, agonising over whether an ambiguous shadow is a tumour.
Given incomplete or ambiguous data doctors must determine whether a condition exists or not
A major way is using ROC curves, as illustrated by the following example based on Swets et.al., Scientific American, October 2000.
What are ROC curves?
Math based aids for making decisions could improve diagnoses
and often save lives in the process

3. Step 1 – Sample of population of known status

4. Step 2 – Calculate the probability for threshold

5. Step 3 – Construct a ROC curve

6. Step 4 – Select threshold for yes/no diagnosis

7. Discussion
Select a threshold that delivers a good rate of true positives without generating an unacceptable rate of false negatives
‘Strict thresholds’ limit false positives at the cost of missing many infected individuals
Lenient thresholds maximise discovery of affected individuals at a cost of many false positives.

8. Implications – which is the most appropriate threshold?
Which threshold is optimal for a given population depends on such factors such as:
Seriousness of condition being diagnosed
Prevalence of the condition in the population
Availability of corrective measures for those that are diagnosed
The financial, emotional and other costs of false alarms.