Project Management Risk and Uncertainty Making decisions using ROC (receiver operating characteristic) curves Graham Collins University College London (UCL) graham.collins@ucl.ac.uk

Diagnostic Decisions A doctor examines a breast x-ray, agonising over whether an ambiguous shadow is a tumour. Given incomplete or ambiguous data doctors must determine whether a condition exists or not A major way is using ROC curves, as illustrated by the following example based on Swets et.al., Scientific American, October 2000. What are ROC curves? Math based aids for making decisions could improve diagnoses and often save lives in the process

Step 1 – Sample of population of known status

Step 2 – Calculate the probability for threshold

Step 3 – Construct a ROC curve

Step 4 – Select threshold for yes/no diagnosis

Discussion Select a threshold that delivers a good rate of true positives without generating an unacceptable rate of false negatives ‘Strict thresholds’ limit false positives at the cost of missing many infected individuals Lenient thresholds maximise discovery of affected individuals at a cost of many false positives.

Implications – which is the most appropriate threshold? Which threshold is optimal for a given population depends on such factors such as: Seriousness of condition being diagnosed Prevalence of the condition in the population Availability of corrective measures for those that are diagnosed The financial, emotional and other costs of false alarms.