1. For internal use only / Copyright © Siemens AG 2006. All rights reserved. Multiple-instance learning improves CAD detection of masses in digital mammography Balaji Krishnapuram, Jonathan Stoeckel, Vikas Raykar, Bharat Rao, Philippe Bamberger, Eli Ratner, Nicolas Merlet, Inna Stainvas, Menahem Abramov, and Alexandra Manevitch CAD and Knowledge Solutions (IKM CKS), Siemens Medical Solutions Inc., Malvern PA 19355, USA Siemens Computer Aided Diagnosis Ltd., Jerusalem, Israel

2. Page 2 July-22, 2008 IWDM 2008Vikas Raykar Outline of the talk 1. CAD as a classification problem 2. Problems with off-the-shelf algorithms 3. Multiple instance learning 4. Proposed algorithm 5. Results 6. Conclusions

3. Page 3 July-22, 2008 IWDM 2008Vikas Raykar Typical CAD architecture Candidate Generation Feature Computation Classification Mammogram Location of lesions Focus of the current talk

4. Page 4 July-22, 2008 IWDM 2008Vikas Raykar Traditional classification algorithms region on a mammogramlesionnot a lesion Various classification algorithms Neural networks Support Vector Machines Logistic Regression …. Often violated in CAD Make two key assumtions (1) Training samples are independent (2) Maximize classification accuracy over all candidates

5. Page 5 July-22, 2008 IWDM 2008Vikas Raykar Violation 1: Training examples are correlated Candidate generation produces a lot of spatially adjacent candidates. Hence there are high level of correlations. Also correlations exist across different images/detector type/hospitals. Proposed algorithm can handle correlations.

6. Page 6 July-22, 2008 IWDM 2008Vikas Raykar Violation 2: Candidate level accuracy is not important Several candidates from the CG point to the same lesion in the breast. Lesion is detected if at least one of them is detected. It is fine if we miss adjacent overlapping candidates. Hence CAD system accuracy is measured in terms of per lesion/image/patient sensitivity. So why not optimize the performance metric we use to evaluate our system? Most algorithms maximize classification accuracy. Try to classify every candidate correctly. Proposed algorithm can optimize per lesion/image/patient sensitivity.

7. Page 7 July-22, 2008 IWDM 2008Vikas Raykar Proposed algorithm Specifically designed with CAD in mind: Can handle correlations among training examples. Optimizes per lesion/image/patient sensitivity. Joint classifier design and feature selection. Selects accurate sparse models. Very fast to train and no tunable parameters. Developed in the framework of multiple-instance learning.

8. Page 8 July-22, 2008 IWDM 2008Vikas Raykar Outline of the talk 1. CAD as a classification problem 2. Problems with off-the-shelf algorithms Assume training examples are independent. Minimize classification accuracy. 3. Multiple instance learning 4. Algorithm summary 5. Results 6. Conclusions

9. Page 9 July-22, 2008 IWDM 2008Vikas Raykar Multiple Instance Learning How do we acquire labels ? Candidates which overlap with the radiologist mark is a positive. Rest are negative. 1 1 0 0 0 0 Single Instance Learning 1 0 0 0 0 Multiple Instance Learning Classify every candidate correctly Positive Bag Classify at-least one candidate correctly

10. Page 10 July-22, 2008 IWDM 2008Vikas Raykar Simple Illustration Single instance learning: Reject as many negative candidates as possible. Detect as many positives as possible. Multiple Instance Learning Single Instance Learning Multiple instance learning: Reject as many negative candidates as possible. Detect at-least one candidate in a positive bag.

11. Page 11 July-22, 2008 IWDM 2008Vikas Raykar Outline of the talk 1. CAD as a classification problem 2. Problems with off-the-shelf algorithms Assume training examples are independent. Minimize classification accuracy. 3. Multiple instance learning Notion of positive bags A bag is positive if at-least one instance is positive. 4. Algorithm summary 5. Results 6. Conclusions

12. Page 12 July-22, 2008 IWDM 2008Vikas Raykar Algorithm Details Logistic Regression model feature vector weight vector

13. Page 13 July-22, 2008 IWDM 2008Vikas Raykar Maximum Likelihood Estimator

14. Page 14 July-22, 2008 IWDM 2008Vikas Raykar Prior to favour sparsity If we know the hyperparameters we can find our desired solution. How to choose them?.

15. Page 15 July-22, 2008 IWDM 2008Vikas Raykar Feature Selection

16. Page 16 July-22, 2008 IWDM 2008Vikas Raykar Feature Selection

17. Page 17 July-22, 2008 IWDM 2008Vikas Raykar Outline of the talk 1. CAD as a classification problem 2. Problems with off-the-shelf algorithms Assume training examples are independent. Minimize classification accuracy. 3. Multiple instance learning Notion of positive bags A bag is positive if at-least one instance is positive. 4. Algorithm summary Joint classifier design and feature selection. Maximizes the performance metric we care about. 5. Results

18. Page 18 July-22, 2008 IWDM 2008Vikas Raykar Datasets used Training set 144 biopsy proven malignant-mass cases. 2005 normal cases from BI-RADS 1 and 2 category. Validation set 108 biopsy proven malignant-mass cases. 1513 normal cases from BI-RADS 1 and 2 category.

19. Page 19 July-22, 2008 IWDM 2008Vikas Raykar Patient level FROC curve for the validation set Proposed method is more accurate

20. Page 20 July-22, 2008 IWDM 2008Vikas Raykar MIL selects much fewer features Total number of features81 Proposed MIL algorithm 40 Proposed algorithm without MIL56

21. Page 21 July-22, 2008 IWDM 2008Vikas Raykar Patient vs Candidate level FROC curve Improves per-patient FROC at the cost of deteriorating per-candidate FROC Message: Design algorithms to optimize the metric you care about.

22. Page 22 July-22, 2008 IWDM 2008Vikas Raykar Conclusions A classifier which maximzes the performance metric we care about. Selects sparse models. Very fast. Takes less than a minute to train for over 10,000 patients. No tuning parameters. Improves the patient level FROC curves substantially.

23. Page 23 July-22, 2008 IWDM 2008Vikas Raykar Questions / Comments?