1. Use of Active Learning for Selective Annotation of Training Data in a Supervised Classification System for Digitized Histology Scott Doyle 1, Michael Feldman 2, John Tomaszewski 2, Anant Madabhushi 1 1 Department of Biomedical Engineering, Rutgers, The State University of New Jersey 2 Department of Surgical Pathology, University of Pennsylvania http://lcib.rutgers.edu

2. Outline  Background  Digital Prostate Histopathology  Supervised Classification  Active Learning  Methodology  Active Learning  Data Description  Experimental Setup  Experimental Results  Concluding Remarks

3. Prostate Cancer Detection  ~1 million biopsies per year in USA  10-12 tissue samples per biopsy  80% benign diagnosis  Large amount of data to analyze

4. Computer-Aided Diagnosis  Identifies regions of interest / suspicion  Quantitative  Automated  Reduces variability  Supervised classification system Doyle, S., Feldman, M., Tomaszewski, J., Madabhushi, A. “A Hierarchical Computer-aided Classification Scheme for Automated Detection of Prostatic Adenocarcinoma from Digitized Histology,” APIII 2006

5. Supervised Classification  Expert segmentation for training  Histopathology:  Expensive, time-consuming to annotate  Cost per training sample is high

6. Supervised Classification  Random training inefficient  Possible redundancy with existing training  No guarantee of improved accuracy

7. Solution: Active Learning  Choose training samples intelligently, not randomly  Increased accuracy per training sample  Forced choice of training, maximized accuracy  Useful where:  Large amount of unlabeled data  Annotations are expensive  Ideally suited for histopathology data

8. Active Learning Classifier Performance Accuracy # of Training Samples Random Learning Active Learning

9. Previous Work  Liu [2004], Vogiatzis and Tsapatsoulis [2006]  Gene microarray data  Yao, et al [2008]  Content-based image retrieval  Little work done in histopathology with Active Learning

10. Outline  Background  Digital Prostate Histopathology  Supervised Classification  Active Learning  Methodology  Active Learning  Data Description  Experimental Setup  Experimental Results  Concluding Remarks

11. Build Classifier Active Learning Methodology Cancer Non-cancer Uncertain Classification Obtained from pathologist Training Data Labeled Unlabeled Build Classifier Classify Unlabeled Training

12. Active Learning Methodology Uncertain Classification Informative Samples Certain Classification Uninformative Obtain Expert LabelsCombine With Original Set Eliminate, labeling these adds no information + Identify Informative Regions

13. Active Learning Methodology Generate New ClassifierNew Training Set

14. Feature Extraction Cancer Region Original Image Feature Images

15. Classification Feature ImagesC4.5 Decision Tree Doyle, S., Madabhushi, A., Feldman, M., Tomaszeweski, J.: A Boosting Cascade for Automated Detection of Prostate Cancer from Digitized Histology, MICCAI, Lecture Notes in Computer Science, Vol. 4191, pp. 504-511, 2006. “Random Forest” [Brieman, 2001] Majority voting determines classification

16. Image Data Description  27 H&E stained digital biopsy samples  Data breakdown:  Initial Training Set  Unlabeled Training Set  Testing Set  Active Learning drawn from Unlabeled Training  Groups rotated so all images are tested

17. Classification  Three training groups evaluated:  Initial set:  Active Learning set:  Random Learning set: Initial Training Active Learning Initial Training Random Learning Initial Training + +

18. Outline  Background  Digital Prostate Histopathology  Supervised Classification  Active Learning  Methodology  Active Learning  Data Description  Experimental Setup  Experimental Results  Concluding Remarks

19. Results: Qualitative Original ImageRandom LearningActive Learning

20. Results: Qualitative Random Learning Active Learning

21. Results: Qualitative Original ImageRandom LearningActive Learning

22. Results: Qualitative Active LearningRandom Learning

23. Quantitative Evaluation

26. Outline  Background  Digital Prostate Histopathology  Supervised Classification  Active Learning  Methodology  Active Learning  Data Description  Experimental Setup  Experimental Results  Concluding Remarks

27. Concluding Remarks  Maximize classification accuracy by choosing training intelligently  Efficiently obtain annotations  Make the most use of “training budget”  Build Active Learning into clinical applications  Online training correction / modification  User feedback

28. Acknowledgements  The Coulter foundation (WHCF 4-29368)  New Jersey Commission on Cancer Research  The National Cancer Institute (R21CA127186- 01, R03CA128081-01)  The US Department of Defense (427327)  The Society for Medical Imaging and Informatics