AlexNet+Max Pooling MIL AlexNet+Label Assign. MIL Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification Wentao Zhu, Qi Lou, Yeeleng Scott Vang, Xiaohui Xie University of California, Irvine SUMMARY Max pooling-based deep multi-instance networks cross-entropy cost function Note: Previous methods all require hand-crafted features or manually annotated mass ROIs. End-to-end trained deep multi-instance networks for mass classification. No hand annotated mass ROIs in the whole mammogram classification. Three schemes for deep multi-instance networks. Robust experimental results on the INbreast dataset. ROC Curve on Test Fold 4 Only considers patch of maximum malignant probability 4. Label Assignment-based Multi-instance Learning How about Explicitly assign labels for each patch? Malignant probability for each patch FRAMEWORK 1. Overall framework Assume patches of the maximum k malignant probabilities are malignant patches, label assignment-based deep multi-instance networks cross-entropy cost function Max pooling-based, label assignment-based, sparse deep multi-instance networks significantly improve the performance than that of baseline network, AlexNet. Visualization of Predicted Malignant Probabilities Learn mass region without explicit mass bounding box; sparse multi-instance deep networks provides better results; deep multi-instance networks can be used for weak mass annotation, which provides diagnosis evidence. The k is hard to estimate. A soft or adaptive assumption should be employed. 5. Sparse Multi-instance Learning 2. Mass/mammogram size distribution The mass of average size (329×325) takes 2% of average whole mammogram (1,474×3,086). Considering mass is very small (sparse) compared with whole mammogram, we employ sparsity to replace the above hard constrain. Sparse deep multi-instance networks cross-entropy cost function EXPERIMENTAL RESULTS Histograms of mass height and width INbreast Dataset 5 fold cross validation (3-1-1); data augmentation, flip horizontally, shift within 10% horizontally and vertically, rotate within 45 degree, randomly set 50×50 square box as 0; AlexNet with pretrained weights; totally automatic method without hand-crafted features and human annotated bounding box. Visualization of predicted malignant probabilities for patches. The first row is the resized mammogram. The red rectangle boxes are mass regions from the annotations on the dataset. Max pooling-based, label assignment-based, sparse deep MIL are in the second row, third row, fourth row respectively. Max-pooling-based, label assignment-based deep MIL misclassify some patches. Histograms of mammogram height and width Methods Acc. AUC. Ball et al. 2007 0.87 N/A Varela et al. 2006 0.81 Domingues et al. 2012 0.89 Pretrained CNN (Dhungel et al. 2015) 0.84±0.04 0.69±0.10 Pretrained CNN+Random Forest (Dhungel et al. 2015) 0.91±0.02 0.76±0.23 AlexNet 0.81±0.02 0.79±0.03 AlexNet+Max Pooling MIL 0.85±0.03 0.83±0.05 AlexNet+Label Assign. MIL 0.86±0.02 AlexNet+Sparse MIL 0.90±0.02 0.89±0.04 3. Max Pooling-based Multi-instance Learning The malignant probability of feature space’s pixel (i, j) is CONTACT Wentao Zhu (wentaozhu1991@gmail.com, code https://github.com/wentaozhu/deep-mil-for-whole-mammogram-classification.git). Qi Lou, Yeeleng Scott Vang, Xiaohui Xie ( {qlou, ysvang, xhx}@uci.edu ). We are looking for available large medical image datasets and collaborations.