CS 698 | Current Topics in Data Science Dr. Usman Roshan Deep Convolutional Neural Networks for Lung Cancer Detection Paper Presentation | Spring 2018 Fadi G. Farhat February 15th, 2018 New Jersey Institute of Technology

Authors Albert Chon • Peter Lu Niranjan Balachandar Department of Computer Science Stanford University

Introduction: Lung cancer is one of the most common and deadliest cancers 225,000 cases; 150,000 deaths; $12 billion in healthcare costs yearly in the United States Only 17% of people in the U.S. diagnosed with lung cancer survive five years after the diagnosis Current diagnostic methods include biopsies and imaging, such as CT scans Early detection of lung cancer significantly improves the chances for survival; difficult to do with fewer symptoms

Objective: Binary classification problem to detect the presence of lung cancer in patient CT scans of lungs with and without early stage lung cancer Build an accurate classifier using 2D and 3D convolutional neural networks Classifier could speed up and reduce costs of lung cancer screening; allow early detection; improve survival Computer-aided diagnosis (CAD) system will take as input patient chest CT scans, and outputs whether or not the patient has (early stage) or is likely to develop lung cancer

Challenges: CAD system must detect the presence of a tiny nodule (less than 10 mm in diameter for early stage) from a large 3D lung CT scan (around 200 mm x 400 mm x 400 mm) Example of an early stage lung cancer nodule (~5mm) CT scan is filled with noise from surrounding tissues, bone, air, water, blood

Data: Primary dataset: patient lung CT scan dataset from Kaggle’s Data Science Bowl 2017 Labeled data for 2101 patients; divided into training set of 1261, validation set of 420, and test set of 420 Data consists of CT scan data (100 to 400 2D slice images per patient) and a label (0 for no cancer, 1 for cancer); Kaggle dataset does not have labeled nodules!

Data (cont.): Secondary dataset: patient lung CT scan data with labeled nodules from the LUng Nodule Analysis 2016 (LUNA16) Challenge LUNA16 dataset has labeled data for 888 patients; divided into training set of 710, validation set of 178 Data consists of CT scan data and a nodule label (list of nodule center coordinates and diameter)

Approach: Preprocess the 3D CT scans using segmentation, normalization, down-sampling, and zero-centering Train a U-Net (2D Convolutional Networks for Biomedical Image Segmentation) for nodule candidate detection Input regions around nodule candidates detected by the U-net into 3D CNNs to classify the CT scans as positive or negative for lung cancer

Preprocessing & Segmentation: Convert the pixel values in each image to Hounsfield units (HU), a measurement of radiodensity, then stack 2D slices into a single 3D image bone: tissue:

Preprocessing & Segmentation: Use segmentation to mask out the bone, outside air, and other substances that would make the data noisy; retain only lung tissue information Watershed and Thresholding segmentation tested; Thresholding used original thresholding watershed

Preprocessing & Segmentation: Normalize the 3D image by applying linear scaling Down-sample each 3D image by a scale of 0.5 in each of the three dimensions Zero-center the data by subtracting the mean of all the images from the training set

U-Net for Nodule Detection: Find small boxes containing top cancerous nodule candidates Train a modified version of the U-Net on the LUNA16 data Model is trained to output images (256x256) where each output pixel has a value between 0 and 1 indicating the probability the pixel belongs to a nodule Trained U-Net is then applied to the segmented Kaggle CT scan slices to generate nodule candidates

U-Net for Nodule Detection:

U-Net for Nodule Detection:

U-Net for Nodule Detection: U-Net produces a strong signal for the actual nodule, but also produces a lot of false positives U-Net labeled input U-Net predicted output true nodule location

U-Net for Nodule Detection: Solution: Locate top 8 (most active) nodule candidates (32x32x32 volumes) and save them Top sectors not permitted to overlap to prevent them from simply being clustered in the brightest region of the image Combine these sectors into a single 64x64x64 volume and use as input to classifiers, which assign a label (cancer or not cancer)

Malignancy Classifiers: Linear classifier used as a baseline, then a vanilla 3D CNN, and a GoogleNet-based 3D CNN were applied Each classifier used weighted loss (weight for a label is the inverse of the frequency of the label in the training set) CNNs use ReLU activation and droupout after each convolutional layer during training ReLU = Rectified Linear Unit

Malignancy Classifiers (cont.): Vanilla 3D CNN (left) and GoogleNet 3D CNN (right) architectures

Results: Kaggle test set accuracy, sensitivity, specificity, and AUC of ROC Sensitivity: true positive rate Specificity: true negative rate AUC: Area Under the ROC Curve ROC: Receiver Operating Characteristic FPR vs. TPR for diff. cutoff points

Results (cont.): Observation: activations showing that cancerous nodule presence (and location) is detected in some outputs

Conclusions: The deep 3D CNN models, and in particular the GoogleNet-based model, performed the best on the test set State-of-the-art performance AUC of 0.83 not achieved; models performed well considering that less labeled data was used (than most state-of-the-art CAD systems) Current model could be extended to determine the exact location of the cancerous nodules, and not only whether or not the patient has cancer (slide 20)

Future Work: Use Watershed method instead of Thresholding as the initial lung segmentation Make the networks deeper Perform more extensive hyper-parameter tuning Generalize: extend models to 3D images for other cancers

CS 698 | Current Topics in Data Science Dr. Usman Roshan CT Scan Classification for Lung Cancer Detection Course Project | Spring 2018 | Update 1 Fadi G. Farhat April 19th, 2018 New Jersey Institute of Technology

Project Data: Primary dataset: patient lung CT scan dataset from Kaggle’s Data Science Bowl 2017 Labeled data for 1595 patients; divided into training set of 1256, validation set of 141, and test set of 198 Data consists of CT scan images (100 to 500+ 2D slice images per patient) and a label (0 for no cancer, 1 for cancer)

Initial Approach & Baseline: Main challenge: number of slices per patient varies, making it difficult to create a uniform dataset to feed to classifier. Initial approach: take average of all CT scan images per patient and condense into one image. Clearly some information is lost by doing that. Initial approach: Create one flattened numpy array from average image, resulting in a dataset with 1595 rows and 262,144 columns (HU pixel values, not RGB).

Initial Approach & Baseline: Visualizing average image taken from all CT scan slices. Image is still in DICOM format; smaller array than JPG.

Initial Approach & Baseline: To establish baseline, data was classified using SVM and Random Forest; feature selection performed. Linear SVC results (accuracy): 1,000 features (training time: 90 minutes): CV: ZERO Label: 75.00%; ONE Label: 37.84% Test: ZERO Label: 69.50%; ONE Label: 21.05% 10,000 features (training time: 180 minutes): CV: ZERO Label: 66.35%; ONE Label: 35.14% Test: ZERO Label: 71.63%; ONE Label: 38.60%

Initial Approach & Baseline: Random Forest results (accuracy): 1,000 features (training time: 90 minutes): Overall accuracy: 70.70% ZERO Label: 96.15%; ONE Label: 5.40% 10,000 features (training time: 240 minutes): Overall accuracy: 72.34% ZERO Label: 97.16%; ONE Label: 5.26% Random Forest classifier does not seem to discriminate between cancer and non-cancer averaged CT scans.

Convolution Neural Network: Flower Classification 2D CNN results (accuracy): Conv2D → RELU → AveragePooling2D → Flatten → Dense(2) → Softmax Overall accuracy: 73.76% (cross validation) ZERO Label: ~100%; ONE Label: ~0% Training time (50 epochs): 1 Days 14 Hours Flower classification CNN does not seem to discriminate between cancer and non-cancer averaged CT scans.

Convolution Neural Network: Image Classification 2D CNN results (accuracy): Conv2D → RELU → Conv2D → RELU → MaxPooling2D → Dropout(0.25) → Flatten → Dense(128) → RELU → Dropout(0.5) → Dense(2) → Softmax Overall accuracy: 73.76% (cross validation) ZERO Label: ~100%; ONE Label: ~0% Training time (50 epochs): 2 Days 4 Hours Image classification CNN does not seem to discriminate between cancer and non-cancer averaged CT scans.

Next steps: Perform Lung Segmentation before classification and build new dataset [took ~10 hours to create set]: Single Segmented Slice Average Segmented Slice

Next steps: Create set of 9 or 10 chunks per CT scan instead of one:

Next steps: Create flattened dataset based on segmented average slice and run through linear classifiers again. Create 3D image dataset based on segmented chunks and run through 3D CNN [took ~80 hours to create set]: Convolution3D → RELU → MaxPooling3D → Convolution3D → RELU → MaxPooling3D → Convolution3D → RELU → MaxPooling3D → Dropout(0.25) → Flatten → Dense(512) → RELU → Dropout(0.5) → Dense(128) → RELU → Dropout(0.5) → Dense(2) → Softmax

Thursday Morning Update: Dataset based on 10 (non-segmented) and averaged chunks is too large for even 1 TB of RAM! Models running on Ubuntu VM with 32 logical processors and 1 TB of RAM, which is a VM Ware limitation.

Thursday Morning Update: Attempt to run 3D CNN on 3D dataset failed. 1 TB of RAM is consumed before the first epoch is completed!