1. Automatic Lung Nodule Detection Using Deep Learning and Hand Crafted Features
Tulane University
University of Central Florida
Problem Overview
Hand Crafted Features
Results
Bag of Frequencies
For 80% training, 20% testing, the percent accuracy is shown below. Negative accuracy reflects the number of true negatives, positive accuracy reflects the number of true positives
Parameter
Description
Example
Number of slices
#of 2D surfaces extracted from nodule
10  [4 slices in XY plane, slices in YZ plane, slices in ZX plane]
Size of slices
Important if sampling outside of radius
1.2  slices will be 120% the size of the nodule
Slice below is 40x40 Radius is 17
Sample Radii
Determines #and location of intensity profiles in relation to the radius
[.5, .9, 1.05]  three intensity profiles will be taken at 50%, 90% and 105% of the radius
Each radius is shown in a separate color (RBG) in the image below
Number of intensity Samples
# of points sampled; determines angles b/t points of intensity profile
8  points will be taken
every 45 degrees, counter clockwise (as
shown; first sample is black, last is bright)
Characterizes nodules based on morphology and obtains feature vectors from frequency based spectral signatures
Input:
Whole image
Nodule radius
Nodule center coordinates
Output:
Feature vector of frequency profile from intensities
Take 2D surfaces from 3D voxel
Sample intensities at each given radius
Get spectral signatures from intensity profiles
missed lung cancer
cancer detected by CAD
false positive from CAD
Motivation
Lung Cancer is the leading cause of cancer-related death worldwide
30-35% of lung nodules are missed by radiologists during lung cancer screening
Early detection of cancer improves mortality and morbidity
Estimated Radius
Annotated Radius
Above: Bag of Frequencies sampling being used to estimate nodule radius (in yellow)
Objective
Improve lung cancer screening by developing a novel, accurate, and efficient CAD system.
Dataset
Taxonomic Diversity & Taxonomic Distinctness
LIDC/IDRI Dataset, publicly available
LUng Nodule Analysis (LUNA)
Annotations
Positive findings: locations of nodules >= 3mm accepted by at least 3 out of 4 radiologists
Negative findings: candidates from 3 existing detection algorithms that were rejected by radiologists
None
Index 0 0 Voxels
Index 1 0 Voxels
63 1 Voxel
43 2 Voxel
Index 2
0 Voxels
42 2 Voxel
Index 107
29 2 Voxel
Voxel
Intensity(HU):
Abundance:
Estimated Radius
Annotated Radius
Characterizes nodules based on texture and patterns, analyzing intensity distribution/relationships utilizing phylogenetic trees (from ecology)
Input:
Whole image
Nodule radius
Nodule center coordinates
Output:
Feature vector of taxonomic indices from ROIs
Determine Regions of Interest
rings, spheres, whole nodule
Cast intensities into a tree for each ROI
Each HU(intensity) is a leaf
Each leaf has a property: # of voxels with that intensity (abundance)
Calculate Taxonomic Indices using
Distance between intensities in the tree of intensities
Abundance of intensity values
Deep Learned Features
Discussion
Above: an example tree; left nodes = height, for simpler calculations; right nodes = intensity value (in Hounsfield Units); Each node has a property, the number of voxels with that intensity
Nodule radius estimation average error: 1.29 pixels
Estimation provides meaningful features for classification;
improving accuracy will improve results to closer to annotated radius results
Simplified illustration of CNN architectures of GoogLeNet
Pretrained Network: GoogLeNet, “Inception-v3”
Current state-of-the-art CNN architecture for the ILSVRC challenge trained on ImageNet
Two convolution layers, two pooling layers, and nine “Inception” layers.
Inception layers of GoogLeNet consist of six convolutional layers with different kernel sizes and one pooling layer (shown above)
Inception-v3: concatenates filters of different sizes and dimensions into a single new filter
Nodule Radius Estimation
Future Work
Iterative algorithm to estimate nodule radius based on the assumption that the nodule lies between bright intensities inside the nodule, and darker intensities outside the nodule.
Creating a pipeline to improve time efficiency
Combining with custom neural network
Compare with commercial software
Evaluate and test the software in an independent cohort
= ending point
= sampled radius
= starting endpoint
Iteration 1: sampled radius lies inside the nodule; start = sampled radius
Iteration 2: sampled radius lies outside the nodule; end = sampled radius
References
Armato III, S. G., McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., ... & Kazerooni, E. A. (2011). The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical physics, 38(2),
Ciompi, F., Jacobs, C., Scholten, E. T., Wille, M. W., de Jong, P. A., Prokop, M., & van Ginneken, B. (2015). Bag-of-Frequencies: A Descriptor of Pulmonary Nodules in Computed Tomography Images. IEEE Transactions On Medical Imaging, 34(4),
de Carvalho Filho, A. O., Silva, A. C., de Paiva, A. C., Nunes, R. A., & Gattass, M. (2016). Lung-Nodule Classification Based on Computed Tomography Using Taxonomic Diversity Indexes and an SVM. Journal of Signal Processing Systems, 1-18.
Shin, H. C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., ... & Summers, R. M. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging, 35(5),
A B
Input:
Image Patch
calculate:
start = image center end = image edge
Output:
Estimated Radius
Samples taken using BoF
At radius halfway between start and end
Pixel value threshold applied:
sampled intensity is classified as inside/outside of nodule
Percentage threshold applied:
Sampled radius is classified as inside/outside of nodule
Radius inside nodule:
(sample was too small)
End = previous end
Start = sampled radius
Radius outside nodule:
(sample was too big)
Start = previous start
End = sampled radius
Above: the difference, in pixels, between the estimation and the true radius; shown out of 1000 nodule patches
Acknowledgements
NSF-REU at the University of Central Florida is greatly appreciated for funding support. Special thanks to Dr. Shah and Dr. Lobo.
Illustration of inception 3a layer of GoogLeNet.