Low Dose CT Image Denoising Using WGAN and Perceptual Loss Qingsong Yang, Pingkun Yan, Ge Wang Biomedical Imaging Center, CBIS/BME, RPI yangq4@rpi.edu Nov 19, 2017

Statistical Reconstruction Low Dose CT Low Dose FBP Recon Full Dose Statistical Recon Reduced X-ray Exposure Increased Noise and Artifacts Noise Suppressing Methods: Sinogram Reconstructed Image FBP Statistical Reconstruction

General flowchart of denoising networking training process Deep Neural Network G Low dose CT image Difference/Errors Full dose/Noiseless CT image General flowchart of denoising networking training process

Normal Dose Low Dose ASD-POCS KSVD BM3D CNN 1 red arrow indicates a small structural detail, maybe a lesion, maybe a calcification. Only the result from CNN, we can see it. All the other methods smooth it. You can enlarge more. 2 blue arrow indicates a region between two material with high attenuation coefficient. ASD-POCS has blocky effect. None of the methods except CNN can effectively eliminate the streak-like artifacts. 3 green arrow also indicates a region that there is no artifacts in CNN’s result but all other methods still have obvious artifacts. Chen, Hu, et al. "Low-dose CT via convolutional neural network." Biomedical optics express 8.2 (2017): 679-694.

A RED-CNN network using paired convolutional and de-convolutional layers for low dose CT denoising Chen, Hu, et al. "Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)." arXiv preprint arXiv:1702.00288 (2017).

A wavelet domain deep convolutional neural network architecture for low-dose CT denoising Kang, Eunhee, Junhong Min, and Jong Chul Ye. "A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction." Medical Physics 44.10 (2017).

Normal-dose TV-POCS K-SVD BM3D WaveNet RED-CNN Chen, Hu, et al. "Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)." arXiv preprint arXiv:1702.00288 (2017).

MSE Loss: pixel-wise errors Perceptual Loss: errors in a defined feature space Deep Neural Network G Low dose CT image Difference/Errors Full dose/Noiseless CT image

Proposed by Visual Geometry Group, University of Oxford Very deep neural network Trained on natural images for image classification VGG-19 network Network structure with perceptual loss.

Network Training MGH dataset Training dataset VGG extractor GE Discovery CT750HD Over 40 Cadavers’ body CT volumes Four noise levels: 10NI, 20NI, 30NI, and 40NI Three reconstruction algorithms: FBP, ASIR and VEO Training dataset Inputs: FBP30NI Labels: VEO10NI Over 10,000 Image patches 80x80 VGG extractor VGG_11, VGG_31, VGG_34 FBP30NI VEO10NI

Comparison of zoomed ROI FBP30NI VEO30NI CNN-MSE Comparison of zoomed ROI CNN-VGG11 CNN-VGG31 CNN-VGG34 Using perceptual can avoid oversmoothing Deep VGG layer capture more details An example of denoising results using MSE loss and different layers of VGG network as feature extractors

Generative Adversarial Network - GAN A game between two players: Discriminator D Generator G D tries to discriminate between: A sample from the real data A sample from the generated data G tries to “trick” D by generating samples that are hard for D to distinguish from real data Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014.

Wasserstein GAN - WGAN Pitfall of GAN WGAN No guarantee to equilibrium The discriminator only gives 0 or 1 but cannot describe how good or bad the image is https://github.com/soumith/ganhacks WGAN Wasserstein distance between two data distributions The discriminator gives a continuous evaluation describe how good or bad the image is Arjovsky, Martin, and Léon Bottou. "Towards principled methods for training generative adversarial networks." arXiv preprint arXiv:1701.04862 (2017). M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” arXiv preprint arXiv:1701.07875, 2017. I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of wasserstein gans,” arXiv preprint arXiv:1704.00028, 2017.

CNN-MSE CNN-VGG WGAN-MSE WGAN-VGG WGAN Overall structure of the denoising network.

Network Training MGH dataset Training dataset Networks: GE Discovery CT750HD Over 40 Cadavers’ body CT volumes Four noise levels: 10NI, 20NI, 30NI, and 40NI Three reconstruction algorithms: FBP, ASIR and VEO Training dataset Inputs: FBP30NI Labels: VEO10NI Over 10,000 Image patches 80x80 Networks: CNN-MSE / CNN-VGG WGAN-MSE / WGAN-VGG / WGAN FBP30NI VEO10NI

Comparison of zoomed in ROI FBP30NI VEO30NI CNN-MSE CNN-VGG WGAN WGAN-VGG FBP30NI VEO30NI CNN-MSE CNN-VGG WGAN WGAN-VGG Comparison of zoomed in ROI An example of denoising results using different loss functions

Network Training Mayo data Training dataset Networks: Two noise levels: full dose and simulated quarter dose FBP reconstruction Training dataset Inputs: quarter dose images Labels: full dose Over 10,000 Image patches 80x80 Networks: CNN-MSE / CNN-VGG WGAN-MSE / WGAN-VGG / WGAN Quarter dose Full dose AAPM, “Low dose ct grand challenge,” 2017. [Online]. Available: http://www.aapm.org/GrandChallenge/LowDoseCT/#

An example of denoisng result using different loss functions Full Dose Quarter Dose CNN-MSE PSNR SSIM ROI Mean (HU) ROI Variance (HU) Full Dose 9 36 Quarter Dose 19.7904 0.7496 11 74 CNN-MSE 24.4894 0.7966 12 18 CNN-VGG 23.2322 0.7926 4 30 WGAN-MSE 24.0637 0.8090 28 WGAN-VGG 23.3942 0.7923 31 WGAN 22.0168 0.7745 23 37 CNN-VGG WGAN-MSE WGAN-VGG WGAN Quantitative analysis using PSNR and SSIM and statistical properties of a small ROI An example of denoisng result using different loss functions

An example of denoisng result using different loss functions Full Dose Quarter Dose CNN-MSE PSNR SSIM ROI Mean (HU) ROI Variance (HU) Full Dose 9 36 Quarter Dose 18.4519 0.6471 118 38 CNN-MSE 23.2649 0.7022 120 15 CNN-VGG 22.0950 0.6972 104 28 WGAN-MSE 22.7255 0.7122 115 25 WGAN-VGG 22.1620 0.6759 111 29 WGAN 20.9051 135 33 CNN-VGG WGAN-MSE WGAN-VGG WGAN Quantitative analysis using PSNR and SSIM and statistical properties of a small ROI An example of denoisng result using different loss functions

Summary Simple Network Structure Perceptual Loss - Image Content WGAN Framework – Data Distribution Outlook - Evaluation