1. Low Dose CT Image Denoising Using WGAN and Perceptual Loss
Qingsong Yang, Pingkun Yan, Ge Wang
Biomedical Imaging Center, CBIS/BME, RPI
Nov 19, 2017

2. Statistical Reconstruction
Low Dose CT
Low Dose FBP Recon
Full Dose Statistical Recon
Reduced X-ray Exposure
Increased Noise and Artifacts
Noise Suppressing Methods:
Sinogram
Reconstructed Image
FBP
Statistical Reconstruction

3. General flowchart of denoising networking training process
Deep Neural Network G
Low dose CT image
Difference/Errors
Full dose/Noiseless CT image
General flowchart of denoising networking training process

4. Normal Dose Low Dose ASD-POCS KSVD BM3D CNN
1 red arrow indicates a small structural detail, maybe a lesion, maybe a calcification. Only the result from CNN, we can see it. All the other methods smooth it. You can enlarge more.
2 blue arrow indicates a region between two material with high attenuation coefficient. ASD-POCS has blocky effect. None of the methods except CNN can effectively eliminate the streak-like artifacts.
3 green arrow also indicates a region that there is no artifacts in CNN’s result but all other methods still have obvious artifacts.
Chen, Hu, et al. "Low-dose CT via convolutional neural network." Biomedical optics express 8.2 (2017):

5. A RED-CNN network using paired convolutional and de-convolutional layers for low dose CT denoising
Chen, Hu, et al. "Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)." arXiv preprint arXiv:  (2017).

6. A wavelet domain deep convolutional neural network architecture for low-dose CT denoising
Kang, Eunhee, Junhong Min, and Jong Chul Ye. "A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction." Medical Physics 44.10 (2017).

7. Normal-dose TV-POCS K-SVD
BM3D WaveNet RED-CNN
Chen, Hu, et al. "Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)." arXiv preprint arXiv:  (2017).

8. MSE Loss: pixel-wise errors
Perceptual Loss: errors in a defined feature space
Deep Neural Network G
Low dose CT image
Difference/Errors
Full dose/Noiseless CT image

9. Proposed by Visual Geometry Group, University of Oxford
Very deep neural network
Trained on natural images for image classification
VGG-19 network
Network structure with perceptual loss.

10. Network Training MGH dataset Training dataset VGG extractor
GE Discovery CT750HD
Over 40 Cadavers’ body CT volumes
Four noise levels: 10NI, 20NI, 30NI, and 40NI
Three reconstruction algorithms: FBP, ASIR and VEO
Training dataset
Inputs: FBP30NI
Labels: VEO10NI
Over 10,000 Image patches 80x80
VGG extractor
VGG_11, VGG_31, VGG_34
FBP30NI
VEO10NI

11. Comparison of zoomed ROI
FBP30NI
VEO30NI
CNN-MSE
Comparison of zoomed ROI
CNN-VGG11
CNN-VGG31
CNN-VGG34
Using perceptual can avoid oversmoothing
Deep VGG layer capture more details
An example of denoising results using MSE loss and different layers of VGG network as feature extractors

12. Generative Adversarial Network - GAN
A game between two players:
Discriminator D
Generator G
D tries to discriminate between:
A sample from the real data
A sample from the generated data
G tries to “trick” D by generating samples that are hard for D to distinguish from real data
Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems

13. Wasserstein GAN - WGAN Pitfall of GAN WGAN No guarantee to equilibrium
The discriminator only gives 0 or 1 but cannot describe how good or bad the image is
WGAN
Wasserstein distance between two data distributions
The discriminator gives a continuous evaluation describe how good or bad the image is
Arjovsky, Martin, and Léon Bottou. "Towards principled methods for training generative adversarial networks." arXiv preprint arXiv:  (2017).
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” arXiv preprint arXiv: , 2017.
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of wasserstein gans,” arXiv preprint arXiv: , 2017.

14. CNN-MSE
CNN-VGG
WGAN-MSE
WGAN-VGG
WGAN
Overall structure of the denoising network.

15. Network Training MGH dataset Training dataset Networks:
GE Discovery CT750HD
Over 40 Cadavers’ body CT volumes
Four noise levels: 10NI, 20NI, 30NI, and 40NI
Three reconstruction algorithms: FBP, ASIR and VEO
Training dataset
Inputs: FBP30NI
Labels: VEO10NI
Over 10,000 Image patches 80x80
Networks:
CNN-MSE / CNN-VGG
WGAN-MSE / WGAN-VGG / WGAN
FBP30NI
VEO10NI

16. Comparison of zoomed in ROI
FBP30NI
VEO30NI
CNN-MSE
CNN-VGG
WGAN
WGAN-VGG
FBP30NI
VEO30NI
CNN-MSE
CNN-VGG
WGAN
WGAN-VGG
Comparison of zoomed in ROI
An example of denoising results using different loss functions

17. Network Training Mayo data Training dataset Networks:
Two noise levels: full dose and simulated quarter dose
FBP reconstruction
Training dataset
Inputs: quarter dose images
Labels: full dose
Over 10,000 Image patches 80x80
Networks:
CNN-MSE / CNN-VGG
WGAN-MSE / WGAN-VGG / WGAN
Quarter dose
Full dose
AAPM, “Low dose ct grand challenge,” [Online]. Available:

18. An example of denoisng result using different loss functions
Full Dose
Quarter Dose
CNN-MSE
PSNR
SSIM
ROI Mean (HU)
ROI Variance (HU)
Full Dose 9 36
Quarter Dose
0.7496 11 74
CNN-MSE
0.7966 12 18
CNN-VGG
0.7926 4 30
WGAN-MSE
0.8090 28
WGAN-VGG
0.7923 31
WGAN
0.7745 23 37
CNN-VGG
WGAN-MSE
WGAN-VGG
WGAN
Quantitative analysis using PSNR and SSIM and statistical properties of a small ROI
An example of denoisng result using different loss functions

19. An example of denoisng result using different loss functions
Full Dose
Quarter Dose
CNN-MSE
PSNR
SSIM
ROI Mean (HU)
ROI Variance (HU)
Full Dose 9 36
Quarter Dose
0.6471
118 38
CNN-MSE
0.7022
120 15
CNN-VGG
0.6972
104 28
WGAN-MSE
0.7122
115 25
WGAN-VGG
0.6759
111 29
WGAN
135 33
CNN-VGG
WGAN-MSE
WGAN-VGG
WGAN
Quantitative analysis using PSNR and SSIM and statistical properties of a small ROI
An example of denoisng result using different loss functions

20. Summary Simple Network Structure Perceptual Loss - Image Content
WGAN Framework – Data Distribution
Outlook - Evaluation