1. Synthesis of X-ray Projections via Deep Learning
Ye Duan
University of Missouri at Columbia

2. Outline Multiview CNN/RNN for shape understanding
Multiview Image Synthesis
Conclusion & Future Work

3. Outline Multiview CNN/RNN for shape understanding
Multiview Image Synthesis
Conclusion & Future Work

4. MV-CNN
Su et al., “Multi-view convolutional neural networks for 3d shape recognition”, ICCV 2015

5. MV-RNN (Truc Le & Ye Duan, Computer & Graphics, 2017)
Convolutional Neural Network
Fully-connected
Long Short-Term Memory
Conditional Random Field

6. MV-RNN: Input K views uniformly distributed Views are ordered
Phong’s shading (light source is always behind the camera)
Image resolution: 128 x 128
Store 2D-3D correspondence

7. CNN Module
Each view shaded image is passed through an identical image-based CNN
Many CNNs can be used:
FCN
DeconvNet
HED (we use this one because it is designed for 2D edge detection)
Output: edge confidence maps

8. Recurrent Neural Network: LSTM Module
Edge probability maps are orderly fed to a two-layer LSTM as sequence
Peep-hole LSTM with 1024 hidden units
Fully-connected layer after the second LSTM
Output: edge images for all views

9. Side-by-side Comparison

10. Side-by-side Comparison

11. Outline Multiview CNN/RNN for shape understanding
Multiview Image Synthesis
Conclusion & Future Work

12. Synthesis of X-ray Projections via Deep Learning

13. Synthesis of X-ray Projections via Deep Learning

14. Synthesis of X-ray Projections via Deep Learning
Collaborated with Dr. Ge Wang and Qingsong Yang.
X-ray imaging has been widely used in medical diagnostics.
The potential risk of x-ray has drawing more and more concerns.
To reduce the x-ray dose, one possible method is to reduce the number of x-ray projections.
We proposed a method that infers the projection image from projections at adjacent angles using deep learning.

15. Synthesis of X-ray Projections via Deep Learning
The proposed method can be used for limited-view CT reconstruction, where only a few projection data are acquired.
The network can generate the missing data from known projections and guide the few-view reconstruction.
The proposed method can be used in x-ray tomosynthesis, in which only two or three projection data are acquired

16. Network Architecture Discriminator 32@256x256 32@128x128 64@64x64

17. Experiments on Laser Scan 3D Data
Result
Ground Trust
Input

18. Experiments on Laser Scan 3D Data
Result
Ground Trust
Input

19. Experiments on Synthetic X-Ray
Result
Ground Trust
Input

20. Experiments on Synthetic X-Ray
Result
Ground Trust
Input

21. Experiments on Synthetic X-Ray
Result
Ground Trust
Input

22. Training details Number of models: 20. Each model contains 72 images.
Image size: 256x256
Batch size: 10
Epoches: 20
Learning rate: 1.0e-4
Adam optimization

23. Image Quality Evaluation
Datasets
PSNR
SSIM
Laser Scan
35.14
0.973
Synthetic X-Ray
34.34
0.969

24. Future Work
Multi-view Image Super Resolution

25. Future Work
3D Volume synthesis

26. Acknowledgements
Truc Le Giang Bui

27. Thank you!