1. Database-Assisted Low-Dose CT Image Restoration Klaus Mueller Computer Science Lab for Visual Analytics and Imaging (VAI) Stony Brook University Wei Xu, Sungsoo Ha and Klaus Mueller

2. Motivation Low-dose CT: * Images from Google.com

3. Motivation Minimize the radiation, while maximize the clarity Enforce better quality directly in the reconstruction process TV-CBCT [J. Xun & S. Jiang] ASD-POCS [E. Sidky & X. Pan] R-OS-SIRT [W. Xu & K. Mueller] Solutions Improve quality in a post- processing de-noising step [Z. Kelm et al.] [H. Yu & G. Wang] [J. Ma & Z. Liang] [W. Xu & K. Mueller]

4. Post-processing De-noising Filter - NLM Neighborhood filters – Non-local Means (NLM) To update pixel x: a mean value of pixels in its search window Weight: by the patch similarity y x Search Window W Central pixel x x’s patch area P x pixel y inside W y’s patch area P y Assumption: there exists a high degree of redundancy to overcome noise by consulting similar patches to average contributions for a more stable outcome

5. Post-processing De-noising Filter - NLM Neighborhood filters – Non-local Means (NLM) x,y,z: spatial variables W : search window, P : patch area around each pixel h : parameter to control the smoothness G a : Gaussian kernel y x

6. NLM’s Results Reduce moderate artifacts Input NLM

7. NLM’s Results But limited for extreme low-dose situation Input NLM TVM

8. What to do now… Information in the input image is not sufficient Extend the search space beyond the current image Utilize prior scans of the same patient: - Z. Kelm, H. Yu & G. Wang, Q. Xu & G. Wang, J. Ma & Z. Liang, W. Xu & K. Mueller - simple, but limited Utilize database of different patients - find reference image and incorporate into the de-noising

9. Reference-based NLM (R- NLM) Compare between central patch and the reference patch Input Ref weight, pixel value y x

10. R-NLM’s Result Input NLM R-NLM Gold Standard Magic ? But…

11. Matched Reference-based NLM (MR-NLM) Input Matched-R Clean-R pixel value weight

12. MR-NLM’s Result Input NLM MR-NLM Magic ? Yes ! Gold Standard

13. Refinement to MR-NLM The refinement to NLM is also applicable to MR-NLM Implement two redundancy control methods Reduce search window redundancy [T. Tasdizen]:  discard unrelated pixels whose mean and variance are different enough Reduce patch redundancy [P. Coupe et al.]:  apply PCA to high-D patch space  project patches to a lower dimensional sub-space Improve not only efficiency but also accuracy

14. Database-Assisted CT Image Restoration (DA-CTIR) Framework

15. Online Database Construction 2D Image Space High-D Image Feature Space Image Scan Global Image Feature Exact as salient local image structure and contextual information Learn the cluster centers of the local features of all images and label them Concatenate local labels to form global descriptor as distinct salient properties of the image

16. Local Image Feature Descriptor In MR-NLM: Input image is low-dose The database contains only high (normal)-dose images Matching is between artifact-free and artifact-contained ones  local feature descriptor should be tolerant to artifact (streak, noise, etc.) and small deformation Scale-Invariant Feature Transform (SIFT) feature Captures histogram of edge orientations in a local neighborhood Scale-invariant, transform-invariant and less sensitive to noise

17. Local Image Feature Descriptor SIFT feature descriptor: Over the neighborhood of size 16  16 dividing to 4  4 blocks In each block, 8-orientation histogram of edges is computed Dense SIFTs over a regular spaced grid: better, robust Grid spacing of 8 pixels, N = 32  32 (64  64) SIFTs for 256 2 (512 2 ) image block 8-bin orientation histogram neighborhood 4  4  8  128-D feature vector

18. Learn visual words To describe one image, the dimension is reduced from 128N to N (N  1024 or 4096). A set of local features {S 0, S 1,.., S N-1 } k-means clustering K cluster centers as visual words {V 0, V 1, …, V K-1 } as visual vocabulary V Local feature vector is assigned to index of the closest visual word Labeling

19. Global Image Feature Descriptor Partition image to multi-resolution to increase the precision Concatenate histograms of labels from each sub-region. Totally, 26K dimensions (K  50 in this paper) A set of labels in fixed grid positions Spatial pyramid based vector quantization Global Image Feature

20. Dimension 2D Image Space High-D Image Feature Space Scan Image Global Image Feature 128k-D per image 1k-D per image 1.3k-D per image 64k-D

21. Online Prior Search 2D Image Space High-D Image Feature Space Target Image M nearest references Support Kd-tree structure (PKD- tree) for fast labeling process, check our paper for details Histogram Intersection Essentially concatenated histograms while not only high-D vector; histogram intersection vs. Euclidean distance

22. Online De-noising Registration FBP De-noised image MR-NLM Target image, M nearest references, Low-dose condition SIFT-flow Tolerant to noise and small deformation Optical-flow to obtain displacement field SIFT instead of pixel Refined MR-NLM Two redundancy controls Fall back to regular NLM for pixel with close to zero normalization factor

23. Experiments Two image databases (not pre-aligned): 48 256 2 head scans - 15 NIH visible human head images - 33 CT cadaver head images 150 512 2 human lung scans from two patients - “give a scan” online database Original reconstructions are utilized as: Basis for low-dose simulation (limited number of projections with noise) Basis for generating target scan (deformed or rotated and then reconstruct with low-dose condition) Gold standard for evaluation Fan-beam geometry

24. Results Head database: low-dose condition: 45-proj SNR 15 Ideal Input Priors DA-CTIR Refined DA-CTIR

25. Results Lung database: low-dose condition: 60-proj SNR 20 Ideal Input DA-CTIR Refined DA-CTIR

26. Future Works PCA reduction to global image feature Larger database for more experiments to verify effectiveness GPU acceleration

27. Questions?