1. Patch-based Nonlocal Denoising for MRI and Ultrasound Images Xin Li Lane Dept. of CSEE West Virginia University

2. Outline I come to see and be seen Motivation: nonlocal (symmetry-related) dependency in medical images Technical Approach –Patch-based image modeling and geometric resampling –From locally linear embedding (LLE) to locally linear transform (LLT) –Nonlocal denoising algorithm Experimental results –Synthetic images, Gaussian noise –MRI images, Rician noise –Ultrasound images, speckle noise

3. Big Picture: Computational Imaging Quality Cost Physical Examples: SMASH/SENSE for fast MRI Super-resolution in PET imaging High-dynamic-range (HDR) imaging Computational

4. Motivation: Modeling Human-related Prior Bilateral symmetryShape boundary regularity

5. Patch-based Image Modeling To overcome the curse of dimensionality, we have to work at the middle ground between pixel- level and image-level An old concept with renewing interest –Vector quantization is patch-based, JPEG used 8-by-8 patches (SP community) –Patch-based recognition (CV community) –Nonlinear dimensionality reduction (ML community) P P Nonparametric: patch-based vs. Parametric: wavelet-based

6. Nonlocal Dependency reflective symmetry translational symmetry Beyond the reach of any localized models (MRF, wavelet-based, PDE-based)

7. Redundant Representation by Geometric Resampling fliplr(x)x flipud(x)flipud(fliplr(x)) Collection of P-by-P patches

8. Exploiting Manifold Constraint B4B4 B2B2 B3B3 B0B0 B1B1 RPPRPP Nonlinear Dimensionality Reduction By Locally Linear Embedding (LLE) Roweis and Saul, Science’2000 Sparsifying transform t Artificial third dimension t records the location information

9. Nonlocal Sparse Representation (NSR) Approximated solution (3D FFT/DCT) Optimal sparsifying transform (KLT) B0B0 BkBk B1B1 Pack into 3D Array D 3D-FFT Thresholding … 3D-IFFT Pack into 3D Array D B0B0 BkBk B1B1 … ^ ^^

10. NSR Image Denoising Algorithm

11. Experimental Results on NSR Computer-generated toy images, additive White Gaussian noise –Illustrate the algorithm procedure and verify the benefit of resampling MRI images, Rician noise –Benchmark: PDE-based scheme (total- variation denoising) Ultrasound images, speckle noise –Benchmark: local schemes (SRAD, SBF, PDE)

12. Denoising Procedure Illustration by Toy Example Noisy image Search similar patches Noisy 3D array LLT Thresholding denoised 3D array Denoised imagedenoised patches

13. Benefit of Resampling Translation only Translation and 1 reflection Translation and 2 reflections Translation and 3 reflections original noisy NSR (ISNR=17.5dB) GSM (ISNR=13.3dB) GSM: Gaussian Scalar Mixture in Wavelet space (state-of-the-art denoising scheme)

14. MRI Image Denoising original Noisy (Rician,  =30) PDE scheme NSR scheme

15. Ultrasound Despeckling Field-II Simulation SBF(local) NSR (nonlocal) Ultrasound Despeckling Assessment Index (USDSAI)* *Tay, P.C.; Acton, S.T.; Hossack, J.A., “A stochastic approach to ultrasound Despeckling,”ISBI’2006

16. Other (Non-medical) Applications of Nonlocal Sparse Representation originalRandomly -sampled (20% data) RUP Scheme* griddata scheme EM+NSR scheme *Candes, E.J.; Romberg, J.; Tao, T., “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency Information,” IEEE Trans. on Infor. Theory, pp. 489- 509, Feb. 2006

17. Concluding Remarks Symmetry – an important piece of prior information about human subjects Patch-based models enable us to better distinguish signal (pattern of interest) from noise using the tool of nonlocal sparsity Our experiments have shown the effectiveness of such models in a variety of imaging modalities and noise conditions Interest in NIH RFP: Innovations in Biomedical Computational Science and Technology (R01)