Compressive Sensing Imaging Pin-Hung Kuo 2016/06/12 Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Introduction Conventional digital sensors follow Shannon’s Nyquist sampling theorem With compressive sensing, a signal can be decoded from incomplete compressive measurements by seeking its sparsity in some domain Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Introduction State-of-the-art method (2DCS)[1,2,3] Exploiting two kinds of prior knowledge of natural images/videos: 1. Piecewise smoothness by total variation(TV) 2. Sparsity in 2D wavelet domain In other words, 2DCS recovers an image I from its random measurements B as: min 𝐼 𝑇𝑉 𝐼 +𝜆 Ψ 2𝐷 (𝐼) 1 𝑠.𝑡. Φ𝐼=𝐵 [1] M. F. Duarte, M. A. Davenport, D. Takhar, and et al. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2):83–91, 2008. [2] M. Lustig, D. Donoho, J. Santos, and J. Pauly. Compressed sensing MRI. IEEE Sig. Proc. Magazine, 2007. [3] S. Ma, W. Yin, Y. Zhang, and A. Chakraborty. An efficient algorithm for compressed MR imaging using total variation and wavelets. In CVPR, 2008. Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

CS by Random Convolution Justin Romberg [4] outlined a framework for CS: convolution with a random waveform followed by random time domain subsampling. The convolution of signal 𝑥 0 and random pule ℎ can be expressed as 𝐻𝑥, where 𝐻= 1 𝑛 𝐹 ∗ Σ𝐹, with F as discrete Fourier matrix, and Σ= 𝜎 1 ⋯ 0 ⋮ ⋱ ⋮ 0 ⋯ 𝜎 𝑛 [4] J. Romberg. Compressive sensing by random convolution. SIAM Journal on Imaging Science, 2009. Compressive Sensing Final Pin-Hung Kuo(郭品宏)

CS by Random Convolution After measurement, 2 kinds of sampling method can be applied: 1. sampling at random location: randomly select m rows from 𝐻 𝑥 0 , i.e., Φ= 𝑅 Ω 𝐻, 𝑤ℎ𝑒𝑟𝑒 Ω =𝑚 2. Randomly pre-modulated summation. With 𝐵 𝑘 = 𝑘−1 𝑛 𝑚 +1,…, 𝑘𝑛 𝑚 𝑘−1 𝑛 𝑚 +1,…, 𝑘𝑛 𝑚 , 𝑘=1,…,𝑚 denoting the index set for block k, and we take a measurement by multiplying the entries of 𝐻 𝑥 0 in Bk by a sequence of random signs and summing. The corresponding row of Φ is then 𝜙 𝑘 = 𝑚 𝑛 𝑡∈ 𝐵 𝑘 𝜖 𝑡 ℎ 𝑡 , Φ=𝑃Θ𝐻, where { 𝜖 𝑝 } 𝑝=1 𝑛 are independent and take a values of ±1 with equal probability Compressive Sensing Final Pin-Hung Kuo(郭品宏)

CS by Random Convolution The system can be seemed as SLM : spatial light modulators Compressive Sensing Final Pin-Hung Kuo(郭品宏)

CS by Random Convolution Although Romberg provided proof, there are experiments by RICE [5] [5] Yin, Wotao, et al. "Practical compressive sensing with Toeplitz and circulant matrices." Visual Communications and Image Processing 2010. International Society for Optics and Photonics, 2010. Compressive Sensing Final Pin-Hung Kuo(郭品宏)

CS by Random Convolution Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS 3-dimensional compressive sensing[6] min 𝐼 𝑇𝑉3𝐷 𝐼 +𝛾𝑃𝑠𝑖3𝐷 𝐼 𝑠.𝑡. Φ 𝐼=𝐵, 𝑤ℎ𝑒𝑟𝑒 Φ =𝑑𝑖𝑎𝑔 Φ 1, Φ 2,…, Φ 𝑇 , 𝑎𝑛𝑑 𝐵=[ 𝐵 1 ; 𝐵 2 ;…; 𝐵 𝑇 ] [6] Shu, Xianbiao, and Narendra Ahuja. "Imaging via three-dimensional compressive sampling (3DCS).", 2011 IEEE International Conference on Computer Vision (ICCV). Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS 3D Total Variation 3D Sparsity Measure 𝑇𝑉3𝐷 𝐼 = 𝐷 1 𝐼 1 + 𝐷 2 𝐼 1 + 𝜌 𝐷 3 𝐼 1 , 𝑤ℎ𝑒𝑟𝑒 𝐷 𝑙 𝑖𝑠 𝑓𝑖𝑛𝑖𝑡𝑒 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟 𝑃𝑠𝑖3𝐷 𝐼 = min 𝑍 , 𝑍 𝜇𝑹𝒂𝒏𝒌 𝒁 +𝜂 𝑍 1 + 𝑍 1 s.t. Ψ 𝑇 𝐼= 𝑍 + 𝑍 , 𝑤ℎ𝑒𝑟𝑒 Ψ =𝑑𝑖𝑎𝑔 Ψ,…,Ψ , 𝑍 = 𝑍 1 ;…; 𝑍 𝑇 𝑎𝑟𝑒 𝑣𝑒𝑐𝑡𝑜𝑟𝑖𝑧𝑒𝑑 𝒁 𝑎𝑛𝑑 𝒁 Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS 2 kinds of Psi3D cost function: Psi3D2 assumes that Rank(I)=1 Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS With , can be rewritten as Given an L-1 norm problem , its augmented Lagrangian function(ALF) is defined as min 𝐼 𝑇𝑉3𝐷 𝐼 +𝛾𝑃𝑠𝑖3𝐷 𝐼 Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS Then, the ALF of eq.(1) is written as Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS 1. solve 2. Update Given a typical l1-norm minimization problem it has closed form solution Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS R=[R_1;…;R_T] Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS Step 4 is , and this equation can be rewritten as Γ𝐼=Θ, where PSF1: 𝛼 1 𝛽 1 [1;−2;1] PSF2: 𝛼 2 𝛽 2 [1,−2,1] PSF3: 𝛼 3 𝛽 3 [1:−2:1] PSF4: γ 𝛽 4 𝛿(𝑥,𝑦) PSF5: 𝛽 5 ℱ −1 ( 𝐻 ∗ .∗𝐻) Compressive Sensing Final Pin-Hung Kuo(郭品宏)

3DCS PSF1: 𝛼 1 𝛽 1 [1;−2;1] PSF2: 𝛼 2 𝛽 2 [1,−2,1] Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Ck Bk: minimize the first frame wavelet sparsity and subsequent frame difference Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS For single image CS, it can be implemented by similar method [7] [6] Shu, Xianbiao, Jianchao Yang, and Narendra Ahuja. "Non-local compressive sampling recovery." IEEE International Conference on Computational Photography (ICCP), 2014 Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS Non-local patch (BM3D[7]) [7] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising with block-matching and 3D filtering,” Proc. SPIE Electronic Imaging '06, no. 6064A-30, San Jose, California, USA, January 2006. Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS 2DCS 3DCS Non-local Wavelet Sparsity Non-local Joint Sparsity Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS IGi Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

NLCS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Test images PSNR NLWS NLJS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experimental Results Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experiment , where we tried Ψ 3𝐷 as wavelet and discrete cosine transform Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Outline Introduction Compressive Sensing by Random Convolution Three Dimensional Compressive Sensing (3DCS) Non-Local Compressive Sampling Recovery (NLCS) Experimental Results Conclusion Compressive Sensing Final Pin-Hung Kuo(郭品宏)

Conclusion & Comments These 2 works adopted 3D sparsity in wavelet transform domain The 3DCS method may very sensitive to motions DWT+DCT seems to be a bad idea Compressive Sensing Final Pin-Hung Kuo(郭品宏)