1. Compressive Sensing Imaging
Pin-Hung Kuo
2016/06/12
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

2. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

3. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

4. Introduction
Conventional digital sensors follow Shannon’s Nyquist sampling theorem
With compressive sensing, a signal can be decoded from incomplete compressive measurements by seeking its sparsity in some domain
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

5. Introduction State-of-the-art method (2DCS)[1,2,3]
Exploiting two kinds of prior knowledge of natural images/videos:
1. Piecewise smoothness by total variation(TV)
2. Sparsity in 2D wavelet domain
In other words, 2DCS recovers an image I from its random measurements B as:
min 𝐼 𝑇𝑉 𝐼 +𝜆 Ψ 2𝐷 (𝐼) 1 𝑠.𝑡. Φ𝐼=𝐵
[1] M. F. Duarte, M. A. Davenport, D. Takhar, and et al. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2):83–91, 2008.
[2] M. Lustig, D. Donoho, J. Santos, and J. Pauly. Compressed sensing MRI. IEEE Sig. Proc. Magazine, 2007.
[3] S. Ma, W. Yin, Y. Zhang, and A. Chakraborty. An efficient algorithm for compressed MR imaging using total variation and wavelets. In CVPR, 2008.
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

6. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

7. CS by Random Convolution
Justin Romberg [4] outlined a framework for CS: convolution with a random waveform followed by random time domain subsampling.
The convolution of signal 𝑥 0 and random pule ℎ can be expressed as 𝐻𝑥, where 𝐻= 1 𝑛 𝐹 ∗ Σ𝐹, with F as discrete Fourier matrix, and
Σ= 𝜎 1 ⋯ 0 ⋮ ⋱ ⋮ 0 ⋯ 𝜎 𝑛
[4] J. Romberg. Compressive sensing by random convolution. SIAM Journal on Imaging Science, 2009.
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

8. CS by Random Convolution
After measurement, 2 kinds of sampling method can be applied:
1. sampling at random location: randomly select m rows from 𝐻 𝑥 0 , i.e., Φ= 𝑅 Ω 𝐻, 𝑤ℎ𝑒𝑟𝑒 Ω =𝑚
2. Randomly pre-modulated summation. With 𝐵 𝑘 = 𝑘−1 𝑛 𝑚 +1,…, 𝑘𝑛 𝑚 𝑘−1 𝑛 𝑚 +1,…, 𝑘𝑛 𝑚 , 𝑘=1,…,𝑚 denoting the index set for block k, and we take a measurement by multiplying the entries of 𝐻 𝑥 0 in Bk by a sequence of random signs and summing. The corresponding row of Φ is then 𝜙 𝑘 = 𝑚 𝑛 𝑡∈ 𝐵 𝑘 𝜖 𝑡 ℎ 𝑡 , Φ=𝑃Θ𝐻, where { 𝜖 𝑝 } 𝑝=1 𝑛 are independent and take a values of ±1 with equal probability
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

9. CS by Random Convolution
The system can be seemed as
SLM : spatial light modulators
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

10. CS by Random Convolution
Although Romberg provided proof, there are experiments by RICE [5]
[5] Yin, Wotao, et al. "Practical compressive sensing with Toeplitz and circulant matrices." Visual Communications and Image Processing International Society for Optics and Photonics, 2010.
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

11. CS by Random Convolution
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

12. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

13. 3DCS 3-dimensional compressive sensing[6] min 𝐼 𝑇𝑉3𝐷 𝐼 +𝛾𝑃𝑠𝑖3𝐷 𝐼
𝑠.𝑡. Φ 𝐼=𝐵, 𝑤ℎ𝑒𝑟𝑒 Φ =𝑑𝑖𝑎𝑔 Φ 1, Φ 2,…, Φ 𝑇 , 𝑎𝑛𝑑 𝐵=[ 𝐵 1 ; 𝐵 2 ;…; 𝐵 𝑇 ]
[6] Shu, Xianbiao, and Narendra Ahuja. "Imaging via three-dimensional compressive sampling (3DCS).", 2011 IEEE International Conference on Computer Vision (ICCV).
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

14. 3DCS 3D Total Variation 3D Sparsity Measure
𝑇𝑉3𝐷 𝐼 = 𝐷 1 𝐼 𝐷 2 𝐼 1 + 𝜌 𝐷 3 𝐼 1 , 𝑤ℎ𝑒𝑟𝑒 𝐷 𝑙 𝑖𝑠 𝑓𝑖𝑛𝑖𝑡𝑒 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
𝑃𝑠𝑖3𝐷 𝐼 = min 𝑍 , 𝑍 𝜇𝑹𝒂𝒏𝒌 𝒁 +𝜂 𝑍 𝑍 s.t. Ψ 𝑇 𝐼= 𝑍 + 𝑍
, 𝑤ℎ𝑒𝑟𝑒 Ψ =𝑑𝑖𝑎𝑔 Ψ,…,Ψ , 𝑍 = 𝑍 1 ;…; 𝑍 𝑇 𝑎𝑟𝑒 𝑣𝑒𝑐𝑡𝑜𝑟𝑖𝑧𝑒𝑑 𝒁 𝑎𝑛𝑑 𝒁
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

15. 3DCS 2 kinds of Psi3D cost function: Psi3D2 assumes that Rank(I)=1
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

16. 3DCS With , can be rewritten as
Given an L-1 norm problem , its augmented Lagrangian function(ALF) is defined as
min 𝐼 𝑇𝑉3𝐷 𝐼 +𝛾𝑃𝑠𝑖3𝐷 𝐼
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

17. 3DCS Then, the ALF of eq.(1) is written as Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

18. 3DCS 1. solve 2. Update Given a typical l1-norm minimization problem
it has closed form solution
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

19. 3DCS
R=[R_1;…;R_T]
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

20. 3DCS Step 4 is , and this equation can be rewritten as Γ𝐼=Θ, where
PSF1: 𝛼 1 𝛽 1 [1;−2;1]
PSF2: 𝛼 2 𝛽 2 [1,−2,1]
PSF3: 𝛼 3 𝛽 3 [1:−2:1]
PSF4: γ 𝛽 4 𝛿(𝑥,𝑦)
PSF5: 𝛽 5 ℱ −1 ( 𝐻 ∗ .∗𝐻)
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

21. 3DCS PSF1: 𝛼 1 𝛽 1 [1;−2;1] PSF2: 𝛼 2 𝛽 2 [1,−2,1]
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

22. Experimental Results
Ck Bk: minimize the first frame wavelet sparsity and subsequent frame difference
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

23. Experimental Results
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

24. Experimental Results
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

25. Experimental Results
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

26. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

27. NLCS For single image CS, it can be implemented by similar method [7]
[6] Shu, Xianbiao, Jianchao Yang, and Narendra Ahuja. "Non-local compressive sampling recovery." IEEE International Conference on Computational Photography (ICCP), 2014
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

28. NLCS Non-local patch (BM3D[7])
[7] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising with block-matching and 3D filtering,” Proc. SPIE Electronic Imaging '06, no. 6064A-30, San Jose, California, USA, January 2006.
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

29. NLCS 2DCS 3DCS Non-local Wavelet Sparsity Non-local Joint Sparsity
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

30. NLCS
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

31. NLCS
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

32. NLCS
IGi
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

33. NLCS
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

34. NLCS
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

35. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

36. Experimental Results Test images PSNR NLWS NLJS
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

37. Experimental Results
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

38. Experiment
, where we tried Ψ 3𝐷 as wavelet and discrete cosine transform
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

39. Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

40. Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

41. Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

42. Experiment NLJSW3D NLJS NLJSDCT NLWS Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

43. Outline Introduction Compressive Sensing by Random Convolution
Three Dimensional Compressive Sensing (3DCS)
Non-Local Compressive Sampling Recovery (NLCS)
Experimental Results
Conclusion
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)

44. Conclusion & Comments
These 2 works adopted 3D sparsity in wavelet transform domain
The 3DCS method may very sensitive to motions
DWT+DCT seems to be a bad idea
Compressive Sensing Final
Pin-Hung Kuo(郭品宏)