1. Compressed sensing Carlos Becker, Guillaume Lemaître & Peter Rennert
3D Digitization Course
Carlos Becker, Guillaume Lemaître & Peter Rennert

2. Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert
Outline
Introduction and motivation
Compressed sensing and reconstruction workflow
Applications: MRI and single-pixel camera
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

3. What is compressed sensing? Signal sparsity
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

4. What is compressed sensing? Why do we care about sparsity?
Original 1 Megapixel image
Non-sparse values
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

5. What is compressed sensing? Why do we care about sparsity?
But, in the wavelet domain we get these coefficients:
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

6. What is compressed sensing? Why do we care about sparsity?
And the histogram of those coefficients is:
The image is a nearly sparse in the wavelet domain…
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

7. What happens if we only keep the highest coefficients in the wavelet domain, set the rest to zero and reconstruct the image ?
Reconstructed image
(only 50k highest wavelet coefficients)
Original image
95% of the original image data was discarded
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

8. What is compressed sensing?
Real world signal x(t), sampled into a discrete version x[n]
Conservative approach:
Nyquist-Shanon sampling theorem:
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

9. What is compressed sensing?
So, why would we sample 1 million pixels if we are going to throw away 95% of image data when compressing the image in JPEG?
Candès et al. showed that it is possible to subsample a signal if it is sparse in some domain, being able to obtain a perfect reconstruction if certain conditions are met.
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

10. What is compressed sensing?
The Candès-Romberg-Tao Framework
Consider the case that we want to estimate a signal x
Suppose we know that x, with dimension N, is sparse in some domain, with at most K non-zero coefficients in that domain (ie: the wavelet domain)
Then, we need to measure least K log(N/K) random samples of x to be able to reconstruct the signal x perfectly.
In the previous case, this means that we could reconstruct an approximation to the original image with only 66,000 measurements (15x subsampling factor)
Candès, E.J.; Romberg, J.; Tao, T.: “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information” (2004)
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

11. What is compressed sensing?
Classic approach to compression:
Measure everything (ie: all pixels)
Apply some compression algorithm (ie: JPEG2000)
But, why would we sample 1 million pixels if we are going to throw away 90% of image data when compressing the image in JPEG?
Compressed sensing approach: if signal is sparse in some domain
Sample M << N random measurements
Reconstruct original signal by L1 minimization
Full-resolution image
(N pixels/measurements)
Lossy compression
Candès et al. showed that it is possible to subsample a signal if it is sparse in some domain, being able to obtain a perfect reconstruction if certain conditions are met.
Random sampling
(M << N measurements)
Image reconstruction
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

12. Compressed Sensing Motivation from mri
2004, Candes came to results that people of his time could not believe
For a simple phantom (a) its possible to sample at only 22 radial lines (b) (equal to a sampling rate of π / 22, about 50 times below the Nyquist rate of 2 π) to retrieve a perfect reconstruction (d)
What does the trick? Simply setting the unknown Fourier coefficients to 0 leads to a very bad result (c)
Candès, E.J.; Romberg, J.; Tao, T.: “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information” (2004)
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

13. Compressed sensing Reconstruction Workflow
Sparse signal gets randomly sampled in another non-sparse domain (k-space)
Reconstruction leads to noisy non-sparse signal with significant peaks where original signal was high
After thresholding of significant peaks the strongest components of the original signal are detected
Using the noisy reconstruction of the newly sampled strongest components in k-space, the impact of this strongest components on the first reconstruction are determined and subtracted, leaving peaks of weaker components
With this iterative strategy weaker and weaker components can be retrieved
Michael Lustig, David Donoho, John M. Pauly: “Sparse MRI: The application of compressed sensing for rapid MR imaging” (2007)
DL Donoho, I Drori, Y Tsaig : Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit” (2006)
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

14. Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

15. Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

16. Compressed sensing Rapid MRI – non-sparse Signal
(Signal here means the underlying image that is sensed in the Fourier space)
Non- sparse signal sampled in sparse domain
That means: reconstruction of samples will produce no significant peaks (since there are no outstanding peaks in the signal domain)
Solution: use other sparse domain of signal for “reconstruction” and filtering of significant peaks
Michael Lustig, David Donoho, John M. Pauly: “Sparse MRI: The application of compressed sensing for rapid MR imaging”
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

17. Single pixel camera General principle
Object
Photodiode Σ
Memory
Several Measurements
Reconstruction
DMD
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

18. Single pixel camera Reconstruction
For M measurements:
Y: Measurements
X: Image input
Φ: Concatenation of test functions
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

19. Single pixel camera Reconstruction
Implication of sparsity:
X: Input Image
Ψ: Dictionnary or Basis Matrix
α : Wavelet Coefficients
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

20. Single pixel camera Reconstruction
Thus:
X: Input Image
Φ: Concatenation of test functions
Ψ: Dictionnary or Basis Matrix
α : Wavelet Coefficients
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

21. Single pixel camera Reconstruction
Reconstruction using:
such as
Number theoric of measurements:
M: Number theoric of measurements
K: Number of non zeros in α
N: Number of pixels
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

22. Single pixel camera Results
2 % measurements
5 % measurements
Original:
16384 pixels
10 % measurements
20 % measurements
M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp , March 2008
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

23. Compressed sensing conclusion
Compressed sensing lets us sub-sample a signal w.r.t. Nyquist rate and reconstruct it perfectly, if this signal is known to be sparse in some domain and some conditions are met
Compressed sensing is promising for a wide range of future technologies, specially for high-frequency signals
Speeds up acquisition process: specially interesting for MRI
Cheaper hardware (ie: IR cameras with only a few sensors)
Sparsity can also be exploited for classification and image processing tasks[Huang, K. and Aviyente, S., Sparse representation for signal classification]
8/11/2010
Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert