1. Bounds for Optimal Compressed Sensing Matrices
Shriram Sarvotham
DSP Group, ECE, Rice University

2. Compressed Sensing Signal has non-zero coefficients
Efficient ways to measure and recover ?
Traditional DSP approach:
Acquisition: first obtain measurements
Then compress, throwing away all but coefficients
Emerging Compressed Sensing (CS) approach:
Acquisition: obtain just measurements
No unnecessary measurements / computations [Candes et al; Donoho]

3. Compressed Sensing CS measurements: matrix multiplication
sparse signal
measurements
sparse

4. Contributions Bounds on quality of CS matrices
In terms of the CS parameters
Quality metric: Restricted Isometry
Designing CS matrices for fast reconstruction
Sparse matrices
Fast algorithms
CS rate-distortion

5. Contributions Bounds on quality of CS matrices
In terms of the CS parameters
Quality metric: Restricted Isometry
Designing CS matrices for fast reconstruction
Sparse matrices
Fast algorithms
CS rate-distortion

6. Restricted Isometry Property (RIP)
Key idea: ensure (approximate) isometry for
restricted to the domain of -sparse signals:
Restricted Isometry Property of order
RIP ensures columns of are locally almost orthogonal rather than globally perfectly orthogonal
Measure of quality of CS matrix:

7. Restricted Isometry Property (RIP)
Good CS matrix:
Question: In , what can we say about for the best CS matrix?
Answer: Determined by 2 bounds
Structural bound
Packing bound

8. Restricted Isometry Property (RIP)
Good CS matrix:
Question: In , what can we say about for the best CS matrix?
Answer: Determined by 2 bounds
Structural bound
Packing bound

9. Tied to SVD’s of sub-matrices
I) Structural bound
Tied to SVD’s of sub-matrices

10. Role of sub-matrices of
RIP depends only on sub-matrices of

11. Role of sub-matrices of
RIP depends only on sub-matrices of 

12. Theorem of Thompson
[Thompson 1972] Denote the SV characteristic equation of a matrix by
The SV characteristic equations of sub-matrices satisfy

13. Significance to RIP Let Zeros of : Then, and
is minimized when ’s are equal

14. Significance to RIP Let Zeros of : Then, and
is minimized when ’s are equal

15. Significance to RIP Let Zeros of : Then, and
is minimized when ’s are equal
Structural bound

16. Geometrical meaning Relates volumes of hyper-ellipse to those of
Special case: when and equating constant terms,

17. Geometrical meaning Relates volumes of hyper-ellipse to those of
Special case: when and equating constant terms,
(Generalized Pythagorean Theorem)

18. GPT for Areas in .

19. Geometrical meaning Thompson Equation extends GPT
Relates -volumes of the hyperellipses in

20. Structural bound

21. Structural bound
Upper bound

22. How tight is the structural bound?

23. How tight is the structural bound?
We answer for

24. Comparison with best . Structural bound good for up to some
Best known
Best known
Structural bound
Structural bound
Structural bound good for up to some
Beyond , RIP of best construction diverges

25. Comparison with best . Structural bound good for up to some
Best known
Best known
Structural bound
Structural bound
Structural bound good for up to some
Beyond , RIP of best construction diverges
 Hints at another mechanism controlling RIP!

26. Connections to Equi-angular Tight Frames (ETF)
that meets the structural bound for is an ETF [Sustik, Tropp et al]
ETF’s satisfy three conditions
Columns are equi-normed
Angle between every pair of columns is same

27. Connections to Equi-angular Tight Frames (ETF)
that meets the structural bound for is an ETF [Sustik, Tropp et al]
ETF’s satisfy three conditions
Columns are equi-normed
Angle between every pair of columns is same
Example:

28. Singular and Eigen values of Sub-matrices
Series of 9 papers by R.C. Thompson
Additional results on Singular vectors
Useful in construction of
Prof. Robert C. Thompson
Born 1931
Ph. D. (CalTech, 1960),
Professor ( UCSB, )
Published 4 books papers
Died Dec. 10, 1995

29. Tied to packing in Euclidean spaces
II) Packing bound
Tied to packing in Euclidean spaces

30. Tied to packing in Euclidean spaces
II) Packing bound
Tied to packing in Euclidean spaces
Derive for

31. Role of column norms on SV’s

32. Role of angle on SV’s
Ratio of SV’s depends only on

33. Role of column norms on RIP

34. Role of column norms on RIP

35. Role of column norms on RIP
 Restrict our attention to with equi-normed columns

36. Maximizing the minimum angle between lines
. and codes in .
Design of good for
is equivalent to
Maximizing the minimum angle between lines
in .

41. Packing bound θ
Packing (converse):

42. Packing bound Packing (converse): Covering (achievable):θ
Packing (converse):
Covering (achievable):
[Shannon, Chabauty, Wyner]

43. Ok, lets put this all together….+ +
Structural bound
Packing bound
Covering bound

44. Comparison of bounds Gaussian iid construction Achievable (covering)
Best known CS matrix
Converse (structural)
Converse (packing)

45. Comparison of bounds Gaussian iid construction Achievable (covering)
Best known CS matrix
Converse (packing)
Converse (structural)

46. Comparison of bounds Gaussian iid construction Achievable (covering)
Best known CS matrix
Converse (structural)
Converse (packing)

47. Relevance of Thompson polynomial
Comparison of
with histograms of

48. Relevance of Thompson polynomial
Comparison of
with histograms of

49. Relevance of Thompson polynomial
Comparison of
with histograms of
Useful in stochastic RIP!

50. Future directions Connections to Johnson-Lindenstrauss Lemma
Packing bounds for
Explicit constructions for
Extensions to Universal Compressed Sensing

51. Summary Derived deterministic bounds for RIP Connections to coding
Structural bound based on SVD
Packing bound based on sphere/cone packing
Connections to coding
Codes on Grassmannian spaces
Geometric interpretations
Generalized Pythagorean Theorem
Equi-angular tight frames

52. BACKUP SLIDES

53. { {

54. Structural bound and Equi-angular tight frames
Columns of such that
norms are equal
Angle between every pair is same
If then A*A=cI

55. Equi-angular tight frames
meets the bound iff an ETF

56. M=4

57. M=8

58. M=16