Bounds for Optimal Compressed Sensing Matrices Shriram Sarvotham DSP Group, ECE, Rice University

Compressed Sensing Signal has non-zero coefficients Efficient ways to measure and recover ? Traditional DSP approach: Acquisition: first obtain measurements Then compress, throwing away all but coefficients Emerging Compressed Sensing (CS) approach: Acquisition: obtain just measurements No unnecessary measurements / computations [Candes et al; Donoho]

Compressed Sensing CS measurements: matrix multiplication sparse signal measurements sparse

Contributions Bounds on quality of CS matrices In terms of the CS parameters Quality metric: Restricted Isometry Designing CS matrices for fast reconstruction Sparse matrices Fast algorithms CS rate-distortion

Contributions Bounds on quality of CS matrices In terms of the CS parameters Quality metric: Restricted Isometry Designing CS matrices for fast reconstruction Sparse matrices Fast algorithms CS rate-distortion

Restricted Isometry Property (RIP) Key idea: ensure (approximate) isometry for restricted to the domain of -sparse signals: Restricted Isometry Property of order RIP ensures columns of are locally almost orthogonal rather than globally perfectly orthogonal Measure of quality of CS matrix:

Restricted Isometry Property (RIP) Good CS matrix: Question: In , what can we say about for the best CS matrix? Answer: Determined by 2 bounds Structural bound Packing bound

Restricted Isometry Property (RIP) Good CS matrix: Question: In , what can we say about for the best CS matrix? Answer: Determined by 2 bounds Structural bound Packing bound

Tied to SVD’s of sub-matrices I) Structural bound Tied to SVD’s of sub-matrices

Role of sub-matrices of RIP depends only on sub-matrices of

Role of sub-matrices of RIP depends only on sub-matrices of 

Theorem of Thompson [Thompson 1972] Denote the SV characteristic equation of a matrix by The SV characteristic equations of sub-matrices satisfy

Significance to RIP Let Zeros of : Then, and is minimized when ’s are equal

Significance to RIP Let Zeros of : Then, and is minimized when ’s are equal

Significance to RIP Let Zeros of : Then, and is minimized when ’s are equal Structural bound

Geometrical meaning Relates volumes of hyper-ellipse to those of Special case: when and equating constant terms,

Geometrical meaning Relates volumes of hyper-ellipse to those of Special case: when and equating constant terms, (Generalized Pythagorean Theorem)

GPT for Areas in .

Geometrical meaning Thompson Equation extends GPT Relates -volumes of the hyperellipses in

Structural bound

Structural bound Upper bound

How tight is the structural bound?

How tight is the structural bound? We answer for

Comparison with best . Structural bound good for up to some Best known Best known Structural bound Structural bound Structural bound good for up to some Beyond , RIP of best construction diverges

Comparison with best . Structural bound good for up to some Best known Best known Structural bound Structural bound Structural bound good for up to some Beyond , RIP of best construction diverges  Hints at another mechanism controlling RIP!

Connections to Equi-angular Tight Frames (ETF) that meets the structural bound for is an ETF [Sustik, Tropp et al] ETF’s satisfy three conditions Columns are equi-normed Angle between every pair of columns is same

Connections to Equi-angular Tight Frames (ETF) that meets the structural bound for is an ETF [Sustik, Tropp et al] ETF’s satisfy three conditions Columns are equi-normed Angle between every pair of columns is same Example:

Singular and Eigen values of Sub-matrices Series of 9 papers by R.C. Thompson Additional results on Singular vectors Useful in construction of Prof. Robert C. Thompson Born 1931 Ph. D. (CalTech, 1960), Professor ( UCSB, 1963 -- 1995) Published 4 books + 120 papers Died Dec. 10, 1995

Tied to packing in Euclidean spaces II) Packing bound Tied to packing in Euclidean spaces

Tied to packing in Euclidean spaces II) Packing bound Tied to packing in Euclidean spaces Derive for

Role of column norms on SV’s

Role of angle on SV’s Ratio of SV’s depends only on

Role of column norms on RIP

Role of column norms on RIP

Role of column norms on RIP  Restrict our attention to with equi-normed columns

Maximizing the minimum angle between lines . and codes in . Design of good for . is equivalent to Maximizing the minimum angle between lines in .

Packing bound θ Packing (converse):

Packing bound Packing (converse): Covering (achievable): θ Packing (converse): Covering (achievable): [Shannon, Chabauty, Wyner]

Ok, lets put this all together…. + + Structural bound Packing bound Covering bound

Comparison of bounds Gaussian iid construction Achievable (covering) Best known CS matrix Converse (structural) Converse (packing)

Comparison of bounds Gaussian iid construction Achievable (covering) Best known CS matrix Converse (packing) Converse (structural)

Comparison of bounds Gaussian iid construction Achievable (covering) Best known CS matrix Converse (structural) Converse (packing)

Relevance of Thompson polynomial Comparison of with histograms of

Relevance of Thompson polynomial Comparison of with histograms of

Relevance of Thompson polynomial Comparison of with histograms of Useful in stochastic RIP!

Future directions Connections to Johnson-Lindenstrauss Lemma Packing bounds for Explicit constructions for Extensions to Universal Compressed Sensing

Summary Derived deterministic bounds for RIP Connections to coding Structural bound based on SVD Packing bound based on sphere/cone packing Connections to coding Codes on Grassmannian spaces Geometric interpretations Generalized Pythagorean Theorem Equi-angular tight frames

BACKUP SLIDES

{ {

Structural bound and Equi-angular tight frames Columns of such that norms are equal Angle between every pair is same If then A*A=cI

Equi-angular tight frames meets the bound iff an ETF

M=4

M=8

M=16