Compressed Sensing for Networked Information Processing Reza Malek-Madani, 311/ Computational Analysis Don Wagner, 311/ Resource Optimization Tristan Nguyen, 311/ Computational Analysis 28 November 2007

Ubiquitous Compressibility DoD acquires and uses huge amount of data –In many scenarios, most of the data in a signal can be discarded with almost no perceptual loss –E.g., lossy compression formats for sounds and images Key Questions: –Why acquire all the data when most will be discarded? –Can we directly measure the relevant information? Challenge: Develop mathematical and computational techniques that allow us to directly acquire relevant information from signals and images in compressed form. Key Words: Adaptivity, Parallelization, Stability, Nonlinearity, Noise

What is Compressed Sensing? Underlying Assumption: Most signals are compressible in some representation (i.e., most coefficients are small relative to some basis) Compressed Sensing: –“Measure” the signal via a random projection to yield a compact representation –Reconstruct the signal from its compact representation

Nyquist vs. Compressed Sensing Nyquist rate samples of wideband signal (sum of 20 wavelets) N = 1024 samples/second Reconstruction from compressed sensing M = 150 random measurements/second MSE < 2% of signal energy

Nyquist rate samples of image N = pixels Reconstruction from compressed sensing M = projections MSE < 3% of signal energy Nyquist vs. Compressed Sensing

Compressed Sensing Forward Problem: Random projection is the key idea Inverse Problem: Reconstruct from ; this is an ill-posed problem [Candes-Romberg-Tao, Donoho, 2004] measurements sparse signal sparse in some basis

Reconstruction: find given Classical L 2 approach: L 2 algorithm is fast, but unfortunately it is wrong CS Signal Recovery

Reconstruction:given (ill-posed inverse problem) find L 2 fast, wrong: L 0 correct, slow: L 1 correct, mild oversampling: [Candes et al, Donoho] CS Signal Recovery Linear-programming problem

Theoretical Result (Donoho, 2004) Theorem: There is a function g, from the interval (0, 1] to itself, with the following characteristics: Fix ε > 0. If K/M > g(M/N)(1 + ε) then, with overwhelming probability for large N, =. If K/M < g(M/N)(1 – ε ), then does not equal.

Applications Image Understanding (feature detection) Communication –Underwater Communication (RF) –Wireless Communication –Channel Parameter Identification (cognitive radio, radar) Distributed Sensing (fusion of partial information) …. …

Mathematical Challenges Randomness versus determinism –Can random sensing matrices be replaced by deterministic ones? –What is the impact on the theoretical development? –Can rigorous bounds be developed for the equivalents of K, M and N? Faster optimization algorithms –Reconstruction via L 1 minimization is relatively slow –Other algorithmic ideas need to be developed First-order vs. second-order methods? Combinatorial vs. linear vs. nonlinear methods? –Need to create baselines for comparing algorithms in terms of reconstruction speed and accuracy Multiple sensors/multiple targets –Can the underlying theory be extended to handle distributed, networked sensors and multiple targets? –Develop the mathematical tools needed to take advantage of the statistical correlations among signals to perform multi-signal reconstruction

Mathematical Challenges How many measurements are needed? –One can get by with much fewer measurements, but at the expense of having to solve a tougher (i.e., non-convex) optimization problem. What is the tradeoff? Important extensions needed –No development to date of distributed reconstruction algorithms Very important for distributed sensor networks –Does the theoretical developments to date adequately address the issue of noise? Random vs. pseudo-random vs. deterministic sensing matrices

Tangential Issues Extension of compressed-sensing optimization to affine rank minimization –Potentially very important in data mining Numerical partial differential equation solvers –Can de-aliasing techniques benefit from the compressed- sensing approach ?

Budget Current seed investment: $250K/year Proposed First Year: $1.2M/year –40% for analytical/theoretical development –40% for algorithmic/computational development –20% for application/sensor development Outyear growth towards applications FYFY+1FY+2FY+3 ONR$1.2M$1.4M$1.5M

Summary Compressed Sensing is an important emerging area –Cuts across of sciences and engineering –Pioneering foundations are in place –ONR is well positioned to be a leader