1. Recovering structured signals: Precise performance analysis Christos Thrampoulidis Joint ITA Workshop, La Jolla, CA February 3, 2016

2. Let’s start “simple”…

3. Given y and A can you find x 0 ? Noiseless linear measurements

4. Does it work? (if so, when?) Let A be random (e.g. Gaussian, random DFT). [Candes,Romberg,Tao’04] then, x 0 is the unique solution with high probability. If,

5. Suppose A has entries iid Gaussian. Please, be precise… Then, [Donoho’06; Donoho,Tanner’09; Stojnic’09; Donoho et.al.’09; Amelunxen et.al.’13; Stojnic’13] In fact, much more is known… Generalization to other structures (e.g. group-sparse, low-rank, binary entries, smooth) and convex regularizer functions f (e.g. l 1,2, nuclear-norm, atomic-norms). Gaussian-width, statistical dimension [Donoho et. al.’09; Chandrasekaran et.al.’12; Amelunxen et.al.’13]

6. Crystal clear understanding!

7. One step further…

8. Add some noise!

9. Can you estimate x 0 ? Noisy linear measurements

10. What is the quality of the estimate? LASSO Suppose A has randomness. then w.h.p.: and If e.g. [Belloni’11]

11. Precise results?

12. Precise results? Yes! [Donoho, Maleki, Montanari’09] and [Bayati, Montanari’11]. Approach is based on the AMP framework, inspired by statistical physics. Over the past couple of year, we have developed a new framework the CGMT that is based on Gaussian process methods. Framework is inspired by [Stojnic’13]. Gives results that are very general. Natural way of analysis… CGMT

13. Precise results? Yes! [C.T., Panahi, Guo, Hassibi’14]

14. Precise results? Yes! [C.T., Panahi, Guo, Hassibi’14]

15. Precise results? Yes! [C.T., Panahi, Guo, Hassibi’14]

16. Precise results? Yes! [C.T., Panahi, Guo, Hassibi’14]

17. Precise results? Yes! [C.T., Panahi, Guo, Hassibi’14]

18. Precise results? Yes! Expected Moreau Envelope summary functional: captures the role of regularizer and of p X0. generalizes the “Gaussian-width” of noiseless CS. [Thrampoulidis, Panahi, Guo, Hassibi’14]

19. Optimal tuning There is value to be precise! Noiseless CS results follow as special case:

20. Generalizations General structures and convex regularizers. Other performance metrics (e.g. support recovery, BER) – Massive MIMO (BER of convex relaxation decoders) General noise distributions and loss functions! Regularized M-estimators

21. Master Theorem [Thrampoulidis, Abbasi, Hassibi ’16]

22. Optimal tuning Precise is good!

23. Optimal loss function? 23 Precise & general is better!

24. Consistent Estimators? 24 sparse noise

25. Robustness to outliers? Huber loss function + Cauchy noise

26. Final challenge

29. measurement device has nonlinearities & uncertainties. can also arise by design Non-linear measurements (or single-index model)

30. measurement device has nonlinearities & uncertainties. can also arise by design Non-linear measurements (or single-index model)

31. What if we use the LASSO? 1.the link function might be unknown/misspecified (robustness) 2.widely available LASSO solvers, used in practice 3.turns out it performs well

32. Theorem (Non-linear=linear) For example, for 1-bit measurements: [Thrampoulidis, Abbasi, Hassibi. NIPS’15] Previous work of Plan & Vershynin ‘15 derives order-wise bounds for the constrained LASSO. Our precise result explicitly captures equivalence & the role of σ.

34. Application: optimal quantization q-bit measurements LASSO Lloyd-Max (LM) algorithm is optimal A LASSO A

35. One last word: How it all works?

36. Gaussian min-max Theorem (Gordon’88) Convex Gaussian Min-max Theorem [Thrampoulidis, Oymak, Hassibi. COLT ’15] CGMT framework

37. Gaussian min-max Theorem (Gordon’88) Convex Gaussian Min-max Theorem [Thrampoulidis, Oymak, Hassibi. COLT ’15] CGMT framework GMT

38. Thrampoulidis, Abbasi, Hassibi: “Precise Error Analysis of Regularized M-estimators in High-dimensions”, available on arXiv, Jan. 2016 More… Thank you!