1. Shriram Sarvotham Dror Baron Richard Baraniuk ECE Department Rice University dsp.rice.edu/cs Sudocodes Fast measurement and reconstruction of sparse signals

2. Motivation: coding of sparse data Distributed delivery of data with sparse representation –Content delivery networks –Peer to peer networks –Distributed file storage systems E.g. thresholded DCT/wavelet coefficients used in JPEG/JPEG2000

3. Motivation: coding of sparse data Distributed coding of sparse data –Can we exploit sparsity? –Efficient? –Low complexity?

4. Sparse signal processing Signal has non-zero coefficients Efficient ways to measure and recover ? Traditional DSP approach: –Acquisition: first obtain measurements –Then exploit sparsity is in the processing stage

5. Sparse signal processing Signal has non-zero coefficients Efficient ways to measure and recover ? Traditional DSP approach: –Acquisition: first obtain measurements –Then exploit sparsity is in the processing stage New compressive sampling (CS) approach: –Acquisition: obtain just measurements –Sparsity is exploited during signal acquisition [Candes et al; Donoho]

6. Compressive sampling Signal is -sparse Measure signal via few linear projections Enough to encode the signal measurements sparse signal nonzero entries

7. Compressive sampling Signal is -sparse Measure signal via few linear projections Random Gaussian measurements will work! measurements sparse signal nonzero entries

8. CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries Find the explanation with smallest L 1 norm [Candes et al; Donoho] If then perfect reconstruction w/ high probability

9. CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries Performance – Polynomial complexity reconstruction – Efficient encoding

10. CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries But… is still impractical for many applications Reconstruction times:  N=1,000t=10 seconds  N=10,000t=3 hours  N=100,000t=~months

11. Why is reconstruction expensive? measurements sparse signal nonzero entries

12. Why is reconstruction expensive? measurements sparse signal nonzero entries Culprit: dense, unstructured

13. Fast CS reconstruction measurements sparse signal nonzero entries Sudocode matrix (sparse) Only 0/1 in Each row of contains randomly placed 1’s

14. Sudocodes measurements sparse signal nonzero entries Sudocode performance –Efficient encoding –Sub-linear complexity reconstruction Encouraging numerical results N=100,000 K=1,000  t=5.47 seconds M=5,132

15. Sudocode reconstruction measurements sparse signal nonzero entries Process each in succession Each can recover some ‘’s

16. Case 1: Zero measurement

17. Resolves all coefficients in the support Can resolve up to coefficients

18. Case 1: Zero measurement Resolves all coefficients in the support Can resolve up to coefficients Reduces size of problem

19. Case 2: #(support set)=1

20. Trivially resolves

21. Case 2: #(support set)=1 Trivially resolves

22. Case 3: Matching measurements

23. Common support Matches originate from same support Disjoint support  coefficients = 0 Common support  contain nonzeros

24. Case 3: Matching measurements Matches originate from same support Disjoint support  coefficients = 0 Common support  contain nonzeros

25. Trigger of revelations Recovery of can trigger more revelations

26. Trigger of revelations Recovery of can trigger more revelations An avalanche of coefficient revelations

27. Trigger of revelations Recovery of can trigger more revelations An avalanche of coefficient revelations

28. Sudocode reconstruction measurements sparse signal nonzero entries Like sudoku puzzles

29. Practical considerations Bottleneck: search for matches –With Binary Search Tree, matches ~ Re-explain measurements: more data structures Search for matches

30. Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed

31. Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed

32. Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Intuition: so that

33. Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Intuition: so that

34. Related work [Cormode, Muthukrishnan] –CS scheme based on group testing – –Complexity [Gilbert et. al.] Chaining Pursuit –CS scheme based on group testing and iterating – –Complexity –Works best for super-sparse signals

35. Performance comparison L 1 reconstruction Chaining Pursuit Sudocodes N=10,000 K=10 M=99 T3 hours M=5,915 t=0.16 sec M=461 t=0.14 sec N=10,000 K=100 M=664 T3 hours M=90,013 t=2.43 sec M=803 t=0.37 sec N=100,000 K=10 M=1,329 Tmonths M=17,398 t=1.13 sec M=931 t=1.09 sec N=10,000 K=1000 M=3,321 T3 hours M>10 6 t>30 sec M=5,132 t=5.47 sec measurements sparse signal nonzero entries

36. Utility in CDNs Measurements come from different sources Needs enough measurements

37. Ongoing work Statistical dependencies between non-zero coefficients Irregular degree distributions Adaptive linear projections Noisy measurements

38. Conclusions Sudocodes for CS –highly efficient –low complexity Key idea: use sparse Applications to content distribution

39. THE END Compressed sensing webpage: dsp.rice.edu/cs

44. Number of measurements Theorem: With, phase 1 requires to exactly reconstruct coefficients Proof sketch:

45. Two phase decoding Phase 1: decode coefficients Phase 2: decode remaining coefficients Why? –When most coefficients are decoded, Phase 2 saves a factor of measurements is not measured

46. Phase 2 measurements and decoding is non-sparse of dimension

47. Phase 2 measurements and decoding is non-sparse of dimension Resolve remaining coefficients by inverting the sub-matrix of

48. Phase 2 measurements and decoding is non-sparse of dimension Resolve remaining coefficients by inverting the sub-matrix of Phase 2 complexity = Key: choose Phase 2 complexity is

49. Compressive Sampling Signal is -sparse in basis/dictionary –WLOG assume sparsity in space domain Measure signal via few linear projections Random sparse measurements will work! measurements sparse signal nonzero entries

51. Signal model measurements sparse signal nonzero entries Signal is strictly sparse Every nonzero ~ continuous distribution  each nonzero coefficient is unique almost surely