1. Sudocodes Fast measurement and reconstruction of sparse signals
Shriram Sarvotham
Dror Baron
Richard Baraniuk
ECE Department Rice University
dsp.rice.edu/cs
Came out of my personal experience with 301 – fourier analysis and linear systems

2. Motivation: coding of sparse data
Streaming in CDNs, distributed storage systems
Delivery of content that has sparse representation
E.g. thresholded DCT/wavelet coefficients in JPEG/JPEG2000
Distributed coding of sparse data

3. Sparse signal Acquisition
Consider that contains only non-zero coefficients
Are there efficient ways to measure and recover ?
Traditional DSP approach:
Acquisition: obtain measurements
Sparsity is exploited only in the processing stage
New Compressed Sensing (CS) approach:
Acquisition: obtain just measurements
Sparsity is exploited during signal acquisition
[Candes et al; Donoho]

4. Compressive Sampling Signal is -sparse in basis/dictionary
WLOG assume sparsity in space domain
Measure signal via few linear projections
sparse signal
measurements
nonzero entries

5. Compressive Sampling Signal is -sparse in basis/dictionary
WLOG assume sparsity in space domain
Measure signal via few linear projections
Random Gaussian measurements will work!
sparse signal
measurements
nonzero entries

6. CS Miracle: L1 reconstruction
Find the solution with smallest L1 norm
[Candes et al; Donoho]
If then perfect reconstruction w/ high probability
sparse signal
measurements
nonzero entries

7. CS Miracle: L1 reconstruction
Performance
Efficient encoding, and
Polynomial complexity reconstruction
sparse signal
measurements
nonzero entries

8. CS Miracle: L1 reconstruction
But… is still impractical for many applications
Reconstruction times:
N=1,000 t=10 seconds
N=10,000 t=3 hours
N=100,000 t=~months
sparse signal
measurements
nonzero entries

9. Why is reconstruction expensive?
sparse signal
measurements
nonzero entries

10. Why is reconstruction expensive?
Culprit: dense, unstructured
sparse signal
measurements
nonzero entries

11. Fast CS reconstruction
Sudocode matrix (sparse)
Only 0/1 in
Each row of contains randomly placed 1’s
sparse signal
measurements
nonzero entries

12. Fast CS reconstruction
Sudocode performance
Efficient encoding
Sublinear complexity reconstruction
Encouraging numerical results
N=100,000 K=1,000  t=5.47 seconds M=5,132
sparse signal
measurements
nonzero entries

13. Signal model Signal is strictly sparse sparse signal measurements
nonzero entries

14. Signal model Signal is strictly sparse
Every nonzero ~ continuous distribution  each nonzero coefficients is unique almost surely
sparse signal
measurements
nonzero entries

15. Sudocode reconstruction
Process each
in succession
Each
can recover some
‘’s
sparse signal
measurements
nonzero entries

16. Sudocode reconstruction
Like sudoku puzzles!
sparse signal
measurements
nonzero entries

17. Case 1: Zero measurement

18. Case 1: Zero measurement
Resolves all coefficients in the support
Can resolve up to coefficients

19. Case 1: Zero measurement
Resolves all coefficients in the support
Can resolve up to coefficients
Reduces size of problem

20. Case 2: #(support set)=1

21. Case 2: #(support set)=1
Trivially resolves

22. Case 2: #(support set)=1
Trivially resolves

23. Case 3: Matching measurements

24. Case 3: Matching measurements
Matches originate from same support
Disjoint support  coefficients = 0 Common support  contain nonzeros
Common
support

25. Case 3: Matching measurements
Matches originate from same support
Disjoint support  coefficients = 0 Common support  contain nonzeros

26. Trigger of revelations
Recovery of can trigger more revelations

27. Trigger of revelations
Recovery of can trigger more revelations
An avalanche of coefficient revelations

28. Trigger of revelations
Recovery of can trigger more revelations
An avalanche of coefficient revelations

29. Auxiliary data structures
Bottleneck: search for matches
With Binary Search Tree, matches ~
Re-explain measurements: more data structures
Search for
matches

30. Design of Sudo measurement matrix
Choice of L
Set L based on
For large N,

31. Number of measurements
Theorem: With , decoder requires
to exactly reconstruct
coefficients
Proof sketch:

32. Choice of L
K=0.02N
For a given choice of N and K

33. Choice of L
Numerical evidence also suggests L = O(N/K)

34. Related work [Cormode, Muthukrishnan]
CS scheme based on group testing
Complexity
[Gilbert et. al.] Chaining Pursuit
CS scheme based on group testing and iterating
Works best for super-sparse signals

35. Performance comparison
Chaining
Pursuit
Sudocodes
N=10,000
K=10
M=5,915
t=0.16 sec
M=461
t=0.14 sec
K=100
M=90,013
t=2.43 sec
M=803
t=0.37 sec
N=100,000
M=17,398
t=1.13 sec
M=931
t=1.09 sec
K=1000
M>106
t>30 sec
M=5,132
t=5.47 sec

36. Sudocode applications
Erasure codes in p2p and distributed file storage
Stream compressed digital content
Thresholded DCT/wavelet coefficients for sudocoding
Partial reconstruction of signals (e.g. detection)

37. Ongoing work Statistical dependencies between non-zero coefficients
Adaptive linear projections
Noisy measurements

38. Erasure coding

39. Conclusions Sudocodes for CS Key idea: use sparse
highly efficient
low complexity
Key idea: use sparse
Applications to erasure codes, P2P networks

45. Number of measurements
Theorem: With , phase 1 requires
to exactly reconstruct
coefficients
Proof sketch:

46. Two phase decoding is not measured Phase 1: decode coefficients
Phase 2: decode remaining coefficients
Why?
When most coefficients are decoded,
Phase 2 saves a factor of measurements

47. Phase 2 measurements and decoding
is non-sparse of dimension

48. Phase 2 measurements and decoding
is non-sparse of dimension
Resolve remaining coefficients by inverting the sub-matrix of

49. Phase 2 measurements and decoding
is non-sparse of dimension
Resolve remaining coefficients by inverting the sub-matrix of
Phase 2 complexity =
Key: choose
Phase 2 complexity is

50. Compressive Sampling Signal is -sparse in basis/dictionary
WLOG assume sparsity in space domain
Measure signal via few linear projections
Random sparse measurements will work!
sparse signal
measurements
nonzero entries