1. Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, lcs}@iis.sinica.edu.tw April 2009 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP2009, Taipei, Taiwan, ROC) Distributed Compressive Video Sensing

2. Distributed Compressive Video Sensing April 24, 2009 2 Distributed Source Coding Vanishing error probability for long sequences No errors [Slepian and Wolf, 1973]

3. Distributed Compressive Video Sensing April 24, 2009 3 Distributed Video Coding “Motion JPEG” Decoder “Motion JPEG” Encoder X’ X Wyner-Ziv Interframe Decoder Wyner-Ziv Intraframe Encoder Side Information [Girod, 2006]

4. Distributed Compressive Video Sensing April 24, 2009 4 Distributed Video Coding The statistical dependency between X and Y  Laplacian distribution

5. Distributed Compressive Video Sensing April 24, 2009 5 Compressive Sensing When data is sparse/compressible, one can directly acquire a condensed representation with no/little information loss Random projection will work [Baraniuk, 2008]

6. Distributed Compressive Video Sensing April 24, 2009 6 Compressive Sensing Directly acquire “compressed” data Replace samples by more general “measurements” [Baraniuk, 2008]

7. Distributed Compressive Video Sensing April 24, 2009 7 Compressive Sensing y = Фx = ФΨθ = Aθ y Ф Ψ θ x = Ψθ A = ФΨ N×1N×1 N×NN×N M×NM×N M×1M×1 [Baraniuk, 2008]

8. Distributed Compressive Video Sensing April 24, 2009 8 Measurement Matrix Scrambled block Hadamard ensemble (SBHE)  partial block hadamard transform and random column permutation Ф = Q M WP N L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled hadamard ensemble,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).

9. Distributed Compressive Video Sensing April 24, 2009 9 Signal Reconstruction The convex unconstrained optimization problem Can be seen as a maximum a posteriori criterion for estimating θ from y = A θ + n, where n is white Gaussian noise

10. Distributed Compressive Video Sensing April 24, 2009 10 Signal Reconstruction Signal recovery from random measurements  Gradient projection for sparse reconstruction (GPSR)  Two-step iterative shrinkage/thresholding algorithm (TwIST)  Orthogonal matching pursuit (OMP) M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. of Selected Topics in Signal Processing, vol. 1,no. 4, pp. 586-597, Dec. 2007. J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Processing, vol. 16, no. 12, pp. 2992-3004, Dec. 2007. T. Blumensath and M. E. Davies, “Gradient pursuits,” IEEE Trans. on Signal Processing, vol. 56, June 2008.

11. Distributed Compressive Video Sensing April 24, 2009 11 Distributed Compressive Video Sensing Measurement matrix Ф: scrambled block Hadamard ensemble (SBHE) Sparse basis matrix Ψ: DWT Video signal sensing (encoder): general random projection Video signal recovery (decoder)  Key frame: GPSR with default settings  CS frame  side information generation (motion compensated interpolation)  GPSR with the proposed initialization and the proposed termination criteria

12. Distributed Compressive Video Sensing April 24, 2009 12 Distributed Compressive Video Sensing Compressive video sensing Video signal recovery

13. Distributed Compressive Video Sensing April 24, 2009 13 Distributed Compressive Video Sensing At the decoder, for a CS frame x t = Ψθ t  its side information S t = Ψθ St can be generated from its previous reconstructed key frames Proposed initialization  initial solution at the 0-th iteration: α(x t, S t ): the Laplacian parameter of (x t - S t )

14. Distributed Compressive Video Sensing April 24, 2009 14 Key frame (t - 1) Non-key frame t Key frame (t + 1) y t-1 = Φx t-1 with higher MR y t = Φx t with lower MR y t+1 = Φx t+1 with higher MR GPSR reconstruction Proposed Modified GPSR reconstruction Side information (t) Reconstructed frame (t) Reconstructed frame (t-1) Side information generation Reconstructed frame (t+1)

15. Distributed Compressive Video Sensing April 24, 2009 15 Distributed Compressive Video Sensing xtxt StSt α(x t, ) α(x t, S t ) α(, S t )

16. Distributed Compressive Video Sensing April 24, 2009 16 Proposed Termination Criterion First: Second: Third:

17. Distributed Compressive Video Sensing April 24, 2009 17 Proposed Termination Criterion MR is low (MR ≤ 20%): if the First criterion with T α = 0.9 is satisfied, the algorithm will stop MR is middle (20% < MR ≤ 70%): if the First criterion with T α = 0.05 or the Second criterion is satisfied, the algorithm will stop MR is high (MR > 70%): if the Third criterion with T F = 0.001 is satisfied, the algorithm will stop

18. Distributed Compressive Video Sensing April 24, 2009 18 Simulation Results Foreman and Coastguard CIF video sequences with 300 Y frames (352×288 = 101376 samples for each Y frame) and GOP size = 3 (Key, Non-key, Non-key, Key, …) The three approaches for comparison (all with default settings)  GPSR, TwIST, OMP For OMP, block size = 32×32 suggested by V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).

19. Distributed Compressive Video Sensing April 24, 2009 19 Simulation Results

20. Distributed Compressive Video Sensing April 24, 2009 20 Simulation Results

21. Distributed Compressive Video Sensing April 24, 2009 21 Simulation Results The reconstruction complexities for the Foreman sequence

22. Distributed Compressive Video Sensing April 24, 2009 22 Simulation Results The PSNR performance at different reconstruction complexities for the Foreman sequence

23. Distributed Compressive Video Sensing April 24, 2009 23 Simulation Results (a) Side information (b) Reconstructed frame

24. Distributed Compressive Video Sensing April 24, 2009 24 Simulation Results The reconstructed Foreman sequences (352×288 for each frame) at measurement rate (MR) = 0.3 using (a) GPSR (gradient projection for sparse reconstruction) (average PSNR = 27.68 dB) (average reconstruction time = 15.14 seconds per frame); and (b) our DCVS (average PSNR = 29.48 dB) (average reconstruction time = 3.68 seconds per frame) (This example shows the 54-th frame).

25. Distributed Compressive Video Sensing April 24, 2009 25 Conclusions The proposed DCVS approach exploits the two characteristics  distributed video coding (DVC)  compressive sensing (CS) The proposed DCVS can outperform or be comparable with the three existing approaches for comparison, especially at lower measurement rates The proposed DCVS can significant outperform the three existing approaches at the same reconstruction complexity