Brief Overview of Wyner-Ziv CODEC and Research Plan Jin-soo KIM

2 J.S.Kim Contents  Overview of Wyner-Ziv CODEC  Application of Wyner-Ziv CODEC  Basic Principle of WZ CODEC  Generation of S.I. at the Decoder  How to Encode WZ frames  Research Plan  Q&A

3 J.S.Kim  Application of WZ CODEC

4 J.S.Kim Coding Efficiency Network awareness + implementation? MPEG4 H MPEG1 Video Conferencing H Mobile Phone Hand PC Mobile TV SVC HDTV Year MPEG2 Video coding : history and trends H.265(?) mobile Mobile : 3low1high Low (battery, bandwidth, CPU) High cost

5 J.S.Kim (Conventional) Interframe Video Coding Predictive Interframe Decoder Predictive Interframe Encoder Side Information X’

6 J.S.Kim Wyner-Ziv Interframe Decoder Wyner-Ziv Intraframe Encoder [Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002] [Aaron, Zhang, Girod, Asilomar 2002] … Low Complexity Encoder X’ Side Information

7 J.S.Kim Applications of WZ codec  Light encoder and light decoder B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, “Distributed video coding,” Proceedings of the IEEE, Vol93, pp71-83, Jan Proceedings of the IEEE

8 J.S.Kim Applications of WZ codec  Wireless low power video surveillance  Disposable video cameras  Sensor network  Multi-view image acquisition  Medical applications  Networked camcoders

9 J.S.Kim Applications of WZ codec  SensorCam  PillCam  WearableCam  Dispos able cam.  ScanCam

10 J.S.Kim  Basic Principle of WZ CODEC

11 J.S.Kim Lossless Compression with Side Information Encoder Decoder R ≥ H(X|Y) Statistically dependent Side Information Encoder Decoder R ≥ H(X|Y) Statistically dependent [Slepian, Wolf, 1973] Side Information Wyner-Ziv showed that the conditional rate-mean squared error distortion function for X is the same whether the side information Y is available only at the decoder, or both at the encoder and the decoder.

12 J.S.Kim Shannon Theory with side info.  Example) x : dice number  H(X) = 6Σlog 2 6 = 2.58 bits  Shannon coding theorem  No error, if H(X) < R(X) = 3 bits  If R(X) = 2, {00,01,10,11}  {1,2,{3,4},{5,6}}  With side information Y=“even number”  H(X|Y) = 3Σlog 2 3 = 1.58 < R(X|Y) = 2 encoder decoder XX R Y Information loss

13 J.S.Kim Wyner-Ziv coding (lossy) A. Majumdar, R. Puri, P. Ishwar, K. Ramchandran, “Complexity/performance trade-offs for robust distributed video coding,” IEEE ICIP2005, Vol. 2, pp678-81, Sept WZ = quantization + Slepian-Wolf Random coset partitioning operation, 3bit-info can be represented by 2bit (LSB first  increase Δ) X : original value U : quantized value Y : side information in the decoder given Y + sent 10  U=101

14 J.S.Kim History of DVC  Slepian and Wolf : lossless DVC (1973)  “Noiseless coding of correlated information sources,” IEEE Tr. On Information Theory,  Wyner and Ziv : lossy DVC (1976)  “The rate-distortion function for source coding with side information at the decoder,” IEEE Tr. Information Theory,  Ramchandran in Berkeley : PRISM (2002)  P ower-efficient, R obust, h I gh-compression, S yndrome-based M ultimedia coding  Girod in Stanford : Good review (2005)  “Distributed video coding,” IEEE Proceedings,  EU : DISCOVER(~2006),  DIStributed COding for Video sERvices

15 J.S.Kim Towards Practical Slepian-Wolf Coding  Convolution coding for data compression [Blizard, 1969]  Convolutional source coding [Hellman, 1975]  Syndrome source coding [Ancheta, 1976]  Coset codes [Pradhan and Ramchandran, 1999]  Trellis codes [Wang and Orchard, 2001]  Turbo codes [García-Frías and Zhao, 2001] [Bajcsy and Mitran, 2001] [Aaron and Girod, 2002]  LDPC codes [Liveris, Xiong, and Georghiades, 2002] ...

16 J.S.Kim  Generation of S.I. at the Decoder

17 J.S.Kim Motion Compensation  Motion-compensated interpolation (MC-I) using the decoded Key frame at time t-1 & t+1

18 J.S.Kim Side Information

19 J.S.Kim Motion Compensation  Motion-compensated extrapolation (MC-E) estimate the motion between the Wyner-ziv frame a t time t-2 and the Key frame at time t-1

20 J.S.Kim Side Information

21 J.S.Kim Motion Compensation

22 J.S.Kim  How to Encode WZ frames

23 J.S.Kim Wyner-Ziv Residual Video Codec WZ Encoder WZ frames X WZ Decoder Y X’ [Aaron, Zhang, Girod, Asilomar 2002] Xer W Residual of a frame with respect to an encoder reference frame ( Xer ) is fed into a Wyner-Ziv encoder. To avoid drift, Xer should be replicable at the decoder. Since the decoder takes into account motion, Y is expected to be a better estimate of frame X than Xer. The Wyner-Ziv decoder uses both Y and Xer to calculate the reconstruction X’.

24 J.S.Kim Pixel-Domain Wyner-Ziv Video Codec Interframe Decoder Scalar Quantizer Turbo Encoder Buffe r WZ frames W Intraframe Encoder Turbo Decoder Request bits Slepian-Wolf Codec Interpolation/ Extrapolation Reconstruction Y Key frames I Conventional Intraframe coding Conventional Intraframe decoding W’ I’ Side informati on [Aaron, Zhang, Girod, Asilomar 2002]

25 J.S.Kim Pixel-Domain Wyner-Ziv Video Codec Decoder side information generated by motion- compensated interpolation PSNR 24.8 dB After Wyner-Ziv Decoding 16-level quantization – 2.0 bpp 0 pixels in error PSNR 36.5 dB [Aaron, Zhang, Girod, Asilomar 2002]

26 J.S.Kim YkYk IDCT DCT-Domain Wyner-Ziv Video Codec Request bits Interpolation/ Extrapolation Recon I Conventional Intraframe coding Conventional Intraframe decoding DCT For each transform band k I’ W’ Y Scalar Quantizer DCT Turbo Encoder Buffer Turbo Decoder Side information WZ frames W Key frames DkDk Dk’Dk’ Interframe DecoderIntraframe Encoder [Aaron, Zhang, Girod, Asilomar 2003]

27 J.S.Kim Rate-Distortion Performance - Salesman Every 8 th frame is a key frame Salesman QCIF sequence at 10fps 100 frames 6 dB 3 dB Interframe 100% Encoder Runtime Pentium 1.73 GHz machine [Aaron, Zhang, Girod, Asilomar 2003]

DCT-based Intracoding 149 kbps PSNR Y =30.0 dB Wyner-Ziv DCT codec 152 kbps PSNR Y =35.6 dB GOP=8 Salesman at 10 fps [Aaron, Zhang, Girod, Asilomar 2003]

29 J.S.Kim Conclusion  Increase efficiency of DVC  Reduce H(X) : simple ME/MC?  Increase H(Y) : better interpolation/extrapolation  Stronger correlation between X and Y. encoderdecoder XX R Y X? Y? P(X/Y)

30 J.S.Kim Conclusion  Distributed coding is a fundamentally new paradigm for video co mpression  Slepian-Wolf encoding, is fundamentally harder for practical appli cations due to the general statistics of the correlation channel  The rate-distortion performance of Wyner-Ziv coding does not yet reach the performance of conventional interframe coder  It is unlikely that distributed video coding algorithm will ever beat conventional video coding schemes in R-D performance  Many authors believe that distributed coding techniques will soon complement conventional video coding to provide the best overal l system performance and enable novel applications

31 J.S.Kim  Research Plan ( with M.S. Vidhya Murthy )

32 J.S.Kim plan (Aug.2008 – June 2009) ▣ Simulation Environment and survey WZ CODEC - survey WZ CODEC - joint simulation model of H.264 CODEC ▣ Survey of Channel Coding Algorithm - channel coding/decoding algorithm - analysis of channel source code ▣ Implementation of Wyner-Ziv CODEC - Performance comparison of Pixel- and Transform-domain - Investigate the statistical characteristics and distributions of the residual signal - Efficient encoding algorithm (Quantization/Entropy coding or Slepian-Wolf encoding/decoding) - Efficient generation of Side Information - Implementation of channel encoding/decoding algorithm - Rate control (or buffer control) algorithm ■ Plan and achievements Research Plan Plan done Now

33 J.S.Kim  Q&A Thank you