1. SIDE INFORMATION GENERATION IN
H.264 STANDARD BASED
SIDE INFORMATION GENERATION IN
WYNER-ZIV CODING
Subrahmanya M V
(Under the guidance of Dr. Rao and Dr.Jin-soo Kim)

2. Traditional Video coding Wyner-Ziv video coding Applications
Contents
Traditional Video coding
Wyner-Ziv video coding
Applications
Wyner-Ziv encoder
Key frame encoding
WZ frame encoding
Wyner-Ziv decoder
Key frame decoding
WZ frame decoding

3. Side Information Generation
Contents( contd.)
Side Information Generation
Inter-prediction
Intra-prediction
Motion estimation (ME)
Full-pixel
Half-pixel
Quarter-pixel
Motion compensation (MC)
Luma
Chroma
Bi-directional interpolation

4. Implementation Testing Results Conclusions References
Contents( contd.)
Implementation
Testing
Results
SI prediction schemes
ME block sizes
ME search ranges
Key frame distances
Bit rates
Conclusions
References

5. Traditional Video coding
Source coding
Encoder removes temporal and spatial redundancies
Intra frame encoding
Inter frame encoding: Computationally intensive
Complex Encoder + Simple decoder
[Dr. Jin-soo Kim]

6. Wyner-Ziv video coding
Source coding + Channel coding
Decoding with side-information(SI)
Slepian-Wolf coding: Lossless
Wyner-Ziv coding: Lossy
Simple encoder + Complex decoder
[Dr. Jin-soo Kim]

7. Applications Light encoder and light decoder [20]
Wireless low power video surveillance
Sensor network
Multi-view image acquisition
Networked camcoders
[Dr. Jin-soo Kim]

8. Wyner-Ziv Encoder Input YUV 420 Output: Bitstream
Encoding Key frame + WZ frame

9. Key frame encoding H.264 Intra frame coding Intra prediction
Integer Transform
Quantization
Entropy Coding
[M. Lee, A. Moore]

10. Intra prediction Predicted from top, left, top left and top right
neighbors
4x4 intra prediction
Nine directional modes
[T. Wiegand and G.J Sullivan]
[J. S. Park and H. J. Song]

11. Key frame encoding Integer Transform Quantization Entropy Coding using
Element by element Multiplication
Entropy Coding using
Exp-Golomb codes

12. WZ frame encoding Residual signal Generation [Dr. Jin-soo Kim]

13. WZ frame encoding(contd.)
Residual bit plane extraction

14. WZ frame encoding(contd.)
Low Density Parity Check(LDPC) encoding
Obtain parity bits for the message bits
Message bits(90%) + Parity bits(10%)
Only parity bits are sent to achieve compression

15. Wyner-Ziv Decoder Decoding Key frame + WZ frame Key Frame WZ Frame
[E. Peixoto, R. L. de Queiroz, and D. Mukherjee]

16. Wyner-Ziv Decoder( Contd. )
Key frame decoding
Intra prediction
Entropy decoding
Inverse Quantization
Inverse Integer transform

17. Key frame decoding Inverse Integer Transform Inverse Quantization
Element by Element Multiplication
Entropy decoding
Exp-Golomb codes are decoded using prefix tables

18. Wyner-Ziv Decoder( Contd. )
WZ frame decoding
SI generation: Prediction to obtain the message bits
LDPC decoding: Correct prediction errors using parity bits
Message contains a pair of values [qij(0), qij(1)] or [rij(0), rij(1)] – Signify the amount of belief that yi is a ”0” or a ”1”.
Initial belief is based on message bits predicted and parity bits sent
Belief is iteratively updated and new message bits are obtained
Iteration proceeds till fixed iteration count or message bits remain unchanged in an iteration

19. Side Information Generation
Motion estimation
Estimate motion of a block between two key frames to obtain motion vectors( MVx, MVy ) for each block
Interpolate motion vectors to obtain motion between key frame and WZ frame
Motion Compensation
Interpolate key frame based on motion vectors to obtain MC prediction for WZ frame
Intra prediction
Prediction for blocks which are absent in key frames

20. Forward ME between key frames – MVF
Motion Estimation
[E. Peixoto, R. L. de Queiroz, and D. Mukherjee 2008]
Forward ME between key frames – MVF
Backward ME between key frames - MVB
Derive MVs for WZ frame using Key frame MVS

21. Higher key frame distance
[A. Aaron, E. Setton and B. Girod 2003]
For larger key frame distance hierarchical structure can be used

22. Best match is a block with least sum of absolute differences
SAD based ME
(W+2SR) X (H+2SR)
Obtain absolute difference between each pixel in the block to be searched and the reference block
Best match is a block with least sum of absolute differences

23. Motion vector is estimated up to quarter-pixel units
Sub-pixel ME
Motion vector is estimated up to quarter-pixel units
Interpolation of reference is done to obtain half-pixel and quarter-pixel units
Half-pixel positions are obtained using five tap filter
Quarter pixels are obtained using bilinear interpolation

24. Half-pixel motion estimation
H33 = [F * F * F * F * F53 + F ] >> 5
G33 = [F * F * F * F * F35 + F ] >> 5
D33 = [H * H * H * H * H35 + H ] >> 5

25. quarter-pixel motion estimation
q1 = ( F33 + G ) >> 1
q2 = ( G33 + F ) >> 1
q3 = ( F33 + H ) >> 1
q4 = ( H33 + G ) >> 1
q5 = (G33 + D ) >> 1
q6 = ( G33 + H ) >> 1
q7 = ( H33 + D ) >> 1
q8 = ( D33 + H ) >> 1
q9 = ( H33 + F ) >> 1
q10 = ( H33 + G ) >> 1
q11 = ( D33 + G ) >> 1
q12 = ( G43 + H ) >> 1

26. Motion Compensation (Luma)
Obtain a block from key frame with offset specified by motion vector

27. Motion Compensation (Chroma)
Chroma MV = Luma MV / 2
Obtain A, B, C, D based on integer part of MV
Floor(MV/8)
Predict based on fractional part of MV
(MV & 7)

28. Bi-directional interpolation
SI frame = (Forward prediction Frame + Backward Prediction frame + 1)/2

29. Prediction mode of co-located block from previous key frame
Intra prediction
4x4 prediction
Prediction mode of co-located block from previous key frame

30. Key frame encoding is done using JM encoder
Implementation
Encoder
Key frame encoding is done using JM encoder
WZ frame decoding is implemented in ‘C’
Residual generation
Quantization
Bit plane extraction
LDPC encoder
Decoder
Key frame is decoded using JM decoder
SI generation
LDPC decoder
De-quantization
Reconstruction
LDPC encoder and decoder are implemented by Dr. Jin- soo Kim [4]

31. Comparison with traditional coding
Testing
Comparison with traditional coding
H.264 is used for comparison
Generate H.264 reconstructed sequence
Encode using H.264 encoder
Decode using H.264 decoder to obtain reconstructed YUV sequence
Generate Wyner-Ziv reconstructed sequence
Encode using Wyner-Ziv encoder
Decode using Wyner-Ziv decoder to obtain reconstructed YUV sequence
Compare two sequences using PSNR and SSIM
Significance of different parameters in Wyner-Ziv coding
SI Prediction method
Key frame distance
ME search range
ME block size

32. SI prediction schemes(Sample reconstructed images)
bus( CIF) [Average, Full pixel ME, Half-pixel ME, Quarter-pixel ME, Bi-prediction, Intra-prediction combined]

33. SI prediction schemes( Plots)

34. ME block sizes(Sample reconstructed images)
coastgaurd (CIF) [16x16, 8x8, 4x4]

35. ME block sizes( Plots)

36. ME search ranges (Sample reconstructed images)
stefan (CIF) [16, 8, 4]

37. ME search ranges( Plots)

38. Key frame distances (Sample reconstructed images)
garden (CIF) [1, 2, 3, 4, 5]

39. Key frame distances( Plots)

40. Comparison with H.264 (Sample reconstructed images)

41. Comparison with H.264( Plots)

42. Higher ME search range produces better results
Conclusions
H.264 based SI generation based on quarter- pixel ME performs better compared to previously done H.263which is based on half- pixel ME[1] .
ME block size 16x16 is ideal
Higher ME search range produces better results
At higher key frame distance quality drops sharply
Traditional video coding has better R-D performance

43. Wyner-Ziv to H.264 transcoder
Future work
Wyner-Ziv to H.264 transcoder
Light encoder + light decoder
Inter and intra prediction done in Wyner-Ziv decoder can be reused for the H.264 encoder

44. Accepted conference paper
Subrahmanya Venkatrav and Dr. K. R. Rao,
“Side information generation in Wyner-Ziv
decoder”,
The 7th International Joint Conference on
Computer Science and Software Engineering
May 13-14, Bangkok, Thailand

45. References
E. Peixoto, R. L. de Queiroz, and D. Mukherjee, “Mobile video communications using a Wyner-Ziv transcoder,” Proc. SPIE 6822, VCIP, R Jan
A. Aaron, E. Setton and B. Girod, "Towards practical Wyner-Ziv coding of video," Proceedings. IEEE International Conference on Image Processing, ICIP 2003., vol.3, pp. III , Sept
K. R. Rao and J. J. Hwang, Techniques and standards for image, video, and audio coding, Prentice Hall PTR, 1996.
Jin-Soo Kim, "Brief overview of Wyner-Ziv CODEC" (Private Communication)
A. Aaron, D. Varodayan, and B. Girod, “Wyner-Ziv residual coding of video,” Proc. International Picture Coding Symposium, Beijing, P. R. China , April 2006.
T. Wiegand and G.J Sullivan, “The H.264/AVC video coding standard”, IEEE SP Magazine, vol. 24, pp , March 2007.
G. J. Sullivan, P. Topiwala, and A. Luthra, "The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions", SPIE Conf. on applications of digital image processing XXVII, vol. 5558, pp , Aug
S.K. Kwon, A. Tamhankar, and K.R. Rao “Overview of H.264/MPEG-4 Part 10” J. VCIR, Vol. 17, pp , April 2006, Special Issue on “Emerging H.264/AVC Video Coding Standard,”.

46. References(Contd)
A. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder,"  IEEE Trans., Information Theory, vol.22, pp , Jan 1976.
D. Slepian and J. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. on Information Theory Vol. 19, pp. 471–480, July 1973.
D. Varodayan, A. Aaron and B. Girod, "Rate-adaptive distributed source coding using low-density parity-check codes,"  Conference Record of the Thirty- Ninth Asilomar Conference on Signals, Systems and Computers, pp , Oct. 28 – Nov. 1, 2005.
Z. Li and E.J. Delp, "Wyner-Ziv video side estimator: conventional motion search methods revisited," IEEE International Conference on Image Processing, ICIP 2005, vol.1, pp. I , Sept
L. Liu and E. J. Delp, "Wyner-Ziv video coding using LDPC codes," Proceedings of the 7th Nordic Signal Processing Symposium, NORSIG 2006.
 D. Kubasov, K. Lajnef and C. Guillemot, "A hybrid encoder/decoder rate control for Wyner-Ziv video coding with a feedback channel," IEEE 9th Workshop on Multimedia Signal Processing, MMSP 2007., pp , 1-3 Oct
C. Brites and F. Pereira, "Encoder rate control for transform domain Wyner- Ziv video coding," IEEE International Conference on Image Processing, ICIP 2007., vol.2, pp.II -5-II -8, Sept. 2007
A. Roca, et al, "Rate control algorithm for pixel-domain Wyner-Ziv video coding ," Proc. SPIE, vol. 6822, 68221T (2008).

47. References(Contd)
D. Mukherjee, “Optimal parameter choice for Wyner-Ziv coding of Laplacian sources with decoder side information,” HP Labs Technical Report HPL , 2007.
Z. Wang, et al, "Image quality assessment: From error visibility to structural similarity," IEEE Trans., Image Processing, vol.13, pp , April 2004
I. Richardson, “The H.264 advanced video compression standard,” Hoboken, NJ: Wiley, 2010.
D. Rebollo-Monedero, S. Rane, A. Aaron and B. Girod, "High-rate quantization and transform coding with side information at the decoder," EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding.
S.-K Kwon, A. Tamhankar and K.R. Rao, ‘Overview of H.264 / MPEG-4 Part 10” ISME, Hong Kong, Oct
JVT documents
ftp://standards.polycom.com
M. Lee and A. Moore, “H.264 Encoder Design“, Group 3, May 17, 2006
J. S. Park and H. J. Song, “Selective Intra Prediction Mode Decision for H.264 /AVC Encoders”, World Academy of Science, Engineering and Technology
B. M.J. Leiner, “LDPC Codes – a brief Tutorial”, April 8, 2005

48. References(Contd)
G. Cote, B. Erol, M. Gallant, and F. Kossentini, “H.263+: Video coding at low bit rates,” IEEE Trans. Circuits and Systems for Video Technology ,Vol.8, pp. 849–866, Nov. 1998
(2008, August) H.264/avc JM reference software. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG. [Online]. Available: