1. 1 Department of Electrical Engineering, Stanford University EE 392J Final Project Presentation Shantanu Rane Hash-Aided Motion Estimation & Rate Control For Distributed Video Coding

2. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 2 Interframe Video Compression  Interframe predictive coding for compression  Encoder is 5-10 times more complex than decoder X’ i-1 Interframe Encoder Interframe Decoder XiXi Xi’Xi’ Standard codec

3. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 3 Low-complexity Video Coding Wyner-Ziv Coding Lossy source coding with decoder side information Interframe Decoder Intraframe Encoder XiXi X i-1 ’ Xi’Xi’ Side Information  Low-complexity encoding, possibly more complex decoding  Applications: video sensors for surveillance, wireless PC cameras, mobile camera-phones Imagine dependence channel between source and side-info. WZ coding = sending parity symbols to correct errors in the dependence channel.

4. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 4 Outline  How to improve side-information at decoder w/o access to current frame?  Hash-Aided Motion Estimation  How to decide encoding bit-rate needed to correct errors in dependence channel?  Hash-Aided Rate Control Project motivated by Wyner-Ziv video codec architecture described in [Aaron et al., VCIP ’04, ICIP ’03].

5. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 5 Generating the Hash  Visual hash originally used in digital watermarking [Fridrich, 1997]  E.g., Generate a 16 bit hash for an image-block of size NxN.  Generate 16 NxN matrices with elements i.i.d. ~ U[0,1].  Low-pass filter each matrix. (e.g., repeated 2x2 averaging.)  Write out image block as vector  Write out each low-pass pattern as vector  Find inner products  ……

6. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 6 Hash-aided Motion Estimation Projecting onto low frequency patterns is equivalent to finding the low-frequency DCT coefficients. Similar looking blocks have almost equal low-frequency DCT coefficients and produce almost similar hashes. Frame n-1 Frame n Side info Unavailable Available B=00110110 received from encoder as helper information A=00111110 Motion Compensated Side info

7. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 7 Motion-Estimation Results 2.5 dB 3.3 dB

8. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 8 Motion-Estimation Results Foreman QCIF Frames 301 to 350 Error between current frame and side information With previous frame As side info With hash-aided motion estimation with excess bit-rate = 0.14 bpp

9. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 9 Hash-Aided Rate Control  Encoder can perform motion-estimation at low-complexity, with small hash store.  Experimentally found that there is a high correlation between MSE of MC prediction and parity bitrate, which the encoder needs to transmit.  For given code parameters, can characterize the required bitrate as a function of MSE of MC prediction.

10. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 10 Rate Control Results Avg Hash bitrate = 0.175 bpp Avg WZ bitrate = 1.27 bpp

11. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 11 Conclusions  Hash-aided motion estimation can be used at the decoder to improve the quality of the side-information at very low bit-rate overhead. This reduces the overall Wyner-Ziv bit-rate.  With a small complexity addition, motion estimation can be performed at the encoder. This enables the encoder to control the Wyner-Ziv bit-rate depending upon the quality of the MC prediction.

12. S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 12 Wyner-Ziv Encoder Wyner-Ziv Decoder Input current video frame Side Info: previous decoded frame(s) + Helper information from encoder Hash-aided motion estimation Motion compensated prediction (extrapolation) Frames encoded independently Frames decoded conditionally Decoded Frame Hash-aided rate control