1. Overview of Fine Granularity Scalability in MPEG-4 Video Standard Weiping Li Presented by : Brian Eriksson

2. Introduction - Problem Tradition System –Video encoded close to channel capacity –Decoded using all of the bits Internet Video System –Channel capacity changes –Partially decodable within a bit range

3. Introduction – Video Coding Performance LowHigh Channel Bitrate Received Quality Bad Good = Distortion- Rate Curve = Dialup Rate = Broadband Rate = Layered Coding = Desired Coding

4. LowHigh Channel Bitrate Received Quality Bad Good Previous Layered Coding Methods = Layered Coding

5. MPEG-2 Overview I-Frame – Intracoded Picture –Contains all frame information P-Frame – Predictive-Coded Picture –Uses previous frame and motion information to reconstruct frame B-Frame – Bidirectional-Coded Picture –Uses previous frame, next frame, and motion information to reconstruct the frame

6. Temporal Scalability Video is encoded into two layers with equal resolution but using different prediction. Base layer has lower frame rate BaseEnh. BaseEnh. BaseEnh. BaseXX XX XX With Enhancement Layer : Frame Rate = 30 fps Without Enhancement Layer : Frame Rate = 10 fps

7. MPEG-2 SNR Scalability Enhancement Layer Stream Base Layer Stream + IDCT Video Output Motion prediction includes enhancement information Efficiency dependent on two factors Encoder uses enhancement layer (drift can occur) Decoder receives enhancement layer Motion Compensation +

8. Spatial Scalability IDCT Enhancement Bitstream Base Bitstream (Motion Compensated) Upsampler + Enhancement Layer Video Base Layer Video Same frame rate, different resolutions Base frame = 128x128, enhancement = 256x256 Enhancement Layer not in prediction loop

9. LowHigh Channel Bitrate Received Quality Bad Good MPEG-4 Fine Granularity Scalability Technique = Desired Coding = Distortion- Rate Curve

10. Bitplane Coding Technique Given vector X: X = [-12, -53, 62, -7, 31, …,180,-43, …,5] Convert to signed magnitude Separate into sign bit and absolute values

11. Bitplane Coding Technique Sign(X) = [0, 0, 1, 0, 1, …,1,0, …,1] Abs(X) = [12, 53, 62, 7,31, …,180,43, …,5] To find the number of bitplanes needed, find the maximum value of abs(X) Max(abs(X)) = 180 = 10110100b 8 bitplanes Sign bit plane 1 bitplane

12. Bitplane Coding Technique c Sign(X) = [0, 0, 1, 0, 1, …,1,0, …,1] X = [-12, -53, 62, -7, 31, …,180,-43, …,5] 0,1,0,1,1, …,0,1, …,1 0,0,1,1,1, …,0,1, …,0 1,1,1,1,1, …,1,0, …,1 1,0,1,0,1, …,0,1, …,0 0,1,1,0,1, …,1,0, …,0 0,1,1,0,0, …,1,1, …,0 0,0,0,0,0, …,0,0, …,0 0,0,0,0,0, …,1,0, …,0 0,0,1,0,1, …,1,0, …,1

13. Run Level Encoding Symbol : (RUN, EOP) –RUN = Number of Consecutive Zeros before a 1 –EOP = 0 if there are more ones –EOP = 1 if the rest of the line are zeros. Example: –{ 1,0,1,0,0,0,1,0,0,0,…} = (0,0),(1,0),(3,1) –{0,0,0,1,0,0,0,…} = (3,1)

14. MPEG-4 Bitplane FGS Technique Base layer reaches lower bound of bit-range Divide image into 8x8 DCT blocks Divide blocks into Y,U,V color components Use bitplane run-level coding to encode/decode

15. Fine Granularity Scalability Encoder + - DCTBitplane Encoding Original Signal Base Layer Signal (Motion Compensated) Enhancement Layer Signal

16. Fine Granularity Scalability Decoder Bitplane Decoding IDCT + Base Layer Video (Motion Compensated) Enhancement Layer Video

17. Truncated Bitplane c Number of Truncated Layers = 0 Received X = [-12, -53, 62, -7,31, …,180,-43, …,-5] Original X = [-12, -53, 62, -7,31, …,180,-43, …,-5] 0,1,0,1,1, …,0,1, …,1 0,0,1,1,1, …,0,1, …,0 1,1,1,1,1, …,1,0, …,1 1,0,1,0,1, …,0,1, …,0 0,1,1,0,1, …,1,0, …,0 0,1,1,0,0, …,1,1, …,0 0,0,0,0,0, …,0,0, …,0 0,0,0,0,0, …,1,0, …,00,0,1,0,1, …,1,0, …,1 c Number of Truncated Layers = 1 Received X = [-12, -52, 62, -6,30, …,180,-42, …,-4] Original X = [-12, -53, 62, -7,31, …,180,-43, …,-5] 0,0,0,0,0, …,0,0, …,0 0,0,1,1,1, …,0,1, …,0 1,1,1,1,1, …,1,0, …,1 1,0,1,0,1, …,0,1, …,0 0,1,1,0,1, …,1,0, …,0 0,1,1,0,0, …,1,1, …,0 0,0,0,0,0, …,0,0, …,0 0,0,0,0,0, …,1,0, …,00,0,1,0,1, …,1,0, …,1 c Number of Truncated Layers = 2 Received X = [-12, -52, 60, -4,28, …,180,-40, …,-4] Original X = [-12, -53, 62, -7,31, …,180,-43, …,-5] 0,0,0,0,0, …,0,0, …,0 1,1,1,1,1, …,1,0, …,1 1,0,1,0,1, …,0,1, …,0 0,1,1,0,1, …,1,0, …,0 0,1,1,0,0, …,1,1, …,0 0,0,0,0,0, …,0,0, …,0 0,0,0,0,0, …,1,0, …,00,0,1,0,1, …,1,0, …,1 c Number of Truncated Layers = 3 Received X = [-8, -48, 56, -0,24, …,176,-40, …,-0] Original X = [-12, -53, 62, -7,31, …,180,-43, …,-5] 0,0,0,0,0, …,0,0, …,0 1,0,1,0,1, …,0,1, …,0 0,1,1,0,1, …,1,0, …,0 0,1,1,0,0, …,1,1, …,0 0,0,0,0,0, …,0,0, …,0 0,0,0,0,0, …,1,0, …,00,0,1,0,1, …,1,0, …,1

18. Advanced Bitplane Techniques

19. Frequency Weighting in Bitplane Normal Bitplane DCT Index Frequency Weighted Bitplane DCT Index

20. Selectively Enhanced Bitplane Detect visually significant area Shift upward in bitplane More likely to be included in truncated bitstream

21. Other Methods Error Resilience in Bitplane Random burst errors in the bitstream Resynchronization markers are used to resynch. Temporal Scalability in Bitplane Uses FGS to encode/decode the entire temporal enhancement frame

22. Simulation - DCT

23. Simulation - DWT

24. Selectively Enhanced / Weighted Bitplane Setup

25. Simulation – Enhanced Weighted Bitplane 1 Layer Removed 2 Layers Removed 3 Layers Removed4 Layers Removed5 Layers Removed6 Layers Removed

26. Simulation – Li

27. Conclusions FGS Bitplane method allows for quality parallel to distortion-rate curve A wavelet-based approach may yield better results Fairly simple implementation

28. Questions?