1. Overview of Fine Granularity Scalability in MPEG-4 Video Standard Weiping Li, Fellow, IEEE

2. System configuration for Internet Streaming Video

3. FGS Scalability Structure n e(n) n+1 e(n+1) n+2 e(n+2) n+3 e(n+3)

4. Layered scalable coding Tech. SNR scalability Temporal scalability Spatial scalability

5. Illustration of video coding performance

6. SNR scalability decoder defined in MPEG-2

7. Layered scalable coding Tech. SNR scalability If the encoder uses the enhancement-layer information in the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, the enhancement-layer coding efficiency is high (low). If the encoder does not use the enhancement-layer information in the motion-prediction loop and the enhancement-layer information is (not) received by the decoder, drift happens in the enhancement layer and coding efficiency is low. (the result is the same as using the base layer only.)

8. Layered scalable coding Tech. Temporal scalability

9. Layered scalable coding Tech. Spatial scalability

10. BIT-PLANE CODING OF THE DCT COEFFICIENTS

11. ← CCIR-601 Sequences SIF and QCIF Sequences →

12. FGS USING BIT-PLANE CODING OF DCT COEFFICIENTS Overall Coding Structure of FGS Some Details of FGS Coding Profile Definitions in the Amendment of MPEG-4

13. Overall Coding Structure of FGS FGS encoder structure

14. Overall Coding Structure of FGS FGS decoder structure

15. Some Details of FGS Coding 1) Different Numbers of Bit-Planes for Individual Color Components 2) Variable-Length Codes 3) Decoding Truncated Bitstreams 4) Variations to the Standardized FGS Coding Structure

16. Different Numbers of Bit-Planes for Individual Color Components

17. Variable-Length Codes Statistics of the (RUN, EOP) symbols in the four VLC tables

18. Coding patterns for syntax element fgs_cbp

20. Decoding Truncated Bitstreams Decoding of the truncated bitstream is not standardized in MPEG-4. One possible method To look ahead 32 bits at every byte-aligned position in the bitstream. If the 32 bits are not fgs vop start code, the first 8 bits of the 32 bits are information bits of the FGS frame to be decoded. The decoder slides the bitstream pointer by one byte and looks ahead another 32 bits to check for fgs vop start code.

21. Variations to the Standardized FGS Coding Structure A possible variation of FGS encoder structure DCT Domain

22. Variations to the Standardized FGS Coding Structure A possible variation of FGS decoder structure

23. “Find Reference” To generate the reference signal in the DCT domain to be subtracted in the encoder and added in the decoder. The operation of taking residue is slightly different in the variation method.

24. The distribution of such a residual signal Taking Residue between Original and Reconstructed DCT Coefficients.

25. The distribution of such a residual signal Taking residue between original and lower boundary point of quantization bin.

26. Comparison of two different ways of generating the DCT residues

27. Encoder structure without the DCT unit in the enhancement layer

28. Experiment results on mismatch errors

29. Profile Definitions in the Amendment of MPEG-4 Advanced Simple Profile contains a subset of nonscalable video-coding tools to achieve high coding efficiency at any given bit rate within a wide range of bit rates. The base-layer coding tools defined in the Advanced Simple Profile include both P-VOP (forward pre-diction only) and B-VOP (bi- directional prediction) for coding motion- compensated residues. The FGS profile is defined to meet the requirement of optimizing video quality over a given bit rate range.

30. CODING EFFICIENCY PERFORMANCE OF FGS FGS versus Multilayer SNR Scalability FGS versus Non-Scalable Coding FGS versus Simulcast

31. Compare FGS with multilyer SNR scalability

32. Compare FGS with non- scalable coding

33. Compare FGS with simulcast

34. ADVANCED FEATURES IN FGS Frequency Weighting Selective Enhancement Error Resilience FGS Temporal Scalability

35. Frequency Weighting

36. Selective Enhancement

37. Error Resilience

38. FGS Temporal Scalability FGST organized into a separate layer from FGS FGST and FGS organized into a single enhancement layer.

39. Conclusion Advantages of using FGS for Internet streaming video applications It allows separation of encoding and transmission. The server can transmit enhancement layer at any bit rate without transcoding. It provides a solution to the video server overload problem.