1. Overview of the Scalable Video Coding
Dan Grois
Ben-Gurion University of the Negev
Based on the presentation of Heiko Schwarz, Detlev Marpe, and Thomas Wiegand,
and the presentation of Thomas Wiegand:
Fraunhofer Institute, Nchrichtentechnik, Heinrich-Hertz Institute.
9/4/2018
Dan Grois, Ph.D. Candidate

2. Outline Introduction Problems Definition Functionality Goal
Competition
Applications
Targets
History of SVC
Structure of SVC
Temporal Scalability
Spatial Scalability
Quality Scalability
Combined Scalability
Profiles of SVC
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

3. Introduction - problem
Non-Scalable Video Streaming
Multiple video streams are needed for heterogeneous clients
8Mb/s
512Kb/s
1Mb/s
6Mb/s
4Mb/s
9/4/2018
Dan Grois, Ph.D. Candidate

4. Introduction - definition
Scalable video stream
Scalability
Removal of parts of the video bit-stream to adapt to the various needs of end users and to varying terminal capabilities or network conditions
Sub-stream n
Sub-stream ki
High quality … …
reconstruction
Sub-stream 2
Sub-stream k2
Sub-stream 1
Sub-stream k1
Low quality
9/4/2018
Dan Grois, Ph.D. Candidate

5. Scalability of Video - Modalities

6. Scalability of Video – Modalities (Cont.)
Dan Grois, Ph.D. Candidate
9/4/2018

7. Introduction – functionality

8. Introduction – functionality (Cont.)
Functionality of SVC
Bit-rate adaptation to match the channel throughput
Format adaptation for backwards compatible extension
Power adaptation for trade-off between runtime and quality
Graceful degradation when “right” parts of the bit-stream are lost
9/4/2018
Dan Grois, Ph.D. Candidate

9. Introduction - goal Goal of SVC Scalability mode
Fidelity reduction (SNR scalability)
Picture size reduction (spatial scalability)
Frame rate reduction (temporal scalability)
Selection of content (ROI or object-based scalability)
Sub-stream ki
H.264/AVC bit-stream … =
(Quality)
Sub-stream k2
Sub-stream k1
9/4/2018
Dan Grois, Ph.D. Candidate

10. Introduction - competition
SVC is an old research topic (> 20 years) and has been included in H.262/MPEG-2, H.263, and MPEG-4 Visual.
Rarely used because
The characteristics of traditional video transmission systems
Significant loss of coding efficiency and large increase in decoder complexity
Competition
Simulcast
Transcoding
9/4/2018
Dan Grois, Ph.D. Candidate

11. Introduction - applications
Heterogeneous clients
Unequal protection
Surveillance
Problems of SVC
Increased decoder complexity
Decreased coding efficiency
Temporal scalability is more often supported than spatial and quality scalability.
9/4/2018
Dan Grois, Ph.D. Candidate

12. Introduction - targets
Little decrease in coding efficiency
Little increase in decoding complexity
Support of temporal, spatial, and quality scalability
A backward compatible base layer
Simple bit-stream adaptations after encoding
9/4/2018
Dan Grois, Ph.D. Candidate

13. Outline Introduction History of SVC Structure of SVC
Temporal Scalability
Hierarchical prediction structure
Spatial Scalability
Quality Scalability
Combined Scalability
Profiles of SVC
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

14. Temporal Scalability Hierarchical prediction structures 9/4/2018
Dan Grois, Ph.D. Candidate

15. Temporal Scalability (Cont.)
9/4/2018
Dan Grois, Ph.D. Candidate

16. Temporal Scalability Combination with multiple reference picture
Arbitrary modification of the prediction structure
Issue of quantization
Lower layers with higher fidelity  Smaller QPs are used in lower layers
Propagation of quantization error  smaller QPs are used in higher layers
9/4/2018
Dan Grois, Ph.D. Candidate

17. Temporal Scalability Video Coding Experiment with H.264/MPEG4-AVC
Foreman, CIF 1320kbps
Performance as a function of N
Cascaded QP assignment
QP(P)  QP(B0)-3  QP(B1)-4  QP(B2)-5
9/4/2018
Dan Grois, Ph.D. Candidate
This slide is copied from JVT-W132-Talk

18. Temporal Scalability Coding efficiency of hierarchical prediction
9/4/2018
Dan Grois, Ph.D. Candidate

19. Outline Introduction History of SVC Structure of SVC
Temporal Scalability
Spatial Scalability
Inter layer prediction
Quality Scalability
Combined Scalability
Profiles of SVC
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

20. Spatial Scalability texture Hierarchical MCP & Intra-prediction
Base layer coding
motion
Inter-layer prediction
Intra
Motion
Residual
Spatial decimation
Hierarchical MCP & Intra-prediction
texture
Base layer coding
Multiplex
Scalable bit-stream
motion
Inter-layer prediction
Intra
Motion
Residual
Spatial decimation
H.264/AVC compatible base layer bit-stream
H.264/AVC MCP & Intra-prediction
texture
Base layer coding
motion
H.264/AVC compatible coder
9/4/2018
Dan Grois, Ph.D. Candidate

21. Spatial Scalability Similar to MPEG-2, H.263, and MPEG-4
Arbitrary resolution ratio
The same coding order in all spatial layers
Combination with temporal scalability
Inter-layer prediction
Spatial 1
Temporal 2
Intra
Spatial 0
Temporal 0
Temporal 1
Intra
9/4/2018
Dan Grois, Ph.D. Candidate

22. Spatial Scalability The prediction signals are formed by
MCP inside the enhancement layer (Temporal)
Up-sampling from the lower layer (Spatial)
Average of the above two predictions (Temporal + Spatial)
Inter-layer prediction
Three kinds of inter-layer prediction
Inter-layer motion prediction
Inter-layer residual prediction
Inter-layer intra prediction
Base mode MB
Only residual are transmitted, but no additional side info.
9/4/2018
Dan Grois, Ph.D. Candidate

23. Spatial Scalability Past spatial scalable video: Single-loop decoding
Inter-layer intra prediction requires completely decoding of base layer.
Multiple motion compensation and deblocking filter are needed.
Full decoding + inter-layer prediction: complexity > simulcast.
Single-loop decoding
Inter-layer intra prediction is restricted to MBs for which the co-located base layer is intra-coded
9/4/2018
Dan Grois, Ph.D. Candidate

24. Spatial Scalability Generalized spatial scalability in SVC
Arbitrary ratio
Only restriction: Neither the horizontal nor the vertical resolution can decrease from one layer to the next.
Cropping
Containing new regions
Higher quality of interesting regions
9/4/2018
Dan Grois, Ph.D. Candidate

25. 9/4/2018
Dan Grois, Ph.D. Candidate

26. Outline Introduction History of SVC Structure of SVC
Temporal Scalability
Spatial Scalability
Quality Scalability
CGS
MGS
Drift control
Combined Scalability
Profiles of SVC
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

27. Coarse-grain quality scalability (CGS)
9/4/2018
Dan Grois, Ph.D. Candidate

28. Coarse-grain quality scalability (Cont.)
Smaller quantization step sizes for higher enhancement residual layers
Designed for only several selected bit-rate points
Supported bit-rate points = Number of layers
9/4/2018
Dan Grois, Ph.D. Candidate

29. Quality Scalability Medium-grain quality scalability (MGS)
More enhancement layers are supported
Refinement quality layers of residual
Key pictures
Drift control
Switch can occur at any access units
CGS + key pictures + refinement quality layers
9/4/2018
Dan Grois, Ph.D. Candidate

30. DRIFT IN PAST SNR SCALABLE CODING

31. SNR SCALABILITY RESULTS: PAST CODECS
9/4/2018
Dan Grois, Ph.D. Candidate

32. SNR SCALABILITY RESULTS: H.264/SVC
9/4/2018
Dan Grois, Ph.D. Candidate

33. SNR SCALABILITY RESULTS: H.264/SVC
9/4/2018
Dan Grois, Ph.D. Candidate

34. CODER CONTROL
9/4/2018
Dan Grois, Ph.D. Candidate

35. ENCODER OPTIMIZATION: JSVM ENCODER CONTROL
9/4/2018
Dan Grois, Ph.D. Candidate

36. ENCODER OPTIMIZATION: JOINT CONTROL
9/4/2018
Dan Grois, Ph.D. Candidate

37. RESULTS FOR SNR SCALABILITY
9/4/2018
Dan Grois, Ph.D. Candidate

38. Outline Introduction History of SVC Structure of SVC
Temporal Scalability
Spatial Scalability
Quality Scalability
Combined Scalability
SVC encoder structure
Dependence and Quality refinement layers
Bit-stream format
Bit-stream switching
Profiles of SVC
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

39. Combined Scalability SVC encoder structure Dependency layer
The same motion/prediction information
Dependency layer
Temporal Decomposition
The same motion/prediction information
9/4/2018
Dan Grois, Ph.D. Candidate

40. Combined Scalability Dependency and Quality refinement layers Q = 2
Scalable bit-stream
D = 1
Q = 1
Q = 0
Q = 2
D = 0
Q = 1
Q = 0
9/4/2018
Dan Grois, Ph.D. Candidate

41. NAL unit header extension
Combined Scalability
Bit-stream format
NAL unit header
NAL unit header extension
NAL unit payload 2 6 3 3 2 1 1 1 1 1 3 P T D Q
P (priority_id): indicates the importance of a NAL unit
T (temporal_id): indicates temporal level
D (dependency_id): indicates spatial/CGS layer
Q (quality_id): indicates MGS/FGS layer
9/4/2018
Dan Grois, Ph.D. Candidate

42. Outline Introduction History of SVC Structure of SVC
Temporal Scalability
Spatial Scalability
Quality Scalability
Combined Scalability
Profiles of SVC
Scalable Baseline
Scalable High
Scalable High Intra
Conclusions
9/4/2018
Dan Grois, Ph.D. Candidate

43. Profiles of h.264/aVC
9/4/2018
Dan Grois, Ph.D. Candidate

44. Profiles of h.264/aVC (CONT.)
9/4/2018
Dan Grois, Ph.D. Candidate

45. Profiles of h.264/aVC (CONT.)
9/4/2018
Dan Grois, Ph.D. Candidate

46. SVC PROFILES
9/4/2018
Dan Grois, Ph.D. Candidate

47. Conclusions
Dan Grois, Ph.D. Candidate
9/4/2018

48. Conclusions (Cont.)
9/4/2018
Dan Grois, Ph.D. Candidate

49. References
9/4/2018
Dan Grois, Ph.D. Candidate