1. H.264/AVC

2. Evolution of Video coding Standards
ITU-T
Standard
Joint
ITU-T/MPEG
Standards
MPEG
1988
1990
1992
1994
1996
1998
2000
2002
2004
H.261
(Version 1)
(Version 2)
H.263
H.263+
H.263++
H.262/MPEG-2
H.264/MPEG-4 AVC
MPEG-1
MPEG-4

3. Applications Entertainment Video (1-8+ Mbps, higher latency)
Broadcast / Satellite / Cable / DVD / VoD/ FS-VDSL / …
DVB/ATSC/SCTE, DVD Forum, DSL Forum
Conversational Services (usu. <1Mbps, low latency)
H.320 Conversational
3GPP Conversational H.324/M
H.323 Conversational Internet/best effort IP/RTP
3GPP Conversational IP/RTP/SIP
Streaming Services (usu. lower bit rate, higher latency)
3GPP Streaming IP/RTP/RTSP
Streaming IP/RTP/RTSP (without TCP fallback)
Other Services
3GPP Multimedia Messaging Services

4. Structure of H.264/AVC video encoder
Control
Data
Video Coding Layer
Coded Macroblock
Data Partitioning
Coded Slice/Partition
Network Abstraction Layer
H.320
MP4FF
H.323/IP
MPEG-2
etc.

5. High-Level VCL Summary
Some new key aspects are:
Enhanced motion compensation
Small blocks for transform coding
Improved de-blocking filter
Enhanced entropy coding
Substantial bit-rate savings relative to other standards for the same quality

6. Macroblocks and Slices
Macroblocks: 16x16 luma + 8x8 for chroma samples
Slice: a set of MBs that can be decoded without use of other slices
I slice: intra-prediction (I-MBs)
P slice: possibly one inter-prediction signal (I- and P-MBs)
B slice: up to two inter-prediction signals (I- and B-MBs)
SP slice: P slice to facilitate switching between coded streams
SI slice: I slice to to facilitate switching between coded streams
The only reason that we may require adjacent slices is for deblocking filter.
The main feature of SP-frames is that identical SP-frames can be reconstructed even when different reference frames are used for their prediction.
SI-frames are used in conjunction with SP-frames.
This property make them useful for applications such as random access, adapt to network condition, and error recovery/ resilience
We next talk about ordering of MBs.

7. Enhanced MC (Inter Prediction)
Every macroblock can be split in one of 7 ways for improved motion estimation

8. Enhanced MC (Inter Prediction)
Accuracy of motion compensation = 1/4 pixel
Up to 5 reference frames for SDTV L3
Reference B pictures
Weighted predictions
Motion Estimation을 수행하는 신호에 적응적으로 가중치를 주어 계산한 후 예측신호를 생성

9. 4 modes for 16x16 intra prediction
Intra Spatial Prediction
using surrounding available samples
Intra 4x4 Prediction Modes
4 modes for 16x16 intra prediction
Intra 16x16 Prediction Modes

10. Transform MBs are divided up into 4x4 or 8x8 blocks
Transformation matrix T4x4 or T8x8 is applied to residual data of every block
For the DC Coefficients,
Hadamard transform H4x4 for the 4×4 array of luma DC coefficients in Intra MBs predicted in 16×16 mode.
Hadamard transform H2x2 for the 2 × 2 array of chroma DC coefficients (in any macroblock).

11. Quantization Logarithmic step size control
A total of 52 values of Qstep are supported by the standard, indexed by a Quantization Parameter, QP.
Qstep doubles in size for every increment of 6 in QP.
Can change to any step size at macroblock level
Quantization reconstruction is one multiply, one add, one shift
The wide range of quantiser step sizes makes it possible for an encoder to control the tradeoff accurately and flexibly between bit rate and quality. The values of QP can be different for luma and chroma.

12. Adaptive Deblocking Filter
Blocking artifacts
4*4 transforms and block-based motion compensation
Block edges are typically reconstructed with less accuracy than interior pixels.
Improve subjective and objective quality of the decoded picture
Result in bit rate savings of around 6~9%
Original Frame
Reconstructed,
QP=36 (no filter)
Reconstructed,
QP=36 (with filter)

13. Entropy Coding CAVLC (Context Adaptive Variable Length Coding)
Context : already coded information of the neighboring blocks and the coding status of the current block
Optimized VLC tables are provided for each context to code the coefficients in different statistical conditions
CABAC (Context Adaptive Binary Arithmetic Codes)
Use a binary arithmetic coding engine
Compression improvement is consequence of
Adaptive probability estimation
Improved context modeling scheme
Exploiting symbol correlations by using contexts
Average bit-rate saving over CAVLC 5~15%

14. Error Resilience Tools
Flexible Macroblock Order (FMO)
Arbitrary Slide Order (ASO)
Redundant Slice (RS)
Data Partitioning (DP)

15. MB to Slice Group Mappings
Flexible Macroblock Ordering (FMO)
a subset of the macroblocks may contain one or more slices
Application example: Error resilience

16. Arbitrary Slice Ordering (ASO)
The decoding order of the slices could be arbitrary
Application example: reduce end-end transmission delay in RT app

17. H.264/AVC의 해상도의 크기

18. Profiles

19. Basic Coding Structure

20. Comparison to Previous Standards
The set of test sequences for this comparison consists of
four QCIF (10 and 15 Hz) sequence
176 x144 resolution at 10–256 kb/s
four CIF (15 Hz and 30 Hz) sequence
CIF 352 x 288 resolution at 128–1024 kb/s

21. Comparision of Standards
Feature/Standard
MPEG-1
MPEG-2
MPEG-4 part 2 (visual)
H.264/MPEG-4 part 10
Macroblock size
16x16
16x16 (frame mode)
16x8 (field mode)
Block Size
8x8
16x16, 16x8, 8x8
16x16, 8x16, 16x8, 8x8, 4x8, 8x4, 4x4
Transform
8x8 DCT
8x8 DCT/Wavelet
4x4, 8x8 Int DCT
4x4, 2x2 Hadamard
Quantization
Scalar quantization with step size of constant increment
Vector quantization
Scalar quantization with step size of increase at the rate of 12.5%
Entropy coding
VLC
VLC, CAVLC, CABAC
Motion Estimation & Compensation
Yes
Yes, more flexible
Up to 16 MVs per MB
Playback & Random Access

22. Comparision of Standards (cont’d..)
Feature/Standard
MPEG-1
MPEG-2
MPEG-4 part 2 (visual)
H.264/MPEG-4 part 10
Pel accuracy
Integer, ½-pel
Integer, ½-pel,
¼-pel
Profiles No 5 8 3
Reference picture
one
multiple
Bidirectional prediction mode
forward/backward
forward/forward
backward/backward
Picture Types
I, P, B, D
I, P, B
I, P, B, SP, SI
Error robustness
Synchronization & concealment
Data partitioning, FEC for important packet transmission
Synchronization, Data partitioning, Header extension, Reversible VLCs
Data partitioning,
Parameter setting,
Flexible macroblock ordering, Redundant slice, Switched slice
Transmission rate
Up to 1.5Mbps
2-15Mbps
64kbps - 2Mbps
64kbps -150Mbps
Compatibility with previous standards
n/a
Yes
Encoder complexity
Low
Medium
High

23. References Related group Test software Test sequences Papers
MPEG website
JVT website: ftp://ftp.imtc-files.org/jvt-experts
Test software
H.264/AVC JM Software:
Test sequences
ftp.tnt.uni-hannover.de/pub/jvt/sequences/
Papers
Wiegand, T.; Sullivan, G.J.; Bjontegaard, G.; Luthra, A., “Overview of the H.264/AVC Video Coding Standard,” in IEEE transactions on circuits and systems for video technology, Vol. 12, No.7, July
Sullivan, G.J.; Wiegand, T., "Video Compression - From Concepts to the H.264/AVC Standard," Proceedings of the IEEE , vol.93, no.1, pp.18-31, Jan. 2005
Jorn Ostermann et al., “Video coding with H.264/AVC: Tools, Performance, and Complexity,” in IEEE Circuit and systems magazine, first quarter
M. Mahdi Ghandi and Mohammad Ghanbari, “The H.264/AVC Video Coding Standard for the Next Generation Multimedia Communication,” in IAEEE Journal