1. Video Coding --- State of the Art and Beyond
Dr. Mona Mathur
Advanced Systems Technology,
STMicroelectronics, GNoida, India
August 8, 2011

2. Agenda Overview of concepts in Video Coding
State of the art Compression standards
Evolving Video Coding standards
The Road ahead ---- Challenges

3. Digital Video characteristics
Frame :
is a 2D rectilinear array of pixels
Frame rate number of frames displayed per second
Frame dimensions width and height of the frame in pixels
Pixel depth Number of bits per pixel
Each pixel has two values associated with it
Luminance – represents the intensity (Y)
Chrominance – represents the colour of the pixel (Cr, Cb)
OBESE

4. Very high secondary storage and transmission requirements.
Why compression ????
Images require considerably more storage space than text
Audio and Video demand even more!!
Text (VGA 2 bytes/char) ~ 640 * 480 * 2 / (8 * 8)
~ 9.4 KB
Uncompressed stereo audio ~ 2 * * 2
(44.1 KHz, 16 bits per sample) ~ 172 KB/s
PAL (720 pixels, 576 lines, 25 frames) ~ 249 Mbits/s !!
A 2 hour movie ~ 1669GB
more than 2000 CDs
Very high secondary storage and transmission requirements.

5. History of Video Standards
ITU --- International Telecommunications Union --- is focused on telecom applications
---- has created H.26x standards for low bit rate telephony, x---1to4
ISO --- International Standards Organization is focused on consumer applications in entertainment & computers
----- defined MPEG standards for videos, MPEG-1, MPEG-2 and MPEG-4
H.263 baseline profile has commonalities with MPEG-4 Part2 Simple profile

6. Use and Spread of Standards
Critical for widespread adoption of codec technology
The three players
Customers
Economies of scale --- so products are affordable
Industry
Interoperability between vendors
Content providers
Long life and broad demand of the content
Ensures widespread adoption
Proprietary standards
AVS (China), WMV9 (Microsoft), On2, VP8, Nancy, RV10 (Real Networks Real Video)
Standards are often Targeted for specific applications
Proprietary standards available basically for media streaming applications
WMV9 – standardized in the SMPTE(Society of Motion Pictures and Television Engineers) as VC-1
On2 --- formerly “Duck Technologies” developed a codec called “True Motion”--- acquired by Google in Feb

7. Compression rates for diff technologies
Intra-frame
Motion Compensation
Integer-pel
Half-pel
Variable Block size
Quarter-pel
Multi-frame

8. Evolution of MPEG4

9. Compression ??? : How Irrelevancy Reduction
Statistical Redundancy in the signal
Spatial: Correlation between adjacent pixels
Temporal: Correlation between pixels in adjacent pictures
Irrelevancy Reduction
Omits parts of the signal not perceived by the Human Visual System (HVS)
No Change in pixels

10. Coding Tools in state of the art codecs
Spatial redundancy
Intra prediction
Temporal redundancy
Inter prediction
Transform -- DCT
Entropy coding
VLC, CABAC
Deblocking Filter

11. Frame – from Macroblock view

12. Spatial Coding

13. Temporal Coding

14. H.264 Encoder

15. Is Need for higher Compression Relevant Today?
Compression Today??
Is Need for higher Compression Relevant Today?
YES !!!
Why ????
When a scene is captured by multiple cameras, e.g. for the purpose of user-controlled navigation through a video scene, or for 3D display with view adaptation, the video data rate may in the worst case linearly increase by the number of cameras that are used for simultaneous capturing. It will therefore be necessary to further compress the overall rate by proper exploitation of coherences
between the different available views (inter-view redundancy) instead of independently compressing the
different views.
Ref -- -Jens papers on AdvancesInVideoCoding

16. Evolving Products
Smart phones
Tablets
Ipads
Net enabled STBs ….

17. Emerging Video Content &Type
Video as a medium is changing.
Spatial resolution
QCIF x pixels
HD x ,073,600 pixels
UHD x3, ,582,912 pixels
Temporal resolution ( fps)
Increasing dynamic range of each component
From 8 bit per component to 16 bit per component
New visual modalities
stereoscopic, Multi-view,
3D, holography,
the myriad of consumer
devices playing stored

18. Demands of Consumers… Imagine…
…being able to see high quality, high definition video, on demand.
…renting a DVD with choices of standard or high definition video, with multiple camera views
Applications like
Facebook, twitter, Google+
Mobile Video
Services like : football highlights, personal video conferencing, or downloads of movie trailers
such services will increase
What about spectrum availability

19. Video Coding today …. New extensions within existing framework
SVC, MVC, Free View point TV
Next evolution of the video coding standards
KTA, H.265
New Technologies
Parametric Video Coding
Texture Analysis and Synthesis

20. Video Coding today …. New extensions within existing framework
SVC, MVC, Free View point TV
Next evolution of the video coding standards
KTA, H.265
New Technologies
Parametric Video Coding
Texture Analysis and Synthesis

21. HEVC ---High Efficiency Video Coding
Proposed Video compression standard
Successor to H.264/MPEG-4 AVC
A joint team by ISO- MPEG and ITU-T --VCEG has been established
JCT-VC --- Joint Collaborative Team on Video Coding in 2010
Nick named --- H.265
Goals:
Increase compression by 50% over H.264 at comparable image/video quality
At the expense of increased computational complexity
Targeted platforms:
Next generation HDTV displays and content capture systems
Display resolutions from QVGA (320x240) upto 1080p and Ultra HDTV (7680x4320)

22. Testing Results --- BDrate with complexity
Source --- JCTVC-D181

23. Benefits of arbitrary unit representation
Codec optimized for content, applications and devices.
Examples are:
low resolution video services
Videos with large homogenous regions
A simple & elegant representation
LCTB size,
max hierarchical depth, &
a series of split flags.
Size-independent syntax representation
Syntactical representation of remaining coding tools is simplified
Simplifies the standard and the parsing process