1. Future Video Coding
Prof. Dr.-Ing. Jörn Ostermann

2. Technology Improvements
Video Coding
Technology Improvements
Infrastructure Device
Constant need for improved compression
More devices, more pixels, more frames, more user-generated content
50% rate reduction in 10 years
Constant need for additional functionality
High dynamic range, wide color gamut, head mounted displays, augmented reality, virtual reality
Prof. Dr-Ing- Jörn Ostermann

3. New usages: Panorama mono
Video Coding
New usages: Panorama mono
Prof. Dr-Ing- Jörn Ostermann

4. New usages: Panorama stereo
Video Coding
New usages: Panorama stereo
Prof. Dr-Ing- Jörn Ostermann

5. Use Cases for Future Video Codec
Existing Markets
Terrestrial, and Satellite broadcasting, cable services, managed IPTV services, surveillance
New Markets
Over-the-top (OTT) services, User generated content, video conferencing, gaming
Emerging Markets
Augmented and Virtual reality, multicast streaming/broadcast; media centric internet of things
Prof. Dr-Ing- Jörn Ostermann

6. Requirements on Future Video Codec
Source material
8k, 0Hz, 120 Hz, WCG, HDR, 10bit
Rectangular, panorama, projected / viewports
Natural, synthetic
Monoscopic, multiview
Network interface
Streaming, error resilience, scalability, low delay
Emerging Markets
Augmented and Virtual reality, multicast streaming/broadcast; media centric internet of things
Prof. Dr-Ing- Jörn Ostermann

7. Relevant Bodies
ISO/IEC JTC1/SC29/WG11 Motion Picture Experts Group (MPEG)
ITU-T Study Group 16 with Q6/16 (VCEG)
Common work between MPEG and VCEG
MPEG-4 AVC (ITU-T H.264, ISO/IEC )
HEVC (ITU-T H.265, ISO/IEC )
MV-HEVC, 3D-HEVC as extensions of HEVC
Joint Video Experts Team (JVET) Future Video Coding
Prof. Dr-Ing- Jörn Ostermann

8. Experimental open source software “Joint Exploration Model“ (JEM)
JEM – where we are
Experimental open source software “Joint Exploration Model“ (JEM)
Intended to investigate potential for better compression beyond HEVC
Open source code available for download
Documentation openly and freely available for download:
HEVC software plus additional or improved compression tools
General architecture similar to HEVC
Hybrid closed-loop, high level syntax, reference picture management, etc.
Improved/new compression tools
Prof. Dr-Ing- Jörn Ostermann

9. Overall: Hybrid Coding / Closed Loop
Decoded
Picture
Buffer
Filter Control
Data
Header Formatting & CABAC
Scaling & Inverse Transform
Motion
Compensation
General
Control
Quantized Transform
Coefficients
Intra Prediction
Intra/Inter Selection
Estimation
Transform, Scaling & Quantization -
Input
Video
Signal
Split into CTUs
Intra-Picture
Prediction
Deblocking &
SAO Filters
Output
Coded Bitstream
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

10. JEM Results compared to HEVC (HM)
Measured average rate reduction for same PSNR (Bjontegaard delta method)
Average encoder and decoder run time
Random Access
Over HM (sequential, gcc6.2) Y U V
EncT
DecT
Class A1
-27.2%
-33.1%
-35.2%
10.7x
5.6x
Class A2
-36.3%
-45.2%
-38.2%
7.8x 6x
Class B
-27.0%
-36.9%
-31.7%
8.8x
6.3x
Class C
-26.3%
-31.2%
-33.6%
11.3x
Class D
-26.0%
-30.0%
-31.5%
11.5x
11.7x
Overall
-28.5%
-35.4%
-33.9%
9.9x
7.2x
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr-Ing. Jens Ohm

11. JEM – HEVC – AV1
Prof. Dr-Ing- Jörn Ostermann

12. Call for Evidence results
Joint Call for Evidence on Video Compression with Capability beyond HEVC (April 2017)
Test cases with eight 4K/SDR, five HD/SDR, five HD/HDR, and five 360-degree video sequences
4 rate points each
Results of Joint Call for Evidence on Video Compression with Capability beyond HEVC (July 2017)
Evaluates JEM anchors and other proposals against HEVC Main 10
Expert viewing tests with double-stimulus approach, allows grading on an MOS (mean opinion score) scale
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

13. Call for Evidence results
Test cases SDR / UHD 4K, MOS over rate [kbps
"Daylight Road"
"Campfire Party"
Source: JVET-G1004
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

14. Call for Evidence results
Test cases HDR / HD, MOS over rate [kbps]
"Hurdles"
"Cosmos"
Source: JVET-G1004
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

15. Call for Evidence results
Test cases 360° Video, dynamic viewports, MOS over rate [kbps]
Source: JVET-G1004
"Kite Flite"
"Harbour"
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

16. Next Steps
Joint Call for Proposals on Video Compression with Capability beyond HEVC
Test cases with 4K/SDR, HD/SDR, HD/HDR, 360-degree video
4 rate points each
HM anchors and JEM anchors
Formal subjective assessment in labs
Timeline:
CfP October 2017
Submissions by February 2018, subjective testing March/April
Evaluation of proposals: April 2018
Next generation standard by end of 2020
Prof. Dr-Ing- Jörn Ostermann
Source: Prof. Dr.-Ing. Jens Ohm

17. Future Video Coding will focus on:
Conclusions
Future Video Coding will focus on:
Rectangular 10 bit WCG HDR UHD video
Future Video Coding will enable:
Support for panoramic video
Edge-based rendering for interactivity, games
Future Video coding will deliver in 2020:
Coding efficiency for UHD at 50% of the rate of HEVC for the same quality
UHD at typically less than 4MBit/s
MPEG continues to deliver compression technology beyond the state of the art
Prof. Dr-Ing- Jörn Ostermann