1. Over-the-Top Content Delivery State of the Art and Challenges
Over-the-Top Content Delivery State of the Art and Challenges
TNC17, Linz, Austria, May 30, 2017
Priv.-Doz. Dr. Christian Timmerer [Ack: Ali C. Begen, Networked Media & Ozyegin University | Thomas Stockhammer, Qualcomm Inc. | Iraj Sodagar, Microsoft]
Alpen-Adria-Universität Klagenfurt (AAU)  Multimedia Communication (MMC) 
Chief Innovation Officer (CIO) at bitmovin GmbH 

2. Dr. Timmerer [AAU/Bitmovin]
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

3. Importance of Multimedia Delivery
Multimedia is predominant on the Internet
Real-time entertainment: Streaming video and audio; >70% of Internet traffic at peak periods
Popular services: YouTube (17.53%), Netflix (35.15%), Amazon Video (4.26%), Hulu (2.68%); all delivered over-the-top (OTT); huge potential for mobile access!
Global Internet Phenomena Report: 2016
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

4. Open Digital Media Value Chain
Dynamic
Ads
Internet Transport
Encapsulation
DRM
Media Protocols
CDNs
Encoding
Clients
Create Content
Aggregate
Monetize
Distribute Content
Consume Content
Any Device
Any Content
Any Network
Any Storefront
Happy User
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

5. Example Platform/Infrastructure
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

6. Common Annoyances in Streaming
Wrong format
Wrong protocol
Plugin requirements
DRM issues
Long start-up delay
Poor quality
Frequent stalls
Quality oscillations
No seeking features
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

7. Over-The-Top – Adaptive Media Streaming
Adaptation logic is within the client, not normatively specified by the standard, subject to research and development
In a nutshell…
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

8. Multi-Bitrate Encoding and Representation Switching
Contents on the Web Server
Movie A – 200 Kbps
Movie A – 400 Kbps
Movie A – 1.2 Mbps
Movie A – 2.2 Mbps
. . .
Request Manifest for Movie A
Start quickly
Manifest
Request Movie A (200 Kbps) for t=0
Keep requesting
. . .
. . .
Segments
Request Movie A (400 Kbps) for t=2
Improve quality
Movie K – 200 Kbps
Movie K – 500 Kbps
Movie K – 1.1 Mbps
Movie K – 1.8 Mbps
. . .
Request Movie A (800 Kbps) for t=4
Loss/congestion detection
Request Movie A (400 Kbps) for t=16
Revamp quality
Request Movie A (800 Kbps) for t=28
Time (s)
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

9. Adaptive Streaming over HTTP
Server
HTTP GETs …
Media
Player
Client
Buffer
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

10. Dr. Timmerer [AAU/Bitmovin]
Formats and Standards
Adobe
HTTP Dynamic Streaming (HDS)
Switched to DASH
Apple
HTTP Live Streaming (HLS)
Required for iOS
Microsoft
Smooth Streaming
Switched to DASH, almost..
MPEG Dynamic Adaptive Streaming over HTTP (DASH)
Supported by Netflix, YouTube, Bitmovin, etc.
MPEG Common Media Application Format (MPEG-A Part 19)
The new kid on the block – support for “fragmented mp4 in HLS”
DASH/HLS convergence at segment level – some open issues with encryption format
Source:
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

11. Scope of DASH: what is specified?
Media Presentation on HTTP Server
Media Presentation Description
DASH-enabled Client . .
DASH Control Engine
Segment
Segment
Segments located by HTTP-URLs … …
MPD Parser
Media Engine . . …
On-time HTTP requests to segments
Segment
Segment … …
HTTP Client
HTTP/1.1
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

12. Scope of DASH: what is specified?
Media Presentation on HTTP Server
Media Presentation Description
DASH-enabled Client . .
DASH Control Engine
Segment
Segment
Segments located by HTTP-URLs … …
MPD Parser
Media Engine . . …
On-time HTTP requests to segments
Segment
Segment … …
HTTP Client
HTTP/1.1
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

13. DASH Data Model MPD Segment Access Adaptation Set 1 Period id = 2
BaseURL=
Period id = 2
start = 100 s
Period id = 1
start = 0 s
Initialization Segment
Representation 3
Rate = 2 Mbps
Resolution = 720p
Adaptation Set 0 subtitle turkish
Representation 1
Rate = 500 Kbps
Period id = 2
start = 100 s
Media Segment 1
start = 0 s
Representation 2
Rate = 1 Mbps
Adaptation Set 1
video
Segment Info
Duration = 10 s
Template:
3/$Number$.mp4
Period id = 3
start = 300 s
Media Segment 2
start = 10 s
Representation 3
Rate = 2 Mbps
Adaptation Set 2
audio english
Representation 4
Rate = 3 Mbps
Period id = 4
start = 850 s
Adaptation Set 3
audio german
Splicing of arbitrary content like ads
Selection of components/tracks
Selection of representations
Well-defined
media format
Chunks with addresses and timing
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

14. MPD Schema Overview Profile identifier “static” | “dynamic”
Set of switchable Representations
Encoded version of a media component
Multiple content locations
Time sequence of Media Presentation
May 30, 2017
Client (QoE) metrics
Dr. Timmerer [AAU/Bitmovin]

15. Audio/Video parameters Container, codec, information
MPD Schema Overview
Descriptors
Audio/Video parameters
Container, codec, information
URL construction
Playlist-based
Template-based
Bandwidth
May 30, 2017
Quality information
Dr. Timmerer [AAU/Bitmovin]

16. Dr. Timmerer [AAU/Bitmovin]
seg1.m4s
Segment Index in MPD only
<MPD>
...
<URL sourceURL="seg1.m4s"/>
<URL sourceURL="seg2.m4s"/>
</MPD>
seg2.m4s
...
<MPD>
...
<URL sourceURL="seg.m4s" range="0-499"/>
<URL sourceURL="seg.m4s" range=" "/>
</MPD>
seg.m4s
Segment Index in MPD + Segment
<MPD>
...
<Index sourceURL="sidx.mp4"/>
<URL sourceURL="seg.m4s"/>
</MPD>
seg.m4s
sidx.m4s
Segment Index in Segment only
<MPD>
...
<BaseURL>seg.m4s</BaseURL>
</MPD>
seg.m4s
sidx
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

17. Switch Point Alignment
Segment alignment
Permits non-overlapping decoding and presentation of segments from different representations
Stream Access Points (SAPs)
Presentation time and position in segments at which random access and switching can occur
Bitstream Switching
Concatenation of segments from different representations results in conforming bitstream
Alignment and SAPs can also apply for subsegments
Preferable switching points are segment/subsegment boundaries for which
Alignment holds across representations
The switch-to representation starts with a SAP
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

18. Dr. Timmerer [AAU/Bitmovin]
type=static typically, for on demand content
Base URL of the segments
Subtitles
Audio adaptation set with different representations (bw)
Video adaptation set with different representations (bw)
Different codecs (profiles)
Segment URL constructed with template and base URL
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

19. https://bitmovin.com/demo/
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

20. ISO/IEC Timeline
CfP Issued April 2010
18 Responses and Working Draft (WD)
July 2010
Committee Draft (CD)
Oct. 2010
Draft International Standard (DIS)
Jan. 2011
Final Draft International Standard
Nov. 2011
Published as International Standard
April 2012
Fastest time ever that a standard was developed in MPEG to address the demand of the market
Other Relevant Specifications
: ISO Base Media File Format
: Carriage of NAL unit structured video in the ISO Base Media File Format
: Common encryption format for ISO base media file format
: Coding-Independent Code Points
: Carriage of Timed Metadata Metrics of Media in ISO Base Media File Format
See also here
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

21. MPEG-DASH Features (Jan‘17)
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

22. Ongoing Work in MPEG-DASH
Currently Running Core Experiments
High Quality VR delivery with DASH (DASH-VR)
New work item proposal
Coded Representation of Immersive Media
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

23. MPEG CMAF: Threat or Opportunity?
+ CMAF Media Object Model compatible with DASH Data Model + Segment formats based on ISOBMFF - Different manifest formats (MPD vs. m3u8) - CENC: AES-128 CBC (HLS) vs. AES-128 CTR (all others) mode
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

24. Common Problems in DASH
Encoding | Packaging | Encryption
Guidelines: 3-20 different representations (mobile to UHD)
Segment length: 4s shows good tradeoff (2s vs. 9s)
Offline vs. on-the-fly
Delivery, distribution, CDN
MMSys’16 keynote by Neill Kipp:
Consumption and Quality of Experience (QoE)
Adaptation strategies: buffer- vs. throughput-based
Multi-client competition: on-off behavior
Quality-aware streaming: highest possible bitrate vs. highest quality
Inter-Destination Media Synchronization (IDMS): new applications
Virtual Reality / 360-degree video: tiled streaming
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

25. Quality of Experience for DASH
Objective
Initial or startup delay (low)
Buffer underrun / stalls (zero)
Quality switches (low)
Media throughput (high)
[Other media-related configuration: encoding, representations, segment length, …]
Subjective
Mean Opinion Score (MOS) – various scales
Various methodologies (e.g., DSCQS, DSIS, ACR, PC, …)
May 30, 2017
M. Seufert, et al., "A Survey on Quality of Experience of HTTP Adaptive Streaming,"IEEE Communications Surveys & Tutorials, vol.17, no.1, doi: /COMST
Dr. Timmerer [AAU/Bitmovin]

26. Dr. Timmerer [AAU/Bitmovin]
DASH QoE in Real-World
Stalls and low quality are bad for QoE but not startup delay
B. Rainer, C. Timmerer, “Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”, Proceedings of International Workshop on VideoNext: Design, Quality and Deployment of Adaptive Video Streaming, Sydney, Australia, Dec
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

27. Dr. Timmerer [AAU/Bitmovin]
10 different adaptation algorithms
Christian Timmerer, Matteo Maiero, Benjamin Rainer, Which Adaptation Logic? An Objective and Subjective Performance Evaluation of HTTP-based Adaptive Media Streaming Systems, In arXiv.org [cs.MM], N.N., vol. abs/ , N.N., pp. 11, 2016. 
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

28. Dr. Timmerer [AAU/Bitmovin]
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

29. Dr. Timmerer [AAU/Bitmovin]
Conclusions
MPEG-DASH defines formats only
Media Presentation Description (MPD)
Segment format: isobmff, m2ts
MPEG-DASH is not
System, protocol, presentation, codec, interactivity, DRM, client specification
Other standards required for a complete ecosystem
DASH-IF, WAVE, HMTL5, MSE, EME, 3GPP, DVB, etc.
Do we need MPEG-DASH? (for adaptive media streaming)
Not necessarily: e.g., WebM + VPx + manifest & control end-to-end
Required to address heterogeneous environments to solve interop. problem
Role of standards sometimes overrated but often underestimated
CMAF and VR addressing new challenges for adaptive media streaming
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]

30. Dr. Timmerer [AAU/Bitmovin]
Deployment Thoughts
Proprietary ecosystems will disappear (Silverlight, Flash)
No more plugins – HTML5!
MSE/EME available on all major browser platforms
Support for both DASH/HLS (+CMAF) and CENC
Rich feature set: codecs, ads, DRM, multi-language/-audio, subtitles, VR/360, UHD, HFR, HDR, live, on-demand, analytics, …
Common implementation issues: start-up, buffering, high-quality, seamless switching, platform support, cost-effective, …
Solutions available for adaptive streaming, advertising, VR/360, live streaming, and DRM
Details available at
May 30, 2017
Dr. Timmerer [AAU/Bitmovin]