xStreamer Modular Multimedia Streaming Nicolas Staelens INTEC Broadband Communication Networks (IBCN) Department of Information Technology (INTEC) Ghent University - IBBT

xStreamer  In-house developed modular multimedia streamer  Alexis Rombaut  Written in C++  Uses libraries:  libavformat/libavcodec (parsing/encoding/decoding)  live555 (RTSP)  jrtplib (RTP)  Released under General Public License (GPL)  Freely available at 2

3 Modular Multimedia Streaming Inspired by Click Modular Router & DirectShow Offers different components  Performs basic functions  Readers, packetizers, multiplexers, schedulers, transmitters, receivers, writers, classifiers, analyzers Streamer is directed graph of components

4 Modular Multimedia Streaming Supports audio and video Using RTP packetization:  MPEG-1/2/4 Video & Audio Using MPEG-2 Transport Streams:  MPEG-1/2/4 Video & Audio  H.264 AVC/SVC

5 Modular Multimedia Streaming Multitude of supported protocols RTP/UDP RTSP/RTP/UDP UDP TCP

What can xStreamer do? Advanced streaming server  Own MPEG-2 TS multiplexer  SVC streaming  Differentiated streaming using classifiers Proxy/client  Proxy: convert differentiated stream into a single stream  Client: save captured stream to file Video tool  No ‘real’ streaming involved  Simulate packet loss  Collect tracefiles during streaming 6

Create xStreamer configuration Configuration saved in XML-based file  Describes directed graph of components and connections between components Graphical User Interface  Visualize directed graph  Drag components and draw connections  Configure components 7

Example: Differentiated SVC streaming 8 Read raw H.264 video stream Packetize frames into packets as defined in RFC3984 Avoid bursts by smoothing packets over time Classify NAL units depending on SVC layer Stream different layers over different connections

Example: Proxy/Client 9 ProxyClient

xStreamer as video tool Offline simulator  No ‘real’ streaming  Simulate packet loss using Classifier component  Random, Gilbert-Elliott  Write resulting packet stream back to file Tracefile generation  Packetizer : video trace  Transmitter : sender trace  Receiver : receiver trace  Classifier : sender & receiver trace 10

xStreamer – HDTV PVS creation HDTV PVS Creation  H.264/AVC  Packet loss Toolchain 1.Encoding 2.Streaming 3.Decoding & syncing 11

HDTV PVS Creation – xStreamer config 12 Packet loss simulator using Gilbert-Elliott model

HDTV PVS Creation – Tracefiles 13

HDTV PVS Creation – Tracefiles 14 Video tracefile generation 0H H H I P B P B P B P P P B P B P …

HDTV PVS Creation – Tracefiles 15 Sender tracefile 0.00id 1udp id 2udp id 3udp id 4udp id 5udp id 6udp id 7udp id 8udp id 9udp id 10udp id 11udp id 12udp id 13udp id 14udp id 15udp id 16udp id 17udp 1430 …

HDTV PVS Creation – Tracefiles 16 Receiver tracefile 0.00id 0udp id 1udp id 2udp id 3udp id 4udp id 5udp id 6udp id 7udp id 8udp id 9udp id 10udp id 11udp id 12udp id 15udp id 16udp id 17udp id 18udp 1430 … 2 packets lost: id 13 & 14

Combining tracefiles 17 0HCORRECT video tracesender tracereceiver trace 0HCORRECT 0IDAMAGED …

PVS Creation: decoding & syncing Decode streamed raw H.264/AVC file  avc-decoder based on libavcodec (ffmpeg)  Only decodes correctly received frames Align decoded and original sequence  Decoded sequence misses pictures  Lost & damaged  Tracefile analysis  Locate missing pictures  Conceal missing pictures  Copy previous frame, gray, black, green, … 18

Conclusion Modular architecture  Logical structure  Easily extendable Advanced streaming server  SVC Support  Differentiated streaming Video tool  Network simulation  Tracefile generation 19

Questions ?