1. Foreleser: Carsten Griwodz
Multimedia Protocols
Foreleser: Carsten Griwodz
05. May 2004 1

2. Non-QoS Multimedia Networking
RTP – Real-Time Transfer Protocol
05. May 2004 2

3. Real-time Transport Protocol (RTP)
RFC 1889
Designed for requirements of real-time data transport
NOT real-time
NOT a transport protocol
Two Components
Real-Time Transfer Protocol (RTP)
RTP Control Protocol (RTCP)
Provides end-to-end transport functions
Scalable in multicast scenarios
Media independent
Mixer and translator support
RTCP for QoS feedback and session information

4. Real-time Transport Protocol (RTP)
No premise on underlying resources
layered above transport protocol
no reservation / guarantees
Integrated with applications
RTP follows principles of
Application Level Framing and
Integrated Layer Processing
UDP
IPv4/6
Ethernet
AAL5
ATM
ST-2
RTP
RTCP
media
encapsulation
application
TCP

5. RTP RTP services are RTP supports RTP is not designed for Sequencing
Synchronization
Payload identification
QoS feedback and session information
RTP supports
Multicast in a scalable way
Generic real-time media and changing codecs on the fly
Mixers and translators to adapt to bandwidth limitations
Encryption
RTP is not designed for
Reliable delivery
QoS provision or reservation

6. RTP Functions RTP with RTCP provides Functional basis for this
Support for transmission of real-time data
Over multicast or unicast network services
Functional basis for this
Loss detection – sequence numbering
Determination of media encoding
Synchronization – timing
Framing - “guidelines” in payload format definitions
Encryption
Unicast and multicast support
Support for stream “translation” and “mixing” (SSRC; CSRC)

7. Typical IETF RFC bit-exact representation
RTP Packet Format
Typical IETF RFC bit-exact representation
a longword (32 bit)
a byte

8. RTP Packet Format
Padding indicator bit if set, number of padding bytes is in last byte of payload
Version number,
Always 2
Header extension bit
True if header extension is present
7 bit payload type
Allows identification of the
payload’s content type
Marker bit
Meaning depends on payload profile,
e.g. frame boundary
4 bit CSRC count, indicates the number of contributing sources in the header

9. RTP Packet Format 16 bit sequence number 32 bit timestamp 32 bit SSRC
Synchronization source identifier, a random number identifying the sender
Several 32 bit CSRC
Contribution source identifier, the number is indicated by CC
A mixer copies the original sources’ SSRCs here
Header extension
multiples of 32 bit

10. RTP Architecture Concepts
Integrated Layer Processing
Typical layered
Data units sequentially processed by each layer
Integrated layer processing
Adjacent layers tightly coupled
Therefore, RTP is not complete by itself: requires application-layer functionality/information in header
16 bit sequence number
32 bit timestamp
7 bit payload type
Allows identification of the
payload’s content type
Marker bit Meaning depends on payload profile, e.g. frame boundary

11. RTP Packet Format Relatively long header (>40 bytes)
overhead carrying possibly small payload
header compression
other means to reduce bandwidth (e.g. silence suppression)
No length field
Exactly one RTP packet carried in UDP packet
Can use TCP or ATM AAL5
do-it-yourself packaging
Header extensions for payload specific fields possible
Specific codecs
Error recovery mechanisms

12. RTP Profile (RFC 1890) Set of standard encodings and payload types
Audio: e.g. PCM-u, GSM, G.721
Video: e.g. JPEG, H.261
Number of samples or frames in RTP packet
Sample-based audio: no limit on number of samples
Frame-based audio: several frames in RTP packet allowed
Clock rate for timestamp
Packetized audio: default packetization interval 20 ms
Video: normally 90 kHz, other rates possible

13. RTP Profiles Payload type identification
RTP provides services needed for generic A/V transport
Particular codecs with additional requirements
Payload formats defined for each codec: syntax and semantic of RTP payload
Payload types
Static: RTP AV profile document
Dynamic: agreement on per-session basis
Profiles and Payload Formats in RTP Framework
RTP / RTCP
AV Profile
Additional
Profiles
Payload
Formats
Dynamic Payload Types
PT mapping outside RTP
(e.g. SDP)

14. RTP Profiles General Video Profile Associated with a media type.
Provides association between PT field and specific media format
Defines sampling rate of timestamp
May also define or recommend a definition for the “marker” bit
Video Profile
Marker bit recommended to mean last packet associated with a timestamp
Timestamp clock: Hz
Defines PT mapping for a number of different video encoding types

15. RTP Profiles Audio Profile
Marker bit set on the first packet after a silence period where no packets sent
Timestamp equals sampling rate
Recommends 20ms minimum frame time
Recommends that samples from multiple channels be sent together
Defines PT for a number of different audio encoding types

16. RTP Profile for MPEG Video Payload
GOP header
Frame headers

17. RTP Profile for MPEG Video Payload
Fragmentation rules
Video sequence header
if present, starts at the beginning of an RTP packet
GOP sequence header
Either at beginning of RTP packet
Or following video sequence header
Picture header
Following GOP header
No header can span packets
Marker Bit
Set to 1 if packet is end of picture

18. RTP Profile for MPEG Video Payload
MPEG Video Profile
| MBZ | TR |MBZ|S|B|E| P | | BFC | | FFC |
FBV FFV
MPEG-1 Video specific payload header
MBZ
Must be zero TR
Temporal reference
The same number for all packets of one frame
For ordering inside an MPEG GOP S
1 is sequence header is in this packet B
1 if payload starts with new slice E
1 if last byte of payload is end of slice P
3 bits that indicate picture type (I,P, or B)
FBV, BFC, FFC, FFC
Indicate how a P or B frame is related to other I and P frames

19. RTP Quality Adaptation
Application
RTCP
RTP
Decoding
Encoding
UDP/IP
Component interoperations for control of quality
Evaluation of sender and receiver reports
Modification of encoding schemes and parameters
Adaptation of transmission rates
Hook for possible retransmissions (outside RTP)

20. RTP Control Protocol (RTCP)
Companion protocol to RTP (tight integration with RTP)
Monitoring
of QoS
of application performance
Feedback to members of a group about delivery quality, loss, etc.
Sources may adjust data rate
Receivers can determine if QoS problems are local or network-wide
Loose session control
Convey information about participants
Convey information about session relationships
Automatic adjustment to overhead
report frequency based on participant count
Typically, “RTP does ...” means “RTP with RTCP does ...”

21. RTCP Packets Several RTCP packets carried in one compound packet
SR / RR
BYE
SDES
APP
Compound (UDP) Packet
Several RTCP packets carried in one compound packet
RTCP Packet Structure
SR Sender Report (statistics from active senders: bytes sent -> estimate rate)
RR Receiver Report (statistics from receivers)
SDES Source Descriptions (sources as “chunks” with several items like canonical names, , location,...)
BYE explicit leave
APP extensions, application specific

22. RTCP Sender / Receiver Reports
Sender report
Sender Information
Timestamps
Packet Count, Byte Count
List of statistics per source
Receiver report
For each source
Loss statistics
Inter-arrival jitter
Timestamp of last SR
Delay between reception of last SR and sending of RR
Analysis of reports
Cumulative counts for short and long time measurements
NTP timestamp for encoding- and profile independent monitoring
Header
Sender Information
Reception Report
Profile Specific Extensions
...
Header
Reception Report
Profile Specific Extensions
...

23. RTP Mixer Mixer Reconstructs constant spacing generated by sender
Translates audio encoding to a lower-bandwidth
Mixes reconstructed audio streams into a single stream
Resynchronizes incoming audio packets
New synchronization source value (SSRC) stored in packet
Incoming SSRCs are copied into the contributing synchronization source list (CSRC)
Forwards the mixed packet stream
Useful in conference bridges

24. RTP Translator Translation between protocols
ATM
UDP
Protocol
Translator
MPEG
Source
Sink
H.263
Profile
Translation between protocols
e.g., between IP and ST-2
Two types of translators are installed
Translation between encoding of data
e.g. for reduction of bandwidth without adapting sources
No resynchronization in translators
SSRC and CSRC remain unchanged

25. RTP Identifiers SSRC chosen by sender S1 SSRC chosen by mixer M1 S1 S3
Translators keep SSRCs and CSRCs
CSRCs from mixed sources S1 and S2
CSRCs contain previous SSRCs, but not previous CSRCs

26. Protocol Development Changes and extensions to RTP
Scalability to very large multicast groups
Congestion Control
Algorithms to calculate RTCP packet rate
Several profile and payload formats
Efficient packetization of Audio / Video
RTCP-based retransmission
Loss / error recovery

27. Non-QoS Multimedia Networking
Signalling Protocols:
RTSP and SIP
05. May 2004 27

28. Signaling Protocols Applications differ
Media delivery controlled by sender or receiver
Sender and receiver “meet” before media delivery
Signaling should reflect different needs
Media-on-demand
Receiver controlled delivery of content
Explicit session setup
Internet telephony and conferences:
Bi-directional data flow, live sources
(mostly) explicit session setup, mostly persons at both ends
Internet broadcast
Sender announces multicast stream
No explicit session setup

29. Real-Time Streaming Protocol (RTSP)
Internet media-on-demand
Select and playback streaming media from server
Similar to VCR, but
Potentially new functionality
Integration with Web
Security
Varying quality
Need for control protocol
Start, stop, pause, …
RTSP is also usable for
Near video-on-demand (multicast)
Live broadcasts (multicast, restricted control functionality)
...

30. RTSP Approach In line with established Internet protocols
Similar to HTTP 1.1 in style
Uses URLs for addressing: rtsp://video.server.com:8765/videos/themovie.mpg
Range definitions
Proxy usage
Expiration dates for RTSP DESCRIBE responses
Other referenced protocols from Internet (RTP, SDP)
Functional differences from HTTP
Data transfer is separate from RTSP connection
typically via RTP
Server maintains state – setup and teardown messages
Server as well as clients can send requests

31. RTSP Features Rough synchronization
Media description in DESCRIBE response
Timing description in SETUP response
Fine-grained through RTP sender reports
Aggregate and separate control of streams possible
Virtual presentations
Server controls timing for aggregate sessions
RTSP Server may control several data (RTP) servers
Load balancing through redirect at connect time
Use REDIRECT at connect time
Caching
Only RTSP caching so far
Data stream caching is under discussion

32. RTSP Methods OPTIONS DESCRIBE ANNOUNCE SETUP RECORD PLAY PAUSE
C  S
determine capabilities of server/client
C  S
DESCRIBE
get description of media stream
ANNOUNCE
C  S
announce new session description
SETUP
create media session
RECORD
start media recording
PLAY
start media delivery
PAUSE
pause media delivery
REDIRECT
redirection to another server
TEARDOWN
immediate teardown
SET_PARAMETER
change server/client parameter
GET_PARAMETER
read server/client parameter

33. presentation description file
RTSP Integration
HTTP
server
HTTP GET
presentation description file
media server
RTSP SETUP
web browser
RTSP
server
RTSP OK
RTSP PLAY
RTSP OK
RTSP
plug-in
RTSP TEARDOWN
RTSP OK
data
source
RTP VIDEO AV
subsystem
RTP AUDIO

34. Session Initiation Protocol (SIP)
Lightweight generic signaling protocol
Internet telephony and conferencing
Call: association between number of participants
Signaling association as signaling state at endpoints (no network resources)
Several “services” needed
Name translation
User location
Feature negotiation
Call control
Changing features

35. SIP Basics Call user Re-negotiate call parameters
Forwarding (manual and automatic)
Call center
Supports personal mobility (change of terminal)
Through proxies or redirection
Terminate / transfer calls
ASCII (readable) protocol – SIP vs. H.323
Similarities (request/response, proxies ...)
Differences (server state, server may initiate actions ...)
Control, location and media description (via SDP)
Extensible towards
Usage for IP-IP, POTS-IP, inter-gateway interaction with firewalls, billing system, ...
Different modes
Proxy mode
Redirect mode

36. SIP Operation – Proxy Mode
Location Server
2. Where?
5. “Ring”
4. Invite
1. Invite 3.
Site A
6. Ok
7. Ok
8. ACK
9. ACK
User with
“symbolic name”
calls another
Proxy Mode
10. Ok
11. Ok
Site B
Proxy forwards requests
Possibly in parallel to several hosts
Cannot accept or reject call
Useful to hide location of callee

37. SIP Operation – Redirect Mode
Location Server
2. Where?
Site A
1. Invite 3.
Redirect Mode
4. Moved temporarily
Location:
User with
“symbolic name”
5. ACK
calls another
7. “Ring”
6. Invite
7. Ok
8. ACK
Site B

38. SIP – Methods Basic Methods (RFC 2543)
TABLE
Additional Methods (partially standardized)
INFO: carry information between User Agents
REFER: ask someone to send an INVITE to another participant
SUBSCRIBE: request to be notified of specific event
NOTIFY: notification of specific event