1. Multimedia Applications Ali Saman Tosun Computer Science Department

2. 2 Compression - Necessity  E.g., video sequence  25 images/sec.  PAL standard  3 byte/pixel  YUV (luminance + 2 chrominance values)  RGB (red-green-blue values)  Image resolution 640 * 480 pixel  Data rate = 640 * 480 * 3 Byte * 25/s = 23040000 byte/s ~ 22 MByte/s  Approx. 1/16 stream over Ethernet  Approx. 1/2 stream over Fast Ethernet  Compression is necessary

3. 3 Multimedia Networking Applications Fundamental characteristics:  Typically delay sensitive  end-to-end delay  delay jitter  But loss tolerant: infrequent losses cause minor glitches  Antithesis of data, which are loss intolerant but delay tolerant. Classes of MM applications 1) Streaming stored audio and video 2) Streaming live audio and video 3) Real-time interactive audio and video Jitter is the variability of packet delays within the same packet stream

4. 4 From Broadcast to True Media-on-Demand  Broadcast (No-VoD)  Traditional, no control  Pay-per-view (PPV)  Paid specialized service  Quasi Video On Demand (Q-VoD)  Distinction into interest groups  Temporal control by group change  Near Video On Demand (N-VoD)  Same media distributed in regular time intervals  Simulated forward / backward  True Video On Demand (T-VoD)  Full control for the presentation, VCR capabilities  Bi-directional connection

5. 5 Streaming Stored Video Streaming  media stored at source  transmitted to client  streaming: client playout begins before all data has arrived  timing constraint for still-to-be transmitted data: in time for playout

6. 6 constant bit rate video transmission Cumulative data time variable network delay client video reception constant bit rate video playout at client client playout delay buffered video Streaming Multimedia  Client-side buffering, playout delay compensate for network-added delay, delay jitter

7. 7 Proxy-based Video Distribution Server Proxy Client Proxy adapts video Proxy caches video

8. 8 Proxy Operations  Drop frames  Drop B,P frames if not enough bandwidth  Quality Adaptation  Transcoding  Change quantization value  Most of current systems don’t support  Video staging, caching, patching  Staging: store partial frames in proxy  Prefix caching: store first few minutes of movie  Patching: multiple users use same video

9. 9 Basic Encoding Steps

10. 10 H.261 (px64)  International Standard  Video codec for video conferences at p x 64kbit/s (ISDN):  Real-time encoding/decoding, max. signal delay of 150ms  Constant data rate  Intraframe coding  DCT as in JPEG baseline mode  Interframe coding, motion estimation  Search of similar macroblock in previous image and compare  Position of this macroblock defines motion vector  Difference between similar macroblocks

11. 11 MPEG (Moving Pictures Expert Group)  International Standard:  Compression of audio and video for playback (1.5 Mbit/s):  Real-time decoding  Sequence of I-, P-, and B-Frames:  Random access  at I-frames  at P-frames: i.e. decode previous I-frame first  at B-frame: i.e. decode I and P-frames first

12. 12 MPEG-2  Beyond MPEG-1:  Higher quality encoding  Higher data rates  Interleaved modes  Higher data rates  MPEG-1: about 1.5 MBit/s  MPEG-2: 2-100 MBit/s  Scaling:  Signal to Noise Ration (SNR) scalin  progressive compression  Spatial scaling  several pixel resolutions  Temporal scaling  frame dropping

13. 13 Scalable coding  Typically used as Layered coding  A base layer  Provides basic quality  Must always be transferred  One or more enhancement layers  Improve quality  Transferred if possible Sending rate Quality Best possible quality at possible sending rate Base layer Enhancement layer

14. 14 Temporal Scalability  Frames can be dropped  In a controlled manner  Frame dropping does not violate dependancies  Low gain example: B-frame dropping in MPEG-1

15. 15 Spatial Scalability  Base layer  Downsample the original image  Send like a lower resolution version  Enhancement layer  Subtract base layer pixels from all pixels  Send like a normal resolution version  If enhancement layer arrives at client  Decode both layers  Add layers 72 7583 61 73 -12 210 Base layer Enhancement layer Better compression due to low values Less data to code

16. 16 SNR Scalability  SNR – signal-to-noise ratio  Idea  Base layer  Is regularly DCT encoded  A lot of data is removed using quantization  Enhancement layer is regularly DCT encoded  Run Inverse DCT on quantized base layer  Subtract from original  DCT encode the result  If enhancement layer arrives at client  Add base and enhancement layer before running Inverse DCT

17. 17 Multiple Description Coding  Idea  Encode data in two streams  Each stream has acceptable quality  Both streams combined have good quality  The redundancy between both streams is low  Problem  The same relevant information must exist in both streams  Old problem: started for audio coding in telephony  Currently a hot topic

18. 18 Delivery Systems Developments Network Saving network resources: Stream scheduling Several Programs or Timelines

19. 19 Patching  Server resource optimization is possible multicast Unicast patch stream Central server 1st client2nd client Join ! cyclic buffer

20. 20 Proxy Prefix Caching  Split movie  Prefix  Suffix  Operation  Store prefix in prefix cache  Coordination necessary!  On demand  Deliver prefix immediately  Prefetch suffix from central server  Goal  Reduce startup latency  Hide bandwidth limitations, delay and/or jitter in backbone  Reduce load in backbone Client Unicast Central server Prefix cache

21. 21 Interval Caching (IC)  caches data between requests  following requests are thus served from the cache  sort intervals on length Video clip 1 S 11 Video clip 1 S 11 S 12 Video clip 1 S 12 S 11 S 13 Video clip 2 S 22 S 21 Video clip 3 S 33 S 31 S 32 S 34 I 11 I 12 I 21 I 31 I 32 I 33 I 32 I 33 I 21 I 11 I 31 I 12

22. 22 Receiver-driven Layered Multicast (RLM)  Requires  IP multicast  layered video codec (preferably exponential thickness)  Operation  Each video layer is one IP multicast group  Receivers join the base layer and extension layers  If they experience loss, they drop layers (leave IP multicast groups)  To add layers, they perform "join experiments“  Advantages  Receiver-only decision  Congestion affects only sub-tree quality  Multicast trees are pruned, sub-trees have only necessary traffic

23. 23 Receiver-driven Layered Multicast (RLM)

24. 24 Multimedia Disk Scheduling  Suitability of classical algorithms  minimal disk arm movement (short seek times)  no provision of time or deadlines  generally not suitable  Continuous media server requirements  serve both periodic and aperiodic requests  never miss deadline due to aperiodic requests  aperiodic requests must not starve  support multiple streams  balance buffer space and efficiency tradeoff

25. 25 Group Sweep Scheduling (GSS) GSS combines Round-Robin (RR) and SCAN  requests are serviced in rounds (cycles)  principle:  divide S active streams into G groups  service the G groups in RR order  service each stream in a group in C-SCAN order  playout can start at the end of the group  special cases:  G = S: RR scheduling  G = 1: SCAN scheduling  tradeoff between buffer space and disk arm movement  try different values for G giving minimum buffer requirement – select minimum  a large G  smaller groups, more arm movements, smaller buffers (reuse)  a small G  larger groups, less arm movements, larger buffers

26. 26 Power Management  Based on functionality  Server  Proxy  Client  Based on component  Disk  Processor