Streaming To Mobile Users In A Peer-to-Peer Network Jeonghun Noh, Mina Makar, and Bernd Girod September 8, 2009 Image, Video and Multimedia Systems Group Information Systems Laboratory Stanford University

Heterogeneous Mobile Devices Supporting heterogeneity Screen size: 160x120, 320x240, 480x320, … Channel characteristics : GPRS, EDGE, 3G, WiFi, … Video capabilities: H.263, H.264, … Mobile phone: more than a communication tool. Personal entertainment center Real time video streaming -> we need to consider phones’ differences List the different categories Conclusion: transcoding is required. EMPHASIZE: TRANSCODING.  Transcoding adapts video to mobile devices

Streaming to Mobile Device server Client-server streaming Peer-to-peer streaming Client-server model: server is a bottlenet P2P is getting popular. <- Explain it. However, technical challenge: unreliable peer. In this talk, we propose a new coding scheme suitable for unreliable p2p network. Mention: mobile phones are leeches in P2P network ? Transcoding Distributed transcoding mobile phone

Outline Transcoding at multiple parents IDT: Interleaved Distributed Transcoding Encoding Decoding under lossy condition Advantage of multiple parents Experimental results for distributed transcoding

Transcoding at Multiple Parents Regular transcoding Parent 1 Parent 2 Parent 3 Multiple Description Coding (MDC) Parent 1 Parent 2 Parent 3 A train of frames represents the raw content to transcode. Blue frames: encoded AND transmitted Orange frames: encoded BUT not transmitted => waste of CPU power MDC: requires multiple decoders at a mobile IDT: produces a single bitstream at a mobile => suitable for mobile phones (any typical H264 decoder will work) Interleaved Distributed Transcoding (IDT) Parent 1 Parent 2 Parent 3

Distributed Transcoding: Encoding Original video 1 2 3 4 5 6 7 8 9 10 11 12 13 14 I1 B2 B3 P4 B5 B6 P7 B8 B9 P10 B11 B12 P13 B14 5 9 1 1 5 9 13 13 14 Parent 1 DETAILS: We use the multiple-reference frame functionality of H.264/AVC baseline profile. Note that it is available in most mobile phones. Fixed-line peers receive original video. -> first downsample it. I frames, P frames.. Encoded only at a peer. POINT 1: GOP size can be made independent of that of the original video POINT 2: P frames in a substream are not used as a reference in the other substreams. To complete substreams, the remaining frames are encoded as P frames, but they contain control bits for copying the entire previous frame => independent of the original video content => thus, very small. 6 10 2 2 6 10 Parent 2 7 11 3 1 3 7 11 13 Parent 3 8 12 4 1 4 8 13 Parent 4 I1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 I1 P2

Distributed Transcoding: Decoding 1 5 9 Substream 1 1 1 1 5 5 5 9 9 9 2 6 10 2 6 10 Substream 2 2 2 6 6 10 parent loss 1 3 7 11 Substream 3 I frames are contained in a substream only. Decoder assembles substreams. For missing frames, decoder takes collocating copy frames. The assembled stream -> can be fed to any typical H.264 decoder. Copy frames will perform copy error concealment at the bit level. 4 12 Substream 4 1 1 4 4 4 8 8 8 8 packet loss Stream assembled at decoder

Advantage of Multiple Parents parent loss time Effective frame rate 4 parents Mention: the average number of missing frames is independent of the number of parents Emphasize how we address peer churn challenge and its implication on video quality Note: No packet loss (due to network congestion) is assumed Trade-off: Severity and frequency of video degradation 4 parents time Effective frame rate

Experimental Setup Video: Foreman and Mthr & Dthr (H.264 main profile, CIF, 30 fps) Distributed transcoding (H.264 baseline profile) Downsampling (CIF to QCIF) Number of parents: 1 ~ 4 peers Lossy scenario Peer lifetime ~ Exp(1/90) Reconnection time ~ Exp(1/3) Gilbert model-based channel 0.01 0.99 0.85 Good Bad No packet loss 50% packet loss 0.15 [Konrad , Zhao, Joseph, Ludwig 2003]

End-to-End Rate-Distortion Performance Foreman sequence

Distribution of Transcoding Load 60% load reduction Mention: the vertical axis shows the average time of encoding one frame Encoder: x264-based software encoder CPU: Pentium 4 with 2.8GHz clock speed

Demo Setup SPPM peers SPPM mobile SPPM server SPPM: Stanford Peer-to-Peer Multicast streaming system [Baccichet, Noh, Setton, Girod 2007]

Graceful Video Degradation

Conclusions Video transcoding to meet mobile device heterogeneity Distributed transcoding in a P2P network Graceful video degradation against peer/packet loss Distribution of transcoding load Extensions Rate control based on wireless channel condition Application to fixed-line peers with limited downlink speed

Thank you! Jeong-hun Noh jhnoh@stanford.edu

Comparison: MDC and IDT