Joint Source Network Coding for Server DSN 30/C/3M A B 30/C/2M C 30/C/1M D 15/Q/1M E F 30/C/1M G 15/C/384k 15/Q/384k A-G : users and their requirements (frame rate/resolution/bit rate) DSN : Data service node, which performs data adaptation inside the network. JSNC uses overlay infrastructure to assist video streaming to heterogeneous users simultaneously by providing light weight support at intermediate overlay nodes. Yufeng Shan, Ivan V. Bajic, Rensselaer Polytechnic Institute, USA Simon Fraser University, Canada

JSNC includes two basic concepts  Integrated Source-Network Video Coding (IVC) Video coding function is distributed both at source and inside the network to facilitate simple and precise adaptation of bitstream for heterogeneous users.  Fine Granular Adaptive FEC (FGA-FEC) Encoding once, the proposed FGA-FEC scheme can adapt the FEC coded bitstream to satisfy multiple heterogeneous users simultaneously without FEC decoding/re-encoding at intermediate overlay nodes.

Heade r Q 2 MV...Q lt MV...Q Lt MV Q 1,1 YUV …Q lt,1 YUV …Q Lt,1 YUV Q 1,2 YUV …Q lt,2 YUV …Q Lt,2 YUV …………… Q 1,ls YUV …Q lt,ls YUV …Q Lt,Ls YUV Subband coefficient bitstream ( Q YUV ) Lower resolution/quality /frame rate bitstreams are embedded in higher frame rate/resolution /quality bistreams and can be directly abstracted. Integrated Source Network Video Coding (IVC) Motion vector bitstream (Q MV )  Server does video coding, DSNs adapt the bitstream based on network conditions and user requirements.  Each GOP coding unit consists of independent bitstreams {Q MV, Q YUV }.

A(F, Q, R) represents an atom of {frame rate, quality, resolution}. A(0,0,0) A(0,1,0)A(0,2,0)A(0,3,0)A(0,4,0) A(0,0,0) A(1,0,0)A(1,1,0)A(1,2,0)A(1,3,0)A(1,4,0) A(0,0,0) A(2,0,0)A(2,1,0)A(2,2,0)A(2,3,0)A(2,4,0) A(0,0,0) A(3,0,0)A(3,1,0)A(3,2,0)A(3,3,0)A(3,4,0) A(0,0,0) A(4,0,0)A(4,1,0)A(4,2,0)A(4,3,0)A(4,4,0) A(4,0.1) A(4,1,1) A(4,2,1)A(4,3,1) A(4,4,1) A(4,0,2)A(4,1,2)A(4,2,2)A(4,3,2)A(4,4,2) Resolution Quality Frame Rate Intermediate DSNs adapt the digital items according to user preferences and network conditions, different subsets of atoms are chosen for different users. Encoded bitstream can be illustrated as Digital Items in view of three forms of scalability (frame rate /quality /resolution).

Scalable Overlay Video Streaming Bitstream is divided into small blocks; FEC is added vertically across blocks; … FEC … B2 B3 … …Bi … FEC ……… C2 C3 … … … C4 FEC Cj FEC ……… … … … … … … … … … … Xn A1B1……C1…… ABC… X …X1 Description 1 Description 2 … Description n FEC …… Shivkumar Kalyanaraman, and John W. Woods Rensselaer Polytechnic Institute, USA FGA-FEC Concept

 When part of the video bitstream is actively dropped (adapted), FEC codes need to be updated accordingly by removing related block(s) from each description, no FEC transcoding is needed. Service Node  Green and blue blocks are removed from each description, including both original data and FEC blocks. Each horizontal line is one description (packet).

Joint Design FGA-FEC encoded Scalable bitstream is reorganized for 3-D adaptatioin; Adaptation of SNR can be easily achieved by removing related vertical blocks from each packets The green bars are FEC data, others are original video data.

Simulations Source Coding vs IVC Effect of block size on IVC JSNC is almost as precise as source coding, only 0.08 dB lower than source coding in this case. Sequence: Foreman CIF; Encoder: MC-EZBC; IVC block size: 8 bytes; Available bandwidth 990 Kbps.

(a) JSNC vs Random Drop (b) 3-D adaptation (a) 1500 Kbps bitstream to suit the 1455 Kbps channel; (b) 2 Mbps bistream is adapted to (1) SNR 512 Kbps; (2) Spatial to QCIF; and (3) Temporal to ¼ frame rate at intermediate overlay node.

Compare the encoding efficiency with MD-FEC JSNC is almost as good as MD-FEC Number of bits in each layer Bit rate as layer adds up

Bit allocation performance compare using JSNC and optimal solution; The average JSNC is only 0.02 dB less than optimal solution But with much faster speed, which can serve more users