CS335 Principles of Multimedia Systems Multimedia Over IP Networks -- III Hao Jiang Computer Science Department Boston College Nov. 13, 2007

CS335 Principles of Multimedia Systems Outline  We will discuss different issues related to robustness of multimedia data over IP network.  Topics include: Error Correction Error Concealment Congestion Control

CS335 Principles of Multimedia Systems Error Correction for Bit Errors  Schemes to make encoded media stream resistant to bit errors: –Adding error correction codes. –Insertion of re-synchronization bits. –Bi-directional entropy code. –Self-synchronization entropy code. –Error resilient entropy coding (EREC).

CS335 Principles of Multimedia Systems Error Resilient Entropy Coding  Basic Idea: Convert variable length code to some kind of fixed length code

CS335 Principles of Multimedia Systems EREC Encoding Phase1Phase2 Phase3 Final

CS335 Principles of Multimedia Systems EREC Encoding Scheme 1. The original code to be trans-coded is {Ci}, i = 0,…,M-1 2. Select slot width {Si} =0,…,M-1. Sum( Si ) = Sum ( length(Ci) ). Select search sequence {D(n)}, n =0,.., M-1. Set Phase = 0; 3. For i = 0 to M-1 If Si >= length(Ci) Put the code in slot Si; Update left space; else Search slot mod( i+D(phase), M ); If there is space left, put the code in; 4. If all the codes fit into the slot Terminate; else phase++; Goto 3

CS335 Principles of Multimedia Systems EREC Decoding Scheme 1. Get the header information about slotlength and slotwidth and M. 2. Set Phase = 0; Set {codelen(i) = 0, i = 0, …, M-1}, Set {cflag(i) =0, I=0,1, …, M-1} 3. For i = 0 to M-1 If codeword in slot i is fully decodable. Store codelen(i) = code length; else search slot mod( i+D(phase), M ); Get bits from the position of codelen(mod( i+D(phase), M )); If fully decodable, update codelen(i) and cflag(i). Else get all the bits left in the searching slot. 4. If no cflag(i) == 0 or some no change since last phase, stop else phase++; Goto 3

CS335 Principles of Multimedia Systems EREC for JPEG Coding EREC Code Decode Header Information Original code Recovered code

CS335 Principles of Multimedia Systems JPEG Coding in Noisy Channels Original imageJPEG coding of Lena. Q=50.

CS335 Principles of Multimedia Systems JPEG Coding in Noisy Channels JPEG coding of Lena. Q=50. BER=0.01JPEG coding of Lena. Q=50. BER=0.001

CS335 Principles of Multimedia Systems JPEG Coding in Noisy Channels JPEG coding of Lena. Q=50. BER=0.0001JPEG coding of Lena. Q=50. BER=

CS335 Principles of Multimedia Systems EREC-JPEG in Noisy Channels EREC-JPEG coding of Lena. Q=50. BER=0.01 EREC-JPEG coding of Lena. Q=50. BER=0.001

CS335 Principles of Multimedia Systems EREC-JPEG in Noisy Channels EREC-JPEG coding of Lena Q=50. BER=0.001 EREC-JPEG coding of Lena Q=50. BER=0.0001

CS335 Principles of Multimedia Systems Another Example on Color Image Coding Original imageJPEG coding picture, Q=50

CS335 Principles of Multimedia Systems JPEG Coding in Noisy Channel JPEG coding in with BER = 0.01JPEG coding in with BER = 0.001

CS335 Principles of Multimedia Systems EREC Coding Results Error resilient image coding in BER = Error resilient image coding in BER = 0.01.

CS335 Principles of Multimedia Systems Comparison Results Performance comparison JPEG and EREC. Lena. Q = 50. BPP = Random error.

CS335 Principles of Multimedia Systems Comparison Results Figure 5. Performance comparison JPEG and EREC. Gold Hill. Q = 50. BPP = Random Error.

CS335 Principles of Multimedia Systems Error Correction for Packet Loss  An error correction method P1 P2 P3 C=P1 xor P2 xor P3 Reconstructed packet P1 P1 xor P3 xor C P3 C

CS335 Principles of Multimedia Systems Another Error Correction Scheme D(n) R(n-1)D(n+1) R(n)D(n+2) R(n+1)D(n+3) R(n+2) P(n)P(n+2)P(n+3) R(n) is usually a “rougher” coding of D(n) Recovered packet n+1

CS335 Principles of Multimedia Systems Dealing with Burst Packet Loss  Transform burst loss into random loss -- interlacing Burst packet lost 235 Received packet loss pattern is random. Original packets Scrambled packets Received packets

CS335 Principles of Multimedia Systems Error Concealment  Error correction usually needs additional error correction bits.  In fact, it is possible to recover or at least cover the lost data using only the correctly received data – error concealment.  Error concealment can be used for images, videos and audios.

CS335 Principles of Multimedia Systems Images with Partial Data Missing 25% blocks lost 50% blocks lost

CS335 Principles of Multimedia Systems Reconstructed Images Reconstructed images for cases of 25% and 50% block loss respectively.

CS335 Principles of Multimedia Systems Directional Adaptive Error Concealment Color continuous Orientation continuous Continuous in the desired orientation Missing image block

CS335 Principles of Multimedia Systems Error Concealment for Videos Previous I or P P frame v  For MPEG video, we can simply recover the lost blocks in P frames by replacing the lost blocks with the corresponding blocks in the previous I or P frame.

CS335 Principles of Multimedia Systems Congestion Control  The need for congestion control –Internet is a network that does not regulate the traffic of each source strictly. –Aggressive sources may obtain more bandwidth than other information sources. –When throughput in a network exceeds some threshold, congestion collapse will happen. Packet sending rate Packet delivering rate Normal operation Congestion collapse

CS335 Principles of Multimedia Systems TCP Congestion Control  TCP uses a slow linear increase and exponential decrease scheme for congestion control. Slow start Fast decrease Time Packet Rate

CS335 Principles of Multimedia Systems TCP Congestion Control (cont) Packet 1 ACK packet2 ACK Send-and-wait protocol. Maximum packet rate is 1/round_trip_time.

CS335 Principles of Multimedia Systems Sliding Window Scheme Packet 1 ACK Packet 2 Packet 3 Packet 4 Packet 5 Packet 6 If there is no bandwidth limitation, we can send data as fast as we can.

CS335 Principles of Multimedia Systems TCP Congestion Control  TCP protocol controls the bandwidth usage by adjusting the sliding window size (the number of packets sent out in a burst).  TCP detects congestion by timeout or the number of duplicate ACK. Packet 1 Packet 2 Packet 3 x Ack1 Packet 1 Packet 2 Packet 3 Ack1 Ack3

CS335 Principles of Multimedia Systems Multimedia and Congestion Control  Should multimedia traffic follow the same rules like TCP?  It depends. But some kind of scheme to avoid the congestion collapse of the whole network is worth to study.  Such congestion control schemes generally follow the same pattern like TCP congestion control schemes: Slowly increase the bandwidth usage to fully use the network resources. Reduce the usage of bandwidth if congestion is likely to happen.

CS335 Principles of Multimedia Systems TCP Friendly Protocols  For applications such as video conferencing, the video source can adapt its bit rate by changing the quantization factor.  Video source detects congestion condition by the parameters such as round trip time, packet loss ratio etc. using RTCP.  The bit rate of the video source can increase linearly in each round trip time if no congestion happens.  If congestion occurs, the video source should decrease the rate (as much as possible like a TCP source).