1. Electrical Engineering National Central University Video-Audio Processing Laboratory Data Error in (Networked) Video M.K.Tsai 04 / 08 / 2003

2. Electrical Engineering National Central University Video-Audio Processing Laboratory  what’s data error ?  cause of data error  kind of data error  effect of data error  resilience of data error  summary Outline

3. Electrical Engineering National Central University Video-Audio Processing Laboratory What’s Data Error ?  the data dealt with lossy processing  the data received in the decoder is different from that transmitted in the encoder encoderdecoder channel Q F -1 F MC/ME VLC Q -1

4. Electrical Engineering National Central University Video-Audio Processing Laboratory  channel  bit error 、 burst error 、 packet loss  fading 、 multi-path 、 ISI 、 overflow in the router  process  lossy coding (ex: Q)  re-quantization (ex: drift)  DCT/IDCT mismatch (ex: SPDT) ……  error concealment Cause of Data Error

5. Electrical Engineering National Central University Video-Audio Processing Laboratory  channel  packet packet sequence number  header invalid code(ex:MBTYPE 、 MCBPC ……)  image data invalid code(ex:VLD 、 MV) 、 out of range(IDCT 、 MV) Kind of Data Error VLD F -1 Q -1 MC/ME MV

6. Electrical Engineering National Central University Video-Audio Processing Laboratory  packet loss  once errors found in a packet, it would be discarded  block/frame loss  once loss of synchronization happens at receiver, blocks or frames would be dropped  depend on the size of packet, packet loss may lead to block /frame loss Effect of Data Error

7. Electrical Engineering National Central University Video-Audio Processing Laboratory  error propagation  VLD  MV  Concealment Effect of Data Error original bit error new original new original new

8. Electrical Engineering National Central University Video-Audio Processing Laboratory  channel  forward error correction (FEC) open loop error recovery method increase transmission overhead complicated decoding process not suitable for mobile network  automatic repeat request (ARQ) close loop error recovery method effective against burst error and packet loss more effective than FEC not realistic for real time transmission Resilience of Data Error

9. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  error resilience tools resynchronization marker is uniquely identifiable data partition (DP) MVs and DCT coefficients are separated correct data can be used for video reconstruction If undetected errors occur in packets, received data can be considered invalid if boundary marker is’ t detected header extension code (HEC) Important header information is repeated following HEC

10. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  error resilience tools reversible variable length coding (RVLC) VLCs is uniquely decodable in both directions error concealment (EC) highly dependent on the performance of the resynchronization scheme

11. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  Multiple Description coding (MDC) allow decoder to extract meaningful information from a subset of the bit stream encoder produce descriptions transmitted over channels MDC Encoder channel Decode 0 Decode 1 Decode 2

12. Electrical Engineering National Central University Video-Audio Processing Laboratory  channel  error mitigation by feedback Reference Picture Selection (RPS) reference picture is replaced adaptively according to the upstream messaging from the decoder NACK is effective for low error rates, ACK is effective for high error rates Error tracking use intra mode for MB severely distorted Resilience of Data Error

13. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  packetization duplicate important information in multiple packet duplicate important packets how many packets per frame packetization delay

14. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  scalable coding with unequal error protection (UEP) SNR : by different quantization spatial : by different resolution temporal : by different frame rate  error confinement video object planes (VOP’s)coding independent segment decoding (ISD)

15. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  Error Resilient Entropy Coding (EREC) Convert VLCs to fixed-length blocks of data not guarantee image/video frame synchronization

16. Electrical Engineering National Central University Video-Audio Processing Laboratory Resilience of Data Error  channel  video redundant coding picture sequences are assigned to more threads and each thread is coded independently

17. Electrical Engineering National Central University Video-Audio Processing Laboratory  process  error refreshment adaptive intra refresh (AIR) avoid large bursts of intra MB’s for short periods channel characteristics can be used as criteria cyclic intra refresh (CIR) in error-free environment  multiple reference frame Resilience of Data Error

18. Electrical Engineering National Central University Video-Audio Processing Laboratory  process  leaky prediction by attenuation of the energy of the prediction signal the energy of superimpose errors decays over time Resilience of Data Error Q F -1 F ME Q -1 0.9 source

19. Electrical Engineering National Central University Video-Audio Processing Laboratory  process  unrestricted motion vector (UMV) range of motion vectors and VLC table is different range of motion vectors depends on the picture size motion vectors are allowed to point outside the picture  advanced prediction (AP) overlapped block motion compensation (OBMC) four motion vectors per macroblock  motion compensation (MC) half-pel 、 quarter-pel using bilinear interpolation Resilience of Data Error

20. Electrical Engineering National Central University Video-Audio Processing Laboratory  I have introduced  what’s data error  cause of data error  kind of data error  effect of data error  resilient of data error  by the introduction, we have basic realization on error  nothing is terrible but error Summary

21. Electrical Engineering National Central University Video-Audio Processing Laboratory Q1 -1 F -1 MC Q2 F -1 F MC/ME Q2 -1 Transcoding  Cascaded Pixel-Domain Transcoder (CPDT)  due to different reconstructed frames, drift is generated

22. Electrical Engineering National Central University Video-Audio Processing Laboratory Transcoding  Simplified Pixel-Domain Transcoder (SPDT)  to reduce complexity of CPDT, by following simplification F -1 = F F = =

23. Electrical Engineering National Central University Video-Audio Processing Laboratory Transcoding  Simplified Pixel-Domain Transcoder (SPDT)  under the following assumptions, SPDT is generated no clipping functions considered DCT and IDCT has infinite accurancy Q1 -1 MC Q2 F -1 Q2 -1 F + - + -