A hybrid error concealment scheme for MPEG-2 video transmission based on best neighborhood matching algorithm Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005

Outline Introduction Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames Simulation results

Introduction (1/2) Transmission errors and error propagation Network Encoded stream Original pictures Decoded pictures

Introduction (2/2) Error control ◦ Channel coding  ◦ Error resilient coding  ◦ Detection and correction  ◦ Detection and concealment  Encoded stream += currentpreviousfuturecurrent Encoder control Decoder control

Proposed hybrid error concealment scheme for MPEG-2 I frame ◦ Spatial search for similar blocks ◦ Selection between H.26L test model and FBNM P frame ◦ Temporal search for similar blocks ◦ Determination of block behavior  Small motions, middle motions, and large motions ◦ Search for motion vectors

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (1/7) H.26L test model [25] Best neighborhood matching (BNM) [15] a1a1 a2a2 a3a3 a4a4 N+mN+m N+mN+m H L Best match

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (2/7) Proposed fast BNM ◦ Step 1: Initial candidate block by MAD ◦ Step 2: Hexagon-based fast search by MAD

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (3/7) Proposed hybrid error concealment scheme ◦ Fitness function  H.26L or fast BNM?  Check the smoothness of the boundaries between the concealed block and its neighbors  Check the similarity between the concealed block and its neighbors  Components  AIDB  AMD  AVD

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (4/7) Average intersample difference across the block boundaries between a concealed block and its four neighbors (AIDB) ◦ Smoothness of the boundaries

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (5/7) Average mean difference between a concealed block and its eight neighbors (AMD) mean

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (6/7) Average variance difference between a concealed block and its eight neighbors (AVD)

Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (7/7) Fitness function Choose of concealed block ◦ Purpose of T C : B H.26L is usually smaller than B FBNM, but it’s visual quality is not always better than B FBNM

Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames (1/2) MV magnitude Determination of types of the corrupted blocks Block matching Small motion blocks Candidates: MV 0 MV ave Middle motion blocks Candidates: MV i for i = 1 to 8 MV ave MV medium Large motion blocks Candidates: Fast search algorithm TlTl ThTh

Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames (2/2) Fast search algorithm for large motion blocks Search range for the corrupted block 2  D+1

Simulation results (1/8) MPEG-2 ◦ GOP size = 12 ◦ IBBP structure ◦ 1.5 Mbps ◦ 30 fps ◦ One packet contains one slice CIF ◦ Coastguard ◦ Football ◦ Salesman ◦ Table tennis

Simulation results (2/8) parameters ◦ I frame  N = 16; m = 4; L = H = 120 ◦ P or B frame  T l = 8; T h = 12; T C = 200; D = 2 N+mN+m N+mN+m H L TlTl ThTh

Simulation results (3/8) Discussions ◦ H and L for I frame  Smaller: good candidates may be excluded  Larger: bad candidates may be included ◦ T l and T h for P frame  Smaller: better concealment results but high computation cost  Larger: worst concealment results but low computation cost ◦ D for P frame  Smaller: worst concealment results but low computation cost  Larger: better concealment results but high computation cost

Simulation results (4/8) 1. replace all corrupted blocks by zeros 2. I frame: H.26L; P/B frame: overlapped region matching [21] 3. I frame: split matching; P/B frame: forward-backward block matching [22] 4. I frame: H.26L; P/B frame: neighbor matching [23] 5. Proposed MV interpolation

Simulation results (5/8) Football Salesman Processing time (10% PLR) 1234

Simulation results (6/8) Visual quality (15% PLR)

Simulation results (7/8) Visual quality (15% PLR)

Simulation results (8/8) Football PLR = 15%

References [15] Z. Wang, Y. Yu, D. Zhang, Best neighborhood matching: an information loss restoration technique for block-based image coding systems, IEEE Trans. Image Process. 7 (7) (1998) 1056–1061. [21] M.C. Hong, L. Kondi, H. Scwab, A.K. Katsaggelos, Video error concealment techniques, Signal Process.: Image Commun. 14 (1999) 437–492. [22] S. Tsekeridou, I. Pitas, MPEG-2 error concealment based on block-matching principles, IEEE Trans. Circuits Syst. Video Technol. 10 (4) (2000) 646–658. [23] S. Valente, C. Dufour, F. Groliere, D. Snook, An efficient error concealment implementation for MPEG-4 video streams, IEEE Trans. Consumer Electron. 47 (3) (2001) 568–578. [25] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, The error concealment feature in the H.26L test model, in: Proceedings IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.