VIPER DSPS 1998 Slide 1 A DSP Solution to Error Concealment in Digital Video Eduardo Asbun and Edward J. Delp Video and Image Processing Laboratory (VIPER) Purdue Multimedia Testbed School of Electrical and Computer Engineering Purdue University West Lafayette, IN (fax)

VIPER DSPS 1998 Slide 2 Outline Introduction Error Concealment Fast Near-Optimal Approach Current Work Conclusions

VIPER DSPS 1998 Slide 3 VIPER Research Projects Scalable Video and Image Compression –color still image compression (CEZW) –high and low bit rate video compression (SAMCoW) –ATM, Internet, and wireless video Error Concealment Content Addressable Video Databases (ViBE) Scene Change Detection and Identification Multimedia Security: Digital Watermarking Embedded Real-Time Image and Video Processing

VIPER DSPS 1998 Slide 4 VIPER Research Projects Multiresolution Edge Detection Analysis of Mammograms Binocular Shape Reconstruction

VIPER DSPS 1998 Slide 5 Error Concealment in Digital Video Scenario: –Compressed digital video streams are transmitted over ATM or wireless networks –Cell or packet loss due to channel errors or congestion causes data to be dropped in the channel –When MPEG video is transmitted, macroblocks and motion vectors are missing from the stream Visual quality is lower Potential loss of synchronization

VIPER DSPS 1998 Slide 6 Error Concealment in Digital Video Problem: –Given an encoded video bit stream, how can the stream be packed into ATM cells and transmitted over a lossy channel in such a way to minimize the effect of cell losses? –How can the bit stream be processed to conceal cell loss? Detect where the lost data is located Process the bit stream to conceal the lost data

VIPER DSPS 1998 Slide 7 Applications of Error Concealment All set-top decoder boxes will require some form of error concealment running in real-time This requires simple and robust approaches that can be implemented on DSP-based embedded systems

VIPER DSPS 1998 Slide 8 Forms of Error Concealment Active Error Concealment: –Error control coding –Retransmission –Unequal error protection Passive Error Concealment: –Postprocessing of video stream to reconstruct missing data –Necessary in many applications where active error concealment fails or cannot be used

VIPER DSPS 1998 Slide 9 Passive Error Concealment: Statistical Spatial Approach Missing information is reconstructed based on data from undamaged portions of the current frame Each frame is modeled as a Markov Random Field A maximum a posteriori (MAP) estimate of the missing macroblocks is obtained using this model Optimal solution obtained iteratively –Drawback: time consuming

VIPER DSPS 1998 Slide 10 Passive Error Concealment: Fast Near-Optimal Approach The median of the neighbors of a missing pixel is an approximate MAP estimate Attractive for real-time implementation due to its simplicity Consists of two stages: –Initialization –Filtering

VIPER DSPS 1998 Slide 11 Initialization Stage

VIPER DSPS 1998 Slide 12 Filtering Stage

VIPER DSPS 1998 Slide 13 Objective Implement fast near-optimal approach in real-time This technique will be used on MPEG video over ATM and in Internet video The Texas Instruments TMS320C6201 will be used

VIPER DSPS 1998 Slide 14 Current Work: Error Concealment System As a proof of concept, focus on implementing the fast near-optimal error concealment technique using a C6x-based Spectrum Signal Detroit board Transmission of MPEG video over ATM involves the implementation of real-time MPEG encoder and decoder Simulate this stage by randomly dropping 8x8 blocks of pixels from uncompressed frames

VIPER DSPS 1998 Slide 15 Block Diagram of Error Concealment System

VIPER DSPS 1998 Slide 16 System Description The C6201-based Detroit board is used as a computational engine Video is digitized using a Matrox Corona board Digitized frames are transferred to the Detroit board where they are processed After processing is completed, frames are transferred back to the Corona board for display

VIPER DSPS 1998 Slide 17 System Block Diagram

VIPER DSPS 1998 Slide 18 Implementation Details Video format used is ITU-601 (CCIR-601), YUV 4:1:1, 720x480 pixels Frames are transferred from the digitizer into the on- board memory of the Detroit board via the PCI bus Kernel of the error concealment system is 3x3 median filter Hand-optimized assembly routine implemented –126 cycles for 3x3 short integers

VIPER DSPS 1998 Slide 19 Results

VIPER DSPS 1998 Slide 20 Conclusion Implemented fast near-optimal error concealment technique on the TMS320C6x Current Work: –Motion Compensation (MPEG-4) Future Work: –Real-time wavelet compression algorithm (SAMCoW)