A Fast and Efficient Multi-View Depth Image Coding Method Based on Temporal and Inter- View Correlations of Texture Images Jin Yong Lee Ho Chen Wey Du Sik Park IEEE Transactions on CSVT 2011

Outline Introduction Proposed Method Temporal Correlation Texture and Depth View Synthesis Inter-View Correlation Evaluations Coding Performance Encoding Complexity Analysis Subjective Quality Assessment

Introduction Encode video information of each view individually with the H.264/AVC A multi-view video coding structure with hierarchical B pictures Multi-view video plus depth

Outline Introduction Proposed Method Temporal Correlation Texture and Depth View Synthesis Inter-View Correlation Evaluations Coding Performance Encoding Complexity Analysis Subjective Quality Assessment

Temporal Correlation Texture images Depth images I P

Temporal Correlation Texture images Depth images P I B

Temporal Correlation

Outline Introduction Proposed Method Temporal Correlation Texture and Depth View Synthesis Inter-View Correlation Evaluations Coding Performance Encoding Complexity Analysis Subjective Quality Assessment

Texture and Depth View Synthesis Synthesis the next depth view by 3D image warping Translate gray values into real depth Project the original points into 3D world Reproject these 3D points into the image plane of neighboring view Hole-filling

Texture and Depth View Synthesis The pixel position(x,y) at the reference frame can be projected into a 3D point (u,v,w) The corresponding pixel location of the virtual image Translate gray values into real depth Project the original points into 3D world Reproject these 3D points into the image plane of neighboring view Hole-filling

Texture and Depth View Synthesis Some pixels in the synthesized image are missing or undefined Occlusion Pixel position quantization Only consider the neighboring pixels around the hole If a view is warped to the right view position, the hole is filled with its left pixel Translate gray values into real depth Project the original points into 3D world Reproject these 3D points into the image plane of neighboring view Hole-filling

Texture and Depth View Synthesis

Outline Introduction Proposed Method Temporal Correlation Texture and Depth View Synthesis Inter-View Correlation Evaluations Coding Performance Encoding Complexity Analysis Subjective Quality Assessment

Inter-View Correlation Texture images Depth images I-view P-view Synthesized Image texturedepth

Inter-View Correlation

Evaluations Coding performance Encoding complexity analysis Subjective Quality Assessment

Coding performance Simulation conditions Test sequences

Coding performance Balloons Kendo Book Arrival Newspaper Champagne tower Pantomime BDBR : Average bit-rate difference in % over the whole range of PSNR BDPR : Average PSNR difference in dB over the whole range of bit-rates

Coding Performance Balloons Kendo Book Arrival Newspaper Champane tower Pantomime

Encoding Complexity Analysis ETR and RM indicate the total encoding time reduction and the reduced number of the modes performing the RD optimization in percentage SB and VS represent the proportion of skipped macro blocks in depth image and the time required for the view synthesis process

Encoding Complexity Analysis

Subjective Quality Assessment 15 professional subjects Two stereoscopic view pairs were rendered using the depth images decoded by the original method and the proposed method respectively

Subjective Quality Assessment Y-axis indicates a differential score that subtract a score of the original method from the proposed method