1 M15338: Depth Map Estimation Software version 2 April, 27th 2008, Archamps Olgierd Stankiewicz Krzysztof Wegner team supervisor: Marek Domański Chair of Multimedia Telecommunications and Microelectronics Poznan University of Technology, Poland

2 Outline Depth map quality measurement Ground-truth map View resynthesis View synthesis tool Depth map estimation tools Belief Propagation based estimation Accuracy refinement by mid-level hypothesis Summary

3 Depth map quality Commonly used: ‘Bad-Pixels’ Miss information about error magnitude and energy Requires ground-truth disparity map

4 Depth map quality NBP-SAD (Normalized Bad Pixel SAD) NBP-SSD (Normalized Bad Pixel SSD) Still, requires ground-truth disparity map

5 Depth map quality measurement by view resynthesis End-user never sees depth-map Resynthesis No standarized method Tool employs straight-forward method PSNR (Peak Signal-to-Noise Ratio) of resynthesized view as quality measure

6 Bad-Pixels vs PSNR

7 View synthesis tool Simple and straight-forward For linearly positioned stereo pairs only Two disparity maps and corresponding reference views Weighting of pixels from side-views, translated according to their disparity

8 View synthesis tool

9 Belief Propagation based depth estimation tool Alternative for Hierarchical-Shape Adaptive Block Matching Employs message passing for optimization of disparity map hierarchical processing in layers Pixel differences (1-point SAD) used as observations

10 Message passing in Belief Propagation

11 Message in Belief Propagation Single message contains information about all possible disparities

12 Hierarchical processing in Belief Propagation Higher resolution Lower resolution from the lowest resolution to the full resolution in coarse-to-fine manner

13 Belief propagation V pq (x p,x q ) – transition cost in node q between disparity x p and x q insisted by nodeł p V p (x p ) – observation in node p about disparity x p (SAD value) m pq (x q ) – message from node p to q about disparity x q

14 Belief propagation Pot model Simpleand computationally efficient. Stable beliefs are prefered

15 Belief propagation results 1th iteration20 iterations300 iterations Middlebury test results – 1,65% of bad-pixels Best Middlebury algorithm – 0,88% of bad-pixels

16 Bad-Pixels vs PSNR

17 Accuracy refinement by mid-level hypothesis Low computational cost Improves accuracy of disparity map (number of disparity levels) Spatial resolution unchanged Focuses on unit-step edges in disparity map

18 Unit-step edges

19 Mid-level hypothesis Hypothesis spread along unit-step edges

20 Refinement by mid-level hypothesis Pixel accurate disparity (1x)After refinement (8x)

21 Works over untextured regions

22 Results

23 Summary New version of experimental Depth Estimation software Quality measurement problem with respect to multi-view applications Simple view resynthesis tool Belief Propagation depth estimation tool Novel technique for accuracy refinement