MESA LAB Depth ordering Guimei Zhang MESA LAB MESA (Mechatronics, Embedded Systems and Automation) LAB School of Engineering, University of California, Merced E: Phone: Lab: CAS Eng 820 (T: ) Sep 22, Monday 4:00-6:00 PM Applied Fractional Calculus Workshop MESA UCMerced

MESA LAB Introduction 09/ AFC Workshop UCMerced Slide-2/1024 What is depth ordering? (a) Imput image(b) Edge image (b) Depth ordering

MESA LAB Introduction (a) Imput image (b) Edge image (c) Contour completion (d) Image layer

MESA LAB Applications (why to do this work?) Image segmentation Object recognition Target tracking Scene understanding 09/22/2014 AFC Workshop UCMerced Slide-4/1024 Introduction

MESA LAB Depth ordering algorithm based on T-junctions and occlusion reasoning 09/22/2014 AFC Workshop UCMerced Slide-5/ Motivation 2.Method 3.Experiments 4.Conclusion

MESA LAB 1. Motivation T-junction points convexity 09/22/2014 AFC Workshop UCMerced

MESA LAB 09/22/2014 AFC Workshop UCMerced Slide-7/ Motivation Problems: Existed methods have limitations to order objects completely, especially in multiple backgrounds.

MESA LAB conventional methods always detect T-junctions before segmentation, which will result in detecting false T-junctions or missing real T-junctions in clutter images 1. Motivation

MESA LAB 2. Method overcomes the first problem by introducing high level occlusion reasoning theory when some regions include no T-junction, no convexity or inconsistent T-junction point

MESA LAB 2. Method We combine low level depth cue (T-junctions) and high level occlusion reasoning, therefore make progress to order the objects completely, even in multiple backgrounds. In addition, conventional methods always detect T- junctions before segmentation, which will result in detecting false T-junction or missing real T-junctions in clutter images.

MESA LAB 2. Method Character 1: T-junction is composed by three boundaries and only two boundaries are collinear, in other words, the angle between them is 180 degree. Two collinear ones are named as occlusion boundaries, and the other is called occluded boundary. Character 2: The region contained occlusion boundaries is in front of the one included occluded boundary. 2.1 T-junction analysis:

MESA LAB In previous work, T-junctions are detected before segmentation. The shortcomings of this kind of methods are as follows : it is easy to detect false T-junctions due to the complexity of the real images and texture of some objects; 09/22/2014 AFC Workshop UCMerced Slide-12/ Method

MESA LAB preserve T-junctions before image segmentation and remove the false T-junctions, in other words the post-processing is time consuming detection T-junction method based on image is more complex than one based on contour. So we first segment real image and get the contour of image, then detect T-junctions on the contour image. 09/22/2014 AFC Workshop UCMerced Slide-13/ Method

MESA LAB 2. Method Detection T-junction points 09/22/2014AFC Workshop UCMerced

MESA LAB 2. Mehtod 2.2 occlusion reasoning visual psychology principle: The figure (foreground) has definition shape, but the background has not, if the background is perceived as having certain shape, that is due to the other gestalt. The background seems continuous stretch without being interrupted behind the figure. The figure always appears in the front and the background is in the back. The figure can give human more deep impression, and easier to remember.

MESA LAB Reasoning laws( inspired by human cognition): Law 1: If the background has not definition shape, the region which has definition shape is in front of the one which has not. Law 2: When the background has definition shape, we first remove part objects formed the boundary of background, and can get the region which has definition shape is in front of the one which has not. 05/05/2014 AFC Workshop UCMerced Slide-16/ Mehtod

MESA LAB Law 3: The lower the background region in the image is more likely to be closer to viewpoint when there are multiple background regions in the scene. 09/22/2014 AFC Workshop UCMerced Slide-17/ Mehtod

MESA LAB the method is as follows: 09/22/2014 AFC Workshop UCMerced Slide-18/ Method

MESA LAB 3. Experiments First: input image Sec: T-junction detection Last: The depth map ( rendered as a gray level image, and high values indicate regions closer to the viewpoint) Experiment result

MESA LAB 3. Experiments Experimental results

MESA LAB 3. Experiments Comparison with the state of the art : (a) input image (b) T-junction detection (c) The depth-map obtained by the method in Ref [7] (d) The depth-map obtained by our method

MESA LAB 3. Experiments (a) input image (b) segmentation (c) The depth-map obtained by the method in Ref [8] (d) The depth-map obtained by our method

MESA LAB 3. Experiments (a) input image (b) T-junction detection of our method (c) The depth- map obtained by our method (d) T-junction detection of Ref [6] (e) The depth-map obtained by the method in Ref [6] 09/22/2014AFC Workshop UCMerced

MESA LAB 09/22/2014 AFC Workshop UCMerced Slide-24/1024

MESA LAB 4. Conclusion  A new T-junctions detection method based on contour is proposed in this paper, which can accurately detect the T- junctions on an already segmented image.  And Monocular depth ordering algorithm based on low level depth cue (T-junctions) and high level occlusion reasoning is proposed in this paper. AFC Workshop UCMerced 09/22/2014

MESA LAB  The initial depth image ordering is first obtained based on T-junction; and then more detail depth ordering can be achieved by using of high level occlusion reasoning.  Results are compared with the method using depth cue (T-junction and convexity) and the method optimization algorithm based frameworks, our method can get the perfect depth ordering, and can establish global and consistent depth interpretation. 4. Conclusion 09/22/2014AFC Workshop UCMerced

MESA LAB Thanks AFC Workshop UCMerced09/22/2014