1. Occlusion and smoothness probabilities in 3D cluttered scenes
Michael Langer
School of Computer Science
McGill University
A fundamental vision problem is to estimate the 3D properties of surrrounding space. These 3D properties can include the layout of ground surfaces, walls and ceiling, the distribution of objects, and the orientation of the viewer with respect to the 3D scene. They can also include the detailed shape of objects.

2. 3D cluttered scene
Today I am going to look at a class of scenes, which I called 3D cluttered scene.s These are scenes that consist of a large number of small surfaces which are distributed in a 3D volume. The canonical examine is a bush, shrub, or tree. From just one photograph, it is obviously difficult to estimate the 3D geometry of such a scene. But if we see such a scene in the real world, with stereo and motion, it is easier – two or more viewpoints provide more information.

3. Motivation E data + E smoothness INPUT: binocular image pair
OUTPUT: depth map (or disparity map)
that minimizes an “energy” :
E data E smoothness
In computer vision, the standard approach to solving 3D geometry from two viewing points is classical.
Input is a pair of images (corresponding to left and right eye views). Output is a depth map. The solution (depth map) should obey two general constraints: it should be faithful to the two images ie. A good match, and it should be as smooth as possible. The constraints are often posed in terms of energy minimization.

4. Motivation E data + E smoothness INPUT: binocular image pair,
OUTPUT: depth map (or disparity map)
that minimizes an “energy” :
E data E smoothness
Some methods are probabilistic. For these methods, one often speaks of a likelihood that a depth map produced the pair of images, and the prior probability of having such a depth map. The idea is that have smooth depth maps has higher probability than having non-smooth depth maps.
“likelihood”
“priors”

5. Motivation E data + E smoothness INPUT: binocular image pair,
OUTPUT: depth map (or disparity map)
that minimizes an “energy” :
E data E smoothness
Why “data”?
What “prior”? Chances of having a discontinuty at any given point is relatively low. EVEN IN CLUTTERED SCENES,
The change is low – its just not as low as in non-cluttered scenes.
“likelihood”
“priors”
TODAY : 3D cluttered scenes

6. Overview of Talk Cluttered scenes model Visibility probabilities
monocular
depth, occlusions, smoothness
binocular
“disparity”, “half occlusions”

7. Cluttered scene model
observer

8. Cluttered scene model

9. Visibility and depth
observer
visible
occluded

10. Visibility and depth
observer
visible
occluded

11. Visibility and depth
observer
visible
occluded

12. Visibility and depth
observer
visible
occluded

13. Poisson model
density h (sphere centers)

14. Poisson model V

15. Poisson model V

16. Cluttered Scene Model
density h , radius R

17. Overview of Talk Cluttered scenes model Visibility probabilities
monocular
depth, occlusions, smoothness
binocular
“disparity”, “half occlusions”

18. Monocular Visibility
What is the probability that the surface seen at a pixel is at depth z ?

19. Monocular Visibility
A point at depth Z is visible if no sphere center lies within a distance R from the line of sight.

20. Poisson model V

21. Poisson model

22. Depth
p(Z)
p( Z )
depth Z
depth Z

23. Occlusions
Given a pixel is at depth z, what is the probability that its right neighbor is closer (on a more nearby i.e. occluding surface) ?

24. Occlusions

25. Discontinuities near sphere far sphere
Given a pixel is at depth z, what is the probability that its right neighbor is further away (on a more distant surface) ?

26. Discontinuities

27. Binocular Occlusions
When a binocular observer looks at a 3D scene, there will typically be many points that are not
Visible to each eye. Points that are hidden are said to be OCCLUDED.
For example, consider the simple scene shown here which consists of a foreground surface in front of a background surface.
The foreground surface is said to occlude the background surface.

28. “Half Occlusions” monocular left right
When we consider the points that are visible to each eye, we find that there
Are points on the background surface that are visible to one eye but not
The other. Such points are said to be MONOCULAR points, or they are
Said to be HALF occluded. Here we see points that are visible to the left
Eye but not the right, and points that are visible to the right eye but noth
The left.

29. visible to right eye only
“Half Occlusions”
visible to both eyes
visible to right eye only
occluded

30. Half Occlusions
Given a pixel in the right eye’s image has depth z, what is the probability that the same surface point is visible to the left eye?

31. Binocular half-occlusions

32. Binocular half-occlusions
monocular
binocular

33. Summary of main results
occlusions
depth
monocular
binocular
half-occlusions
discontinuities

34. E data + E smoothness & occlusions
Future Work
INPUT: binocular image pair,
OUTPUT: depth map that minimizes
the “energy” :
E data + E smoothness & occlusions
Why “data”?
What “prior”? Chances of having a discontinuty at any given point is relatively low. EVEN IN CLUTTERED SCENES,
The change is low – its just not as low as in non-cluttered scenes.
priors based on 3D
cluttered scene models

35. Questions ?

36. Disparity (inverse depth)

37. Disparity (inverse depth)

38. Image Velocity