1. Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT

2. Motivation Optimization based stereo over greed based –No early commitment –Enforce interactions: each pixel sees unique item –Penalize interactions: non-smoothness

3. Stereo by Optimization Early algorithms: dynamic programming –(Baker ‘81, Belumeur & Mumford ‘92…) –Don’t generalize beyond 2 camera, single scanline

4. Stereo by Optimization Recent Algorithms: iterative  expansion –(… Kolmogorov & Zabih ‘01) –very general –NP-Complete Local opt found quickly in practice Recent algorithms: MIN-CUT –(Roy & Cox ‘96, Ishikawa & Geiger ‘98) –Polynomial time global optimum –New interpretation to such methods

5. Contributions Stereo as a discrete minimal surface problem Algorithms: Polynomial time globally optimal surface –Using MIN-CUT (Sullivan ‘90) –Build from shortest path Applications to stereo vision –Rederive previous MIN-CUT stereo approaches –New 3-camera stereo formulation (Ayache ‘88)

6. Planar Graph Shortest Path Given: an embedded planar graph –faces, edges, vertices

7. Planar Graph Shortest Path A non negative cost on each edge 57

8. Planar Graph Shortest Path Two boundary points on the exterior of the complex

9. Planar Graph Shortest Path Find minimal curve: (collection of edges) with given boundary

10. Planar Graph For stereo Camera LeftCamera Right Selected Match Selected Occlusion

11. Algorithms Classic: Dijkstra’s –Works even for non-planar graphs Wacky: use duality –But this will generalize to higher dimension

12. Duality

13. face vertex edge cross edge - same cost 57

14. Duality Split exterior

15. Source Sink Source Duality Add source and sink

16. Cuts Source Sink Cuts of dual graph = partitions of dual verts Cost = sum of dual edges spanning the partition MIN-CUT can be found in polynomial time

17. Source Sink Cuts Claim: Primalization of MIN-CUT will be shortest path

18. Sink Source Sink Why this works Cuts of dual graph = partitions of dual verts

19. Sink Source Sink Why this works Partition of dual verts = partition of primal faces

20. Sink Source Sink Source Sink Why this works Partition of primal faces = primal path

21. Sink Source Sink Source Sink Why this works Cuts in dual correspond to paths in primal MIN-CUT in dual corresponds to shortest path in primal

22. Same idea works for surfaces!

23. Increasing the dimension Planar graph: verts, edges, faces cost on edges boundary: 2 points on exterior sol: min path Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface

24. Increasing the dimension Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface

25. Increasing the dimension Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface

26. Dual construction for min surf face vertex edge cross edge Sink Source cell vertex face cross edge MIN-CUT primalizes to min surf

27. Checkpoint Solve for minimal paths and surfaces –MIN-CUT on dual graph Apply these algorithms to stereo vision

28. Flatland Stereo Camera LeftCamera Right Geometric interpretation of Cox et al. 96 pixel

29. Flatland Stereo Camera LeftCamera Right Geometric interpretation of Cox et al. 96 pixel

30. Flatland Stereo Camera LeftCamera Right Cost: unmatched/discontinuity, β

31. Flatland Stereo Camera LeftCamera Right Cost: correspondence quality

32. Flatland Stereo Camera LeftCamera Right

33. Camera LeftCamera Right Match Unmatched Uniqueness & monotonicity solution is directed path Flatland Stereo

34. Camera LeftCamera Right Match Occlusion, discontinuity Note: unmatched pixels also function as discontinuities Flatland Stereo

35. Flatland to Fatland Camera LeftCamera Right

36. Flatland to Fatland Camera LeftCamera Right

37. 2 cameras, 3d

39. One Cuboid Among Many Solve for minimal surface

40. Geometric interpretation IG98

41. Three Camera Rectification (Ayache ‘88)

42. Three Camera

46. One cuboid

47. Dual graph of one cuboid

48. One Cuboid Among Many Solve for minimal surface

49. More divisions of middle cell

50. More expressive decomposition

51. Complexity Vertices and edges: 20 n d –n: pixels per image –d: max disparity Time complexity O((nd) 2 log(nd)) About 1 min

52. Results LL image RC KZ01 MS

53. LL image RC KZ01 MS

54. Future Application of MS to n cameras –Monotonicity/oriented manifold enforces more than uniqueness –see Kolmogorov & Zabih (today 11:00am) Other applications of MS