Corp. Research Princeton, NJ Computing geodesics and minimal surfaces via graph cuts Yuri Boykov, Siemens Research, Princeton, NJ joint work with Vladimir Kolmogorov, Cornell University, Ithaca, NY

Corp. Research Princeton, NJ Two standard object extraction methods Interactive Graph cuts [Boykov&Jolly ‘01] Discrete formulation Computes min-cuts on N-D grid-graphs Geodesic active contours [Caselles et.al. ‘97, Yezzi et.al ‘97] Continuous formulation Computes geodesics in image- based N-D Riemannian spaces Geo-cuts Minimal geometric artifacts Solved via local variational technique (level sets) Possible metrication errors Global minima

Corp. Research Princeton, NJ Geodesics and minimal surfaces n The shortest curve between two points is a geodesic Riemannian metric (space varying, tensor D(p)) n Geodesic contours use image-based Riemannian metric Euclidian metric (constant) A B A B n Generalizes to 3D (minimal surfaces) distance map distance map

Corp. Research Princeton, NJ Graph cuts (simple example à la Boykov&Jolly, ICCV’01) n-links s t a cut hard constraint hard constraint Minimum cost cut can be computed in polynomial time (max-flow/min-cut algorithms)

Corp. Research Princeton, NJ Metrication errors on graphs discrete metric ??? Minimum cost cut (standard 4-neighborhoods) Continuous metric space (no geometric artifacts!) Minimum length geodesic contour (image-based Riemannian metric)

Corp. Research Princeton, NJ Cut Metrics : cuts impose metric properties on graphs C n Cut metric is determined by the graph topology and by edge weights. n Can a cut metric approximate a given Riemannian metric? n Cost of a cut can be interpreted as a geometric “length” (in 2D) or “area” (in 3D) of the corresponding contour/surface.

Corp. Research Princeton, NJ Our key technical result n The main technical problem is solved via Cauchy-Crofton formula from integral geometry. We show how to build a grid-graph such that its cut metric approximates any given Riemannian metric

Corp. Research Princeton, NJ Integral Geometry and Cauchy-Crofton formula C Euclidean length of C : the number of times line L intersects C a set of all lines L a subset of lines L intersecting contour C

Corp. Research Princeton, NJ Cut Metric on grids can approximate Euclidean Metric C Euclidean length graph cut cost for edge weights: the number of edges of family k intersecting C Edges of any regular neighborhood system generate families of lines {,,, } Graph nodes are imbedded in R2 in a grid-like fashion

Corp. Research Princeton, NJ Cut metric in Euclidean case “standard” 4-neighborhoods (Manhattan metric) 256-neighborhoods8-neighborhoods n “Distance maps” (graph nodes “equidistant” from a given node) : n (Positive!) weights depend only on edge direction k.

Corp. Research Princeton, NJ Reducing Metrication Artifacts original noisy image Image restoration [BVZ 1999] restoration with “standard” 4-neighborhoods restoration with 8-neighborhoods using edge weights

Corp. Research Princeton, NJ Cut Metric in Riemannian case n The same technique can used to compute edge weights that approximate arbitrary Riemannian metric defined by tensor D(p) Idea: generalize Cauchy-Crofton formula 4-neighborhoods8-neighborhoods 256-neighborhoods n Local “distance maps” assuming anisotropic D(p) = const n (Positive!) weights depend on edge direction k and on location/pixel p.

Corp. Research Princeton, NJ Convergence theorem Theorem: For edge weights set by tensor D(p) C

Corp. Research Princeton, NJ “Geo-Cuts” algorithm image-derived Riemannian metric D(p) regular grid edge weights Boundary conditions (hard/soft constraints) Global optimization Graph-cuts [Boykov&Jolly, ICCV’01] min-cut = geodesic

Corp. Research Princeton, NJ Minimal surfaces in image induced Riemannian metric spaces (3D) 3D bone segmentation (real time screen capture)

Corp. Research Princeton, NJ Our results reveal a relation between… Level Sets Graph Cuts [Osher&Sethian’88,…] [Greig et. al.’89, Ishikawa et. al.’98, BVZ’98,…] Gradient descent method VS. Global minimization tool variational optimization method for combinatorial optimization for fairly general continuous energies a restricted class of energies [e.g. KZ’02] finds a local minimum finds a global minimum near given initial solution for a given set of boundary conditions anisotropic metrics are harder anisotropic Riemannian metrics to deal with (e.g. slower) are as easy as isotropic ones numerical stability has to be carefully addressed [Osher&Sethian’88]: continuous formulation -> “finite differences” numerical stability is not an issue discrete formulation ->min-cut algorithms (restricted class of energies)

Corp. Research Princeton, NJ Conclusions n “Geo-cuts” combines geodesic contours and graph cuts. The method can be used as a “global” alternative to variational level-sets. n Reduction of metrication errors in existing graph cut methods stereo [Roy&Cox’98, Ishikawa&Geiger’98, Boykov&Veksler&Zabih’98, ….] image restoration/segmentation [Greig’86, Wu&Leahy’97,Shi&Malik’98,…] texture synthesis [Kwatra/et.al’03] n Theoretical connection between discrete geometry of graph cuts and concepts of integral & differential geometry

Corp. Research Princeton, NJ Geo-cuts (more examples) 3D segmentation (time-lapsed)