Graph Cuts Marc Niethammer

Segmentation by Graph-Cuts A way to compute solutions to the optimization problems we looked at before. Example: Binary Segmentation Images from ECCV Tutorial, Kumar/Kohli Need to partition the picture into foreground and background.

Segmentation Graph G=(V,E)‏ Images from ECCV Tutorial, Kumar/Kohli But there is much more to graph cuts than binary segmentation(as we will see later)‏ Approach: Interpret the image as a graph, where pixels are connected to its neighbors. Goal: Cut the graph into pieces to obtain the desired image partition. Assign a label to every pixel.

Segmentation Optimization Problem: Looking at it on a pixel-by-pixel basis (where f is the labeling): Problem: We cannot try all possible pairings for the labels f. Need an efficient algorithm to solve this problem.

Maximum Flow and Minimum Cut Solution: Borrow algorithms for network flows and use them on images. Preview: Graph structure for binary labeling Represent the image such that pixels are connected to neighbors (pairwise interaction cost) as well as to the source and a target vertices (data cost). Images: Boykov The cut that separates source (s) from sink (t) gives the segmentation.

Max-Flow/Min-Cut Theorem For any network having a single origin and a single destination node, the maximum possible flow from origin to destination equals the minimum cut value for all the cuts in the network. If we want to find the minimum cut, we can compute the maximum flow and look for the cut(s) that separate origin and destination by cutting through bottlenecks of the network.

Example: Ford & Fulkerson The Ford and Fulkerson algorithm is one of many possibilities to compute the maximal flow.

Example: Ford & Fulkerson

Cost of min-cut is 4. Divides the nodes (pixels) into two groups. Which equals the maximum flow.

Example: Ford & Fulkerson This is where everything flows. Flow is conserved at nodes. Look at the difference between initial and final capacities. Ignore negative capacities.

Why to update backward flow? Image: John Chinneck Can undo flows. Final flow does not include the central edge.

Image Denoising This approach allows us (amongst many things) to compute binary segmentations. Image: Boykov OriginalNoisyReconstructed

Multi-Label Case Image: Boykov

Multi-Label Case Image: Boykov

Graph Cut Properties Combinatorial optimization for a restricted class of energies Globally optimal (for certain problem classes)‏ Discrete formulation with combinatorial optimization (compare with continuous variational approaches)‏ Problem: Large memory consumption, in particular in 3D.

Metrication Errors Minimum cost cut (standard 4-neighborhoods)‏ Minimum length geodesic contour (image-based Riemannian metric)‏ Images: Boykov Can choose different weighted neighborhood to reduce metrication errors. Also, continuous maximal flows exist.

Interactive Segmentations Images: Boykov

Continuous Maximum Flow Continuous version of maximum flow [Appleton]. Results in a PDE and can be solved as such. Energy to be minimized: Play the usual trick of introducing the auxiliary variable p, which we maximize for: Now solve this by gradient descent.

Continuous Maximum Flow The variation is The gradient descent scheme becomes

Segmentation with Continuous Max-Flow Images: Unger Thresholding Continuous max-flow with seeds.

Bone Segmentation Also works nicely in 3D.