Graph-Cut Algorithm with Application to Computer Vision Presented by Yongsub Lim Applied Algorithm Laboratory

Contents Image Labeling Problem Maxflow-Mincut Theorem (remind) Energy Minimization Problem –Graph Construction Conclusion Our Related Work Applied Algorithm Laboratory, KAIST 2

Image Labeling Problem We want to identify an object in an image It can be done by labeling a binary value, 0 or 1, to each pixel The goal is to find a labeling minimizing an energy function defined on an image, it is generally NP-Hard Applied Algorithm Laboratory, KAIST 3

Image Labeling Problem Applied Algorithm Laboratory, KAIST 4

Maxflow-Mincut Theorem Given a directed graph Definition (Graph-Cut). Two disjoint subsets of nodes such that s-t cut has a node s in S and t in T Min-cut is a cut minimizing the capacity Applied Algorithm Laboratory, KAIST 5

Maxflow-Mincut Theorem Definition (Flow Network). A directed graph with a source and a sink where an edge has nonnegative capacity Definition (Flow). A real-valued function that satisfies the following constraints –Capacity Constraint: for all –Skew Symmetry: for all –Flow Conservation: for all Applied Algorithm Laboratory, KAIST 6 where s is a source and t is a sink

Maxflow-Mincut Theorem ts 1/2 2/5 0/4 2/2 1/5 0/3 3/3 Applied Algorithm Laboratory, KAIST 7

Maxflow-Mincut Theorem The value of,, is Maxflow-Mincut Theorem. If is a maximum flow, where is a s-t min-cut Thus, we can find a min-cut in graphs by solving the max-flow problem Applied Algorithm Laboratory, KAIST 8

Maxflow-Mincut Theorem s t This line determines min-cut and max- flow, too Applied Algorithm Laboratory, KAIST 9

Energy Minimization Problem To minimize an energy function defined on a pixel-grid graph Definition (Submodular). is a submodular function if Applied Algorithm Laboratory, KAIST 10

Energy Minimization Problem When all pairwise functions ‘s are submodular, we can find a labeling minimizing by a Min-Cut of the graph Applied Algorithm Laboratory, KAIST 11 all of these are submodular

Energy Minimization Problem Applied Algorithm Laboratory, KAIST Min-Cut

Energy Minimization Problem Our goal is to find a binary labeling minimizing an energy function defined on a pixel-grid graph An labeling minimizing an energy function is found using a min-cut By Maxflow-Mincut theorem, we can compute a min-cut using the max-flow algorithm Applied Algorithm Laboratory, KAIST 13

Graph Construction Remaining is how we construct a graph from given an energy function and an image We should define sets of nodes and edges, and weights of edges Applied Algorithm Laboratory, KAIST 14

Graph Construction A set of nodes –Nodes each of which corresponds to each pixel, a source and a sink A set of edges –If two pixels are adjacent, there is a bidirectional edge between corresponding nodes –Edges from a source to all nodes except a sink –Edges from all nodes except a source to a sink Applied Algorithm Laboratory, KAIST 15

Graph Construction Applied Algorithm Laboratory, KAIST 16 s t

Graph Construction Defining weights Assume that there is a two-pixel image Applied Algorithm Laboratory, KAIST 17

Graph Construction Applied Algorithm Laboratory, KAIST 18

Graph Construction Applied Algorithm Laboratory, KAIST 19 s t 1

Graph Construction Applied Algorithm Laboratory, KAIST 20 s t 1 2

Graph Construction Applied Algorithm Laboratory, KAIST 21 3 s t 1 2 4

Graph Construction Applied Algorithm Laboratory, KAIST 22 s t

Graph Construction Applied Algorithm Laboratory, KAIST 23 Note that Now, we can embed all terms of an energy function to a graph as weights

Graph Construction Finally we get the following graph Applied Algorithm Laboratory, KAIST 24 s t cd ab e f

Graph Construction We need two verifications We should verify that the capacity of a s-t min- cut is same with the minimum energy And that all weights are non-negative for the max-flow algorithm Applied Algorithm Laboratory, KAIST 25

Graph Construction Applied Algorithm Laboratory, KAIST 26 s t The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 27 s t The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 28 s t 11 The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 29 s t 11 The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 30 s t The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 31 s t 01 The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Applied Algorithm Laboratory, KAIST 32 s t 01 The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function

Graph Construction Are all weights non-negative? First, weights from unary terms may negative It does not change a minimum labeling even if we add a constant term to the energy function Applied Algorithm Laboratory, KAIST 33

Graph Construction Applied Algorithm Laboratory, KAIST 34 s t cd ab e f b or d is negative

Graph Construction Second, weights from pairwise terms are always non-negative because of the submodularity Applied Algorithm Laboratory, KAIST 35

Graph Construction Applied Algorithm Laboratory, KAIST 36

Graph Construction Applied Algorithm Laboratory, KAIST 37 ∴ All of weights from pairwise terms are always non-negative

Graph Construction All of weights are non-negative We can compute a min-cut through Maxflow- Mincut Theorem And obtain a labeling minimizing an energy function Applied Algorithm Laboratory, KAIST 38 s t cd ab e f

Conclusion Given an energy function defined on an image Minimizing is generally NP-Hard For the class of energy functions whose pairwise functions are all submodular, we can compute a minimum labeling through a graph min-cut in polynomial time Applied Algorithm Laboratory, KAIST 39

Our Related Work We want a minimum labeling with fixed 1’s –Label count = # of 1’s labeled Previous work computes minimum labeling for some label counts Our recent work computes approximate minimum labeling for almost all label counts (submitted to CVPR10) Applied Algorithm Laboratory, KAIST 40

Our Related Work Applied Algorithm Laboratory, KAIST 41 originalNo decomposition 3x3 decomposition 5x5 decomposition Used image is of size 300x300.Results for label count

Our Related Work Applied Algorithm Laboratory, KAIST 42 No decomposition3x3 decomposition5x5 decomposition The average energy value The number of label counts

Q&A Applied Algorithm Laboratory, KAIST 43

Thanks Applied Algorithm Laboratory, KAIST 44