Graph Cut Weizhen Jing 2015.12.8

Image Segmentation methods Threshold based method Edge detecting based method Energy functionals based method Snake model 、Level Set region based method region growing 、split-and-merge Graph based method GraphCut 、GrabCut、Random Walk ......

Introduction Application stereo vision Image matting Image segmentation Minimum cut of Graph

Introduction ► goal:Divide an image into two segments:"background" and "object" . ► pixel label problem Boykov Y, Funka-Lea G. Graph cuts and efficient ND image segmentation[J]. International journal of computer vision, 2006, 70(2): 109-131.

Construct S-T Graph G=<V,E> V=P {S ,T} P : pixel nodes in image s : terminal node, object t : terminal node,background E=n-Links t-Links n-Links : edges between two pixel nodes t-Links: edges between s or t and a pixel node t Wp(t) WP(s) s Wp(t): penalty for assigning pixel p to t(background). Wp(s): penalty for assigning pixel p to s(object). weights on n-links: penalty for a discontinuity between two adjacent nodes

Graph Cut cut a cut C={S,T} s S,t T,S T= , Cost of cut sum of the costs of the edges from S to T Min Cut a cut t s

steps of Graph Cut ► Initialize a binary vector A=(A1,A2,...,AP) (a cut) ► Generate a energy function E(A) ► Minimize E(A),find the min cut

Initialize a binary vector A A=(A1,A2,...,AP) a binary vector which represents certain segmentation each AP can be either 1(obj)or 0(bkg) 1

Initialize a binary vector A ► Automatic Segmentation ● reflect on how the intensity of pixel P fits into a known intensity model(e.g. histogram)of the object and background ► Interactive Segmentation ● Accurately positioned at the desired boundary (easily generalized to 3D data) ● Indicate segmentation regions rather than boundary , some pixel are marked as internal and some as external for the given object of interest. Automatic Segmentation Interactive Segmentation Automatic Segmentation

Energy Founction Boykov Y Y, Jolly M P. Interactive graph cuts for optimal boundary & region segmentation of objects in ND images[C]//Computer Vision, 2001. ICCV 2001. etc. Times Cited: 3118.

(1) E(A) — Energy function global cost of cut A R(A) — regional term B(A) — boundary term — the coefficient specifies a relative important of R(A) versus B(A) A=(A1,A2,...,AP) a binary vector which represents certain segmentation each AP can be either 1(obj)or 0(bkg)

Reginal term (2) RP(AP) : penalty for assigning pixel P to "object"or "background" Rp(obj) = -ln Pr(Ip|obj) Rp(bkg) = -ln Pr(Ip|bkg) RP(.) may reflect on how the intensity of pixel P fits into a known intensity model(e.g. histogram)of the object and background. (2)

Boundary term B{p,q} : penalty for a discontinuity between p and q (AP,Aq) :only consider pixel pairs on the boundary

Weights of edges

Max flow/Min Cut s 16 13 10 a b b 4 12 14 9 c 7 d d 4 20 t ● flow : f(u,v) is a flow from u to v ● max flow : the max flow from s to t ● cut : C={S,T} s S,t T,S T= , ● min cut : sum of the costs of the edges from S to T is minimum ● |f|max=C{S,T} s 16 13 10 a b b 4 12 14 9 c 7 d d 4 20 t

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 16 13 10 a b 4 12 14 9 c 7 d 4 20 t FLOW=0

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 16 13 10 a b 4 12 14 9 c 7 d 4 20 t FLOW=4

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 16-4 13 +4 10 a b 4 12-4 +4 9-4 14-4 +4 +4 c 7 d 4-4 20 +4 t FLOW=4

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 13 4 10 a b 4 8 4 5 10 4 4 c 7 d 20 4 t FLOW=4

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 13 4 10 a b 4 8 4 5 10 4 4 c 7 d 20 4 t FLOW=4

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 13 4 10 a b 4 8 4 5 10 4 4 c 7 d 20 4 t FLOW=11

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12-7 13 4+7 10-7 a b 4+7 8 4 5 10-7 4+7 4 7-7 c d +7 20-7 4 +7 t FLOW=11

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 5 13 11 3 a b 11 8 4 5 3 11 4 c d 7 13 4 7 t FLOW=11

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 5 13 11 3 a b 11 8 4 5 3 11 4 c d 7 13 4 7 t FLOW=11

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 5 13 11 3 a b 11 8 4 5 3 4 c d 7 13 4 7 t FLOW=19

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s +8 5 13-8 11 3+8 a b 11-8 8-8 4+8 5 3 11 4 c d 7 13-8 4 7+8 t FLOW=19

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 8 5 5 11 11 a b 3 12 5 3 11 4 c 7 d 5 4 15 t FLOW=19

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 8 5 5 11 11 a b 3 12 5 3 11 4 c 7 d 5 4 15 t FLOW=19

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 8 5 5 11 11 a b 3 12 5 3 11 4 c 7 d 5 4 15 t FLOW=23

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 8+4 5 5-4 11 11 a b 3 12 5+4 3 11 4-4 c 7 d 5-4 4 15+4 t FLOW=23

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 5 1 11 11 a b 3 12 9 3 11 c 7 d 1 4 19 t FLOW=23

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 5 1 11 11 a b 3 12 9 3 11 c 7 d 1 4 19 t FLOW=23

Ford & Fulkerson algorithm Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the augmenting path in the residual graph End s 12 5 1 11 11 a b 3 12 9 3 11 c 7 d 1 4 19 t FLOW=23

Max flow/Min Cut DFS Gf from S The min cut : S={a,b,d},T={c} s b a 12 5 1 11 DFS Gf from S The min cut : S={a,b,d},T={c} 11 a b 3 12 9 3 11 c 7 d 1 4 19 t FLOW = 23

Image Segmentation

Conclusion Advantage Disadvantage ●Robust and Efficient Optimization in N-D images ●global optimal ,better Segmentation ●reginal term and boundary term,better Segmentation ●Suitable for color image segmentation ●Simple and effective user interaction Disadvantage ●gray value of background and object are similar ●background and object model are grey level histogram

Thank you!