1. Graphical Models and Learning
Lecture 3: Maximum Flow, Minimum Cut
Yuliya Tarabalka
Slides courtesy of Pawan Kumar

2. Outline Preliminaries Maximum Flow Algorithms
Functions and Excess Functions
s-t Flow
s-t Cut
Flows vs. Cuts
Maximum Flow
Algorithms
Energy minimization with max flow/min cut

3. Context
Example: network optimization problems

4. Maximum flow problem Applications
Maximize the flow through a company’s distribution network from factories to customers
Maximize the flow of oil through a system of pipelines
Maximize the flow of vehicles through a transportation network

5. Functions on Arcs D = (V, A) s 1 1 10 8 Arc capacities c(a) 4 6 v1 v2
Function f: A  Reals 3 3 2
Excess function Ef(v)
Incoming value -
Outgoing value v3 v4 5 7 3 3 t

6. Functions on Arcs D = (V, A) s 1 1 10 8 Arc capacities c(a) 4 6 v1 v2
Function f: A  Reals 3 3 2
Excess function Ef(v)
Σain-arcs(v) f(a) -
Outgoing value v3 v4 5 7 3 3 t

7. Functions on Arcs D = (V, A) s 1 1 10 8 Arc capacities c(a) 4 6 v1 v2
Function f: A  Reals 3 3 2
Excess function Ef(v)
Σain-arcs(v) f(a) -
Σaout-arcs(v) f(a) v3 v4 5 7 3 3 t

8. Functions on Arcs D = (V, A) s 1 1 10 8 Arc capacities c(a) 4 6 v1 v2
Function f: A  Reals 3 3 2
Excess function Ef(v)
f(in-arcs(v)) -
f(out-arcs(v)) v3 v4 5 7 3 3 t
Ef(v1) -6

9. Functions on Arcs D = (V, A) s 1 1 10 8 Arc capacities c(a) 4 6 v1 v2
Function f: A  Reals 3 3 2
Excess function Ef(v)
f(in-arcs(v)) -
f(out-arcs(v)) v3 v4 5 7 3 3 t
Ef(v2) 14

10. Excess Functions of Vertex Subsets
Excess function Ef(U) 1 1 10 8
Incoming Value -
Outgoing Value 4 6 v1 v2 3 3 2 v3 v4 5 7 3 3 t

11. Excess Functions of Vertex Subsets
Excess function Ef(U) 1 1 10 8
Σain-arcs(U) f(a) -
Outgoing Value 4 6 v1 v2 3 3 2 v3 v4 5 7 3 3 t

12. Excess Functions of Vertex Subsets
Excess function Ef(U) 1 1 10 8
Σain-arcs(U) f(a) -
Σaout-arcs(U) f(a) 4 6 v1 v2 3 3 2 v3 v4 5 7 3 3 t

13. Excess Functions of Vertex Subsets
Excess function Ef(U) 1 1 10 8
f(in-arcs(U)) -
f(out-arcs(U)) 4 6 v1 v2 3 3 2
Ef({v1,v2}) 8 v3 v4 5 7 3 3 t

14. Excess Functions of Vertex Subsets
Excess function Ef(U) 1 1 10 8
f(in-arcs(U)) -
f(out-arcs(U)) 4 6 v1 v2 3 3 2
Ef({v1,v2}) v3 v4 5
Ef(U) = ΣvU Ef(v) 7 3 3 t

15. Outline Preliminaries Maximum Flow Algorithms
Functions and Excess Functions
s-t Flow
s-t Cut
Flows vs. Cuts
Maximum Flow
Algorithms
Energy minimization with max flow/min cut

16. s-t Flow s Function flow: A  R 1 8 Flow of arc ≤ arc capacity 6 v1 v2
Flow is non-negative 3 2
For all vertex except s,t v3 v4
Incoming flow 5
= Outgoing flow 7 3 t

17. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2
Flow is non-negative 3 2
For all vertex except s,t v3 v4
Incoming flow 5
= Outgoing flow 7 3 t

18. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2 flow(a) ≥ 03 2
For all vertex except s,t v3 v4
Incoming flow 5
= Outgoing flow 7 3 t

19. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2 flow(a) ≥ 03 2
For all v  V \ {s,t} v3 v4
Incoming flow 5
= Outgoing flow 7 3 t

20. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2 flow(a) ≥ 03 2
For all v  V \ {s,t} v3 v4
Σ(u,v)A flow((u,v)) 5
= Outgoing flow 7 3 t

21. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2 flow(a) ≥ 03 2
For all v  V \ {s,t} v3 v4
Σ(u,v)A flow((u,v)) 5
= Σ(v,u)A flow((v,u)) 7 3 t

22. s-t Flow s Function flow: A  R 1 8 flow(a) ≤ c(a) 6 v1 v2 flow(a) ≥ 03 2
For all v  V \ {s,t} v3 v4
Eflow(v) = 0 5 7 3 t

23. ✗ s-t Flow s Function flow: A  R 1 1 10 8 4 flow(a) ≤ c(a) 6 v1 v23 3 2
For all v  V \ {s,t} v3 v4
Eflow(v) = 0 5 7 3 3 ✗ t

24. ✗ s-t Flow s Function flow: A  R -1 1 8 flow(a) ≤ c(a) 6 v1 v23 2
For all v  V \ {s,t} v3 v4
Eflow(v) = 0 5 7 -1 3 ✗ t

25. ✓ s-t Flow s Function flow: A  R 1 1 8 flow(a) ≤ c(a) 6 v1 v23 2
For all v  V \ {s,t} v3 v4
Eflow(v) = 0 5 7 1 3 ✓ t

26. Value of s-t Flow s 1 8 6 v1 v2 Outgoing flow of s 3 2
- Incoming flow of s v3 v4 5 7 3 t

27. Value of s-t Flow s -Eflow(s) Eflow(t) 1 8 6 v1 v2
Σ(s,v)A flow((s,v)) 3 2
- Σ(u,s)A flow((u,s)) v3 v4 5 7 3 t

28. Value of s-t Flow s -Eflow(s) Eflow(t) 1 1 8 6 v1 v2
Σ(s,v)A flow((s,v)) 1 3 2
- Σ(u,s)A flow((u,s)) v3 v4
Value = 1 5 7 1 3 t

29. Outline Preliminaries Maximum Flow Algorithms
Functions and Excess Functions
s-t Flow
s-t Cut
Flows vs. Cuts
Maximum Flow
Algorithms
Energy minimization with max flow/min cut

30. Cut D = (V, A) Let U be a subset of V 10 v1 v2 3 2
C is a set of arcs such that
(u,v)  A
u  U
v  V\U v3 v4 5
C is a cut in the digraph D

31. ✓ Cut D = (V, A) U What is C? 10 v1 v2 {(v1,v2),(v1,v4)} ? 3 25
V\U

32. ✓ Cut D = (V, A) V\U U What is C? 10 v1 v2 {(v1,v2),(v1,v4),(v3,v2)} ?5

33. ✓ Cut D = (V, A) U V\U What is C? 10 v1 v2 {(v1,v2),(v1,v4),(v3,v2)} ?5

34. Cut
D = (V, A) 10 v1 v2 3 2
C = out-arcs(U) v3 v4 5

35. Capacity of Cut 10 v1 v2 3 2
Sum of capacity of all
arcs in C v3 v4 5

36. Capacity of Cut 10 v1 v2 3 2
Σa  C c(a) v3 v4 5

37. Capacity of Cut U 10 v1 v2 3 3 2 v3 v4 5
V\U

38. Capacity of Cut U
V\U 10 v1 v2 15 3 2 v3 v4 5

39. s-t Cut s D = (V, A) 1 8 A source vertex “s” 6 v1 v2 A sink vertex “t”3 2
C is a cut such that
s  U
t  V\U v3 v4 5 7 3
C is an s-t cut t

40. Capacity of s-t Cut s 1 8 6 v1 v2
Σa  C c(a) 3 2 v3 v4 5 7 3 t

41. Capacity of s-t Cut s 1 8 6 v1 v2 5 3 2 v3 v4 5 7 3 t

42. Capacity of s-t Cut s 1 8 6 v1 v2 17 3 2 v3 v4 5 7 3 t

43. Outline Preliminaries Maximum Flow Algorithms
Functions and Excess Functions
s-t Flow
s-t Cut
Flows vs. Cuts
Maximum Flow
Algorithms
Energy minimization with max flow/min cut

44. Flows vs. Cuts An s-t flow function : A  Reals
An s-t cut C such that s  U, t  V\U
Value of flow ≤ Capacity of C

45. Flows vs. Cuts Value of flow = -Eflow(s)
= -Eflow(s) - ΣvU\{s} Eflow(v)
= -Eflow(U)
= flow(out-arcs(U))
- flow(in-arcs(U))
≤ Capacity of C
- flow(in-arcs(U))

46. Flows vs. Cuts Value of flow = -Eflow(s)
= -Eflow(s) - ΣvU\{s} Eflow(v)
= -Eflow(U)
= flow(out-arcs(U))
- flow(in-arcs(U))
≤ Capacity of C
When does equality hold?

47. Flows vs. Cuts Value of flow = -Eflow(s)
= -Eflow(s) - ΣvU\{s} Eflow(v)
= -Eflow(U)
= flow(out-arcs(U))
- flow(in-arcs(U))
≤ Capacity of C
flow(a) = c(a), a  out-arcs(U)
flow(a) = 0, a  in-arcs(U)

48. Flows vs. Cuts Value of flow = -Eflow(s)
= -Eflow(s) - ΣvU\{s} Eflow(v)
= -Eflow(U)
= flow(out-arcs(U))
- flow(in-arcs(U))
= Capacity of C
flow(a) = c(a), a  out-arcs(U)
flow(a) = 0, a  in-arcs(U)

49. Outline Preliminaries Maximum Flow Algorithms
Residual Graph
Max-Flow Min-Cut Theorem
Algorithms
Energy minimization with max flow/min cut

50. Maximum Flow Problem s Find the flow with the maximum value !! 2 4 3
Σ(s,v)A flow((s,v)) 1 2
- Σ(u,s)A flow((u,s)) t
First suggestion to solve this problem !!

51. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 2 4 3 v1 v2 1 2 t

52. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 1 2 4 3 v1 v2
Pass maximum allowable
flow through the arcs 1 1 2 t

53. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 1 2 4 3 v1 v2 1 1 2 t

54. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 1 2 4 2 3 v1 v2
Pass maximum allowable
flow through the arcs 1 1 2 2 t

55. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 1 2 4 2 3 v1 v2
No more paths.
Stop. 1 1 2 2 t
Will this give us maximum flow?
NO !!!

56. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 2 2 4 2 3 v1 v2
Pass maximum allowable
flow through the arcs 1 2 2 t

57. Passing Flow through s-t Paths
Find an s-t path where
flow(a) < c(a) for all arcs 2 2 4 2 3 v1 v2
No more paths.
Stop. 1 2 2 t
Another method?
Incorrect Answer !!

58. Outline Preliminaries Maximum Flow Algorithms
Residual Graph
Max-Flow Min-Cut Theorem
Algorithms
Energy minimization with max flow/min cut

59. Residual Graph s s 2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Arcs where flow(a) < c(a)

60. Residual Graph s s 2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Including arcs to s and from t is not necessary
Inverse of arcs where flow(a) > 0

61. Maximum Flow using Residual Graphs2 4 3 v1 v2 v1 v2 1 2 t t
Start with zero flow.

62. Maximum Flow using Residual Graphs2 4 3 v1 v2 v1 v2 1 2 t t
Find an s-t path in the residual graph.

63. Maximum Flow using Residual Graphs2 4 3 v1 v2 v1 v2 1 2 t t
Find an s-t path in the residual graph.

64. Maximum Flow using Residual Graphs2 4 3 v1 v2 v1 v2 1 2 t t
For inverse arcs in path, subtract flow K.

65. Maximum Flow using Residual Graphs2 4 3 v1 v2 v1 v2 1 2 t t
Choose maximum allowable value of K.
For forward arcs in path, add flow K.

66. Maximum Flow using Residual Graphs2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Choose maximum allowable value of K.
For forward arcs in path, add flow K.

67. Maximum Flow using Residual Graphs2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Update the residual graph.

68. Maximum Flow using Residual Graphs2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Find an s-t path in the residual graph.

69. Maximum Flow using Residual Graphs2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Find an s-t path in the residual graph.

70. Maximum Flow using Residual Graphs2 2 4 2 3 v1 v2 v1 v2 1 2 2 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v1,t). Subtract K from (v1,v2).

71. Maximum Flow using Residual Graphs2 2 4 1 1 3 v1 v2 v1 v2 1 1 2 2 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v1,t). Subtract K from (v1,v2).

72. Maximum Flow using Residual Graphs2 2 4 1 1 3 v1 v2 v1 v2 1 1 2 2 t t
Update the residual graph.

73. Maximum Flow using Residual Graphs2 2 4 1 1 3 v1 v2 v1 v2 1 1 2 2 t t
Find an s-t path in the residual graph.

74. Maximum Flow using Residual Graphs2 2 4 1 1 3 v1 v2 v1 v2 1 1 2 2 t t
No more s-t paths. Stop.

75. Maximum Flow using Residual Graphs2 2 4 1 1 3 v1 v2 v1 v2 1 1 2 2 t t
Correct Answer.

76. Maximum Flow using Residual Graphs1 4 3 v1 v2 v1 v2 2 2 t t
Start with zero flow

77. Maximum Flow using Residual Graphs1 4 3 v1 v2 v1 v2 2 2 t t
Find an s-t path in the residual graph.

78. Maximum Flow using Residual Graphs1 4 3 v1 v2 v1 v2 2 2 t t
Find an s-t path in the residual graph.

79. Maximum Flow using Residual Graphs1 4 3 v1 v2 v1 v2 2 2 t t
For inverse arcs in path, subtract flow K.

80. Maximum Flow using Residual Graphs1 4 3 v1 v2 v1 v2 2 2 t t
Choose maximum allowable value of K.
For forward arcs in path, add flow K.

81. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 2 2 1 t t
Choose maximum allowable value of K.
For forward arcs in path, add flow K.

82. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 2 2 1 t t
Update the residual graph.

83. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 2 2 1 t t
Find an s-t path in the residual graph.

84. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 2 2 1 t t
Find an s-t path in the residual graph.

85. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 2 2 1 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v1,t). Subtract K from (v1,v2).

86. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 1 2 2 1 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v1,t). Subtract K from (v1,v2).

87. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 1 2 2 1 t t
Update the residual graph.

88. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 1 2 2 1 t t
Find an s-t path in the residual graph.

89. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 1 2 2 1 t t
Find an s-t path in the residual graph.

90. Maximum Flow using Residual Graphs1 1 4 1 3 v1 v2 v1 v2 1 2 2 1 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v2,t).

91. Maximum Flow using Residual Graphs1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
Choose maximum allowable value of K.
Add K to (s,v2) and (v2,t).

92. Maximum Flow using Residual Graphs1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
Update residual graph.

93. Maximum Flow using Residual Graphs1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
No more s-t paths. Stop.

94. Maximum Flow using Residual Graphs1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
How can I be sure this will always work?

95. Outline Preliminaries Maximum Flow Algorithms
Residual Graph
Max-Flow Min-Cut Theorem
Algorithms
Energy minimization with max flow/min cut

96. Let the subset of vertices U be reachable from s.
Max-Flow Min-Cut s s 1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
t is not in U.
Let the subset of vertices U be reachable from s.

97. Max-Flow Min-Cut s s 1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
Or else a will be in the residual graph.
For all a  out-arcs(U), flow(a) = c(a).

98. Max-Flow Min-Cut s s 1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
Or else inverse of a will be in the residual graph.
For all a  in-arcs(U), flow(a) = 0.

99. Max-Flow Min-Cut s s 1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
For all a  out-arcs(U), flow(a) = c(a).
For all a  in-arcs(U), flow(a) = 0.

100. Flows vs. Cuts Value of flow = -Eflow(s)
= -Eflow(s) - ΣvU\{s} Eflow(v)
= -Eflow(U)
= flow(out-arcs(U))
- flow(in-arcs(U))
= Capacity of C
flow(a) = c(a), a  out-arcs(U)
flow(a) = 0, a  in-arcs(U)

101. Capacity(C) = Value(flow)
Max-Flow Min-Cut s s 1 1 4 2 3 v1 v2 v1 v2 1 2 2 2 t t
Minimum Cut
Maximum Flow
Capacity(C) = Value(flow)

102. Outline Preliminaries Maximum Flow Algorithms
Ford-Fulkerson Algorithm
Dinits Algorithm
Energy minimization with max flow/min cut

103. Ford-Fulkerson Algorithm
Start with flow = 0 for all arcs.
Find an s-t path in the residual graph.
Pass maximum allowable flow.
REPEAT
Subtract from inverse arcs.
Add to forward arcs.
Until s and t are disjoint in the residual graph.

104. Ford-Fulkerson Algorithm1 v1 v2 v1 v2
10k
10k t t
Start with zero flow

105. Ford-Fulkerson Algorithm1 v1 v2 v1 v2
10k
10k t t
Find an s-t path in the residual graph.

106. Ford-Fulkerson Algorithm1 v1 v2 v1 v2
10k
10k t t
Find an s-t path in the residual graph.

107. Ford-Fulkerson Algorithm1 v1 v2 v1 v2
10k
10k t t
Pass the maximum allowable flow.

108. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Pass the maximum allowable flow.

109. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Update the residual graph.

110. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Find an s-t path in the residual graph.

111. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Find an s-t path in the residual graph.

112. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Complexity is exponential in k.

113. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
For examples, see Uri Zwick, 1993
Irrational arc lengths can lead to infinite iterations.

114. Ford-Fulkerson Algorithm1
10k
10k 1 1 v1 v2 v1 v2
10k
10k t 1 t
Choose wisely.
There are good paths and bad paths.

115. Outline Preliminaries Maximum Flow Algorithms
Ford-Fulkerson Algorithm
Dinits Algorithm
Energy minimization with max flow/min cut

116. Dinits Algorithm Start with flow = 0 for all arcs.
Find the minimum s-t path
in the residual graph.
Pass maximum allowable flow.
REPEAT
Subtract from inverse arcs.
Add to forward arcs.
Until s and t are disjoint in the residual graph.

117. Dinits Algorithm s s 10k 10k 1 v1 v2 v1 v2 10k 10k t t
Start with zero flow

118. Find the minimum s-t path in the residual graph.
Dinits Algorithm s s
10k
10k 1 v1 v2 v1 v2
10k
10k t t
Find the minimum s-t path in the residual graph.

119. Find the minimum s-t path in the residual graph.
Dinits Algorithm s s
10k
10k 1 v1 v2 v1 v2
10k
10k t t
Find the minimum s-t path in the residual graph.

120. Pass the maximum allowable flow.
Dinits Algorithm s s
10k
10k 1 v1 v2 v1 v2
10k
10k t t
Pass the maximum allowable flow.

121. Pass the maximum allowable flow.
Dinits Algorithm s s
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t t
Pass the maximum allowable flow.

122. Update the residual graph.
Dinits Algorithm s s
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t t
Update the residual graph.

123. Find the minimum s-t path in the residual graph.
Dinits Algorithm s s
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t t
Find the minimum s-t path in the residual graph.

124. Find the minimum s-t path in the residual graph.
Dinits Algorithm s s
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t t
Find the minimum s-t path in the residual graph.

125. Pass the maximum allowable flow.
Dinits Algorithm s s
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t t
Pass the maximum allowable flow.

126. Pass the maximum allowable flow.
Dinits Algorithm s s
10k
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t
10k t
Pass the maximum allowable flow.

127. Update the residual graph.
Dinits Algorithm s s
10k
10k
10k
10k 1 v1 v2 v1 v2
10k
10k
10k t
10k t
Update the residual graph.

128. Dinits Algorithm s s 10k 10k 10k 10k 1 v1 v2 v1 v2 10k 10k 10k t 10k t
No more s-t paths. Stop.

129. Solvers for the Minimum-Cut Problem
Augmenting Path and Push-Relabel
n: #nodes
m: #arcs
U: maximum
arc length
[Slide credit: Andrew Goldberg]

130. Max-Flow in Computer Vision
Specialized algorithms for vision problems
Grid graphs
Low connectivity (m ~ O(n))
Dual search tree augmenting path algorithm
[Boykov and Kolmogorov PAMI 2004]
Finds approximate shortest augmenting paths efficiently
High worst-case time complexity
Empirically outperforms other algorithms on vision problems
Efficient code available on the web

131. Outline Preliminaries Maximum Flow Algorithms
Energy minimization with max flow/min cut
Two-Label Energy Functions

132. Interactive Binary Segmentation
Foreground histogram of RGB values FG
Background histogram of RGB values BG
‘1’ indicates foreground and ‘0’ indicates background

133. Interactive Binary Segmentation
More likely to be foreground than background

134. Interactive Binary Segmentation
θp(0) proportional to -log(BG(da))
θp(1) proportional to -log(FG(da))
More likely to be background than foreground

135. Interactive Binary Segmentation
More likely to belong to same label

136. Interactive Binary Segmentation
θpq(i,k) proportional to exp(-(da-db)2) if i ≠ k
θpq(i,k) = 0 if i = k
Less likely to belong to same label

137. Overview Energy E Digraph D One vertex per random variable
+ Additional vertices “s” and “t”
Compute
Minimum
Cut
vp  U implies xp = 0
Labeling x
U and V\U
vp  V\U implies xp = 1

138. Digraph for Unary PotentialsB
θp(1)
xp = 0
θp(0)
xp = 1 Xp

139. Digraph for Unary PotentialsB s
xp = 0
xp = 1 vp t

140. Digraph for Unary PotentialsB
Let A ≥ B s
xp = 0
xp = 1 vp
Constant
A-B
A-B B t +

141. Digraph for Unary PotentialsB
Let A ≥ B s
xp = 0
xp = 1
xp = 1 vp
Constant
A-B
A-B B t +

142. Digraph for Unary PotentialsB
Let A ≥ B s
xp = 0
xp = 1
xp = 0 vp
Constant
A-B
A-B
A-B B t +

143. Digraph for Unary PotentialsB
Let A < B s
xp = 0
B-A
xp = 1 vp
Constant
B-A A t +

144. Digraph for Unary PotentialsB
Let A < B s
xp = 0
B-A
xp = 1
xp = 1 vp
Constant
B-A
B-A A t +

145. Digraph for Unary PotentialsB
Let A < B s
xp = 0
B-A
xp = 1
xp = 0 vp
Constant
B-A A t +

146. Digraph for Pairwise Potentials
xp = 0
xp = 1
θpq(1,1) A C B D
xq = 0
θpq(0,1)
θpq(1,0)
xq = 1
θpq(0,0) Xp Xq A
B-A
D-B
C+B-D-A + + +

147. Digraph for Pairwise Potentials
xp = 0
xp = 1 A C B D
xq = 0 s
xq = 1
Constant vp vq A t
B-A
D-B
C+B-D-A + + +

148. Digraph for Pairwise Potentials
xp = 0
xp = 1 A C B D
xq = 0 s
B-A
xq = 1 vp vq
Unary Potential
xq = 1 t
B-A
D-B
C+B-D-A + +

149. Digraph for Pairwise Potentials
xp = 0
xp = 1 A C B D
xq = 0 s
D-B
B-A
xq = 1 vp vq t
Unary Potential
xp = 1
D-B
C+B-D-A +

150. Digraph for Pairwise Potentials
xp = 0
xp = 1 A C B D
xq = 0 s
D-B
B-A
xq = 1
C+B-D-A vp vq t
Pairwise Potential
xp = 1, xq = 0
C+B-D-A

151. Digraph for Pairwise Potentials
xp = 0
xp = 1 A C B D
xq = 0 s
D-B
B-A
xq = 1
C+B-D-A vp vq t
C+B-D-A ≥ 0
Submodular Energy
General 2-label MAP estimation is NP-hard

152. Results – Image Segmentation
Boykov and Jolly, ICCV 2001

153. Results – Image Segmentation
Boykov and Jolly, ICCV 2001

154. Results – Image Segmentation
Boykov and Jolly, ICCV 2001

155. Results – Image Synthesis
Kwatra et al., SIGGRAPH 2003

156. Results – Image Synthesis
Kwatra et al., SIGGRAPH 2003

157. Outline Preliminaries Maximum Flow Algorithms
Energy minimization with max flow/min cut
Multi-Label Energy Functions

158. St-mincut based Move algorithms
E(x) = ∑ θi (xi) + ∑ θij (xi,xj)
i,j i
x ϵ Labels L = {l1, l2, … , lk}
Commonly used for solving non-submodular multi-label problems
Extremely efficient and produce good solutions
Not Exact: Produce local optima

159. Move-Making Algorithms
Space of All Labelings x

160. Move Making Algorithms
Energy
Solution Space

161. Move Making Algorithms
Current Solution
Search Neighbourhood
Optimal Move
Energy
Solution Space

162. Computing the Optimal Move
Current Solution
Search Neighbourhood
Optimal Move xc
(t)
Key Property
Move Space
Energy
Solution Space
Better solutions
Finding the optimal move hard
Bigger move space

163. Moves using Graph Cuts Expansion and Swap move algorithms
[Boykov Veksler and Zabih, PAMI 2001]
Makes a series of changes to the solution (moves)
Each move results in a solution with smaller energy
Current Solution
Search Neighbourhood N
Number of Variables
Move Space (t) : 2N L
Number of Labels
Space of Solutions (x) : LN

164. Moves using Graph Cuts Expansion and Swap move algorithms
[Boykov Veksler and Zabih, PAMI 2001]
Makes a series of changes to the solution (moves)
Each move results in a solution with smaller energy
Current Solution
Move to new solution
How to minimize move functions?
Construct a move function
Minimize move function to get optimal move

165. Expansion Algorithm Variables take label lα or retain current label
Slide courtesy Pushmeet Kohli

166. Expansion Algorithm Variables take label lα or retain current label
Tree
Ground
House
Status:
Expand Sky
Expand House
Initialize with Tree
Expand Ground
Sky
Slide courtesy Pushmeet Kohli

167. Expansion Algorithm
Initialize labeling x = x0 (say x0p = 0, for all Xp)
For α = 1, 2, … , h-1
xα = argminx’ E(x’)
s.t. x’p  {xp} U {lα}
Repeat
until
convergence
Update x = xα
End
Boykov, Veksler and Zabih, 2001

168. θij (la,lb) + θij (lb,lc) ≥ θij (la,lc)
Expansion Algorithm
Restriction on pairwise potentials?
Move energy is submodular if:
Unary Potentials: Arbitrary
Pairwise potentials: Metric
θij (la,lb) + θij (lb,lc) ≥ θij (la,lc)
Example: Potts model
[Boykov, Veksler, Zabih]

169. Swap Move Variables labeled α, β can swap their labels
Chi square
[Boykov, Veksler, Zabih]

170. Swap Move Variables labeled α, β can swap their labels Swap Sky, House
Tree
Ground
Swap Sky, House
House
Sky
[Boykov, Veksler, Zabih]

171. Swap Move Move energy is submodular if: Unary Potentials: Arbitrary
Variables labeled α, β can swap their labels
Move energy is submodular if:
Unary Potentials: Arbitrary
Pairwise potentials: Semimetric
θij (la,lb) ≥ 0
θij (la,lb) = a = b
Chi square
Example: Potts model
[Boykov, Veksler, Zabih]

172. Minimize over move variables t
General Binary Moves
x = t x1 + (1-t) x2
New solution
First solution
Second solution
Minimize over move variables t
Move Type
First Solution
Second Solution
Guarantee
Expansion
Old solution
All alpha
Metric
Fusion
Any solution 

173. Solving Continuous Problems using Fusion Move
x = t x1 + (1-t) x2
Optical Flow
Example
Solution from Method 2 x2
Solution from Method 1 F
Final Solution x1 x
(Lempitsky et al. CVPR08, Woodford et al. CVPR08)

174. Results – Denoising + Inpainting
Label = intensity value of the image

175. Results – Denoising + Inpainting

176. Results – Denoising + Inpainting

177. Results – Denoising + Inpainting

178. Paper presentation
Tarabalka et al., “Spatio-temporal video segmentation with shape growth or shrinkage constraint”, Transactions on Image Processing 2014.