Algorithm Design and Analysis (ADA) 242-535, Semester 1 2014-2015 12. Max Flow Networks Objective describe the maxflow problem, explain and analyse the Ford-Fulkerson, Edmonds-Karp, and capacity scaling algorithms, and look at two application areas (bipartite matching and edge disjoint paths)

Overview A Flow Network The Maxflow Problem The Mincut Problem Ford-Fulkerson (FF) Algorithm Residual Network Ford-Fulkerson (FF) Again Edmonds-Karp Algorithm Capacity Scaling Two Flow Problems Again Bipartite Matching Edge Disjoint Paths Making Maxflow Faster

1. A Flow Network postive flow capacities source sink

A flow network is a directed graph G = (V, E) with two special vertices: a source s and a sink t Each edge (u, v) ∈ E has a non-negative capacity c(u, v) If(u, v) ∉ E, then c(u, v) = 0

Two Flow Problems The maxflow problem The mincut problem find a flow of maximum value The mincut problem find a cut of minimum capacity Two problems that appear very different but are actually two versions of the same question. Most of this part is about the theory and implementation of maxflow.

2. The Maxflow Problem A flow is an assignment of values to edges such that: capacity constraint: 0 ≤ edge's flow ≤ edge's capacity flow conservation: inflow = outflow at every vertex (except s and t). inflow at v = 5 + 5 + 0 = 10 outflow at v = 10 + 0 = 10 flow: f(u,v) capacity: c(u,v) [ / is not division]

The flow value f is the inflow at the sink t or the outflow at the source s Maxflow problem: find a flow of maximum value flow = 8 + 10 + 10 = 28

Another Example A flow network G labelled with its edge capacities c(u,v) A flow f in G with value 19.

Flow Notation A flow on G is a function f(u,v) satisfying the following conditions: capacity constraint:  u, v ∈ V, f(u, v) ≤ c(u, v) flow conservation:  u ∈ V - {s, t}, 𝒗𝝐𝑽 𝒇 𝒖,𝒗 =𝟎 skew symmetry:  u, v ∈ V, f(u, v) = -f(v, u) The flow value f = f(V, t) ( or f(s, V) )

Many Applications of Max Flow Bipartite matching Data mining Network reliability Image processing Network connectivity Distributed computing many more ... blood flow analysis

3. The Mincut Problem A cut is a partition of a flow network's vertices into two disjoint sets, with the source (s) in one set A and the sink (t) in the other set B. A cut's capacity is the sum of the capacities of the edges from A to B. The minimum cut (mincut) problem: find a cut of minimum capacity.

find a cut of minimum capacity. Examples don't count edges from B to A capacity = 10 + 5 + 15 = 30 capacity = 10 + 8 + 16 = 34 The minimum cut (mincut) problem: find a cut of minimum capacity.

4. Ford-Fulkerson (FF) Algorithm Initialization. Start with a 0 flow on every edge. Find an augmenting path from s to t such that: we can increase flow on forward edges (while they are not full) we can decrease flow on backward edge (while they are not empty) initially flow value =0

First Augmenting Path

Second Augmenting Path

Third Augmenting Path

Fourth Augmenting Path

Termination No more augmenting paths All paths from s to t are blocked by either: a full forward edge, or a empty backward edge maxflow = 28 mincut = 28

5. Residual Network Let f be a flow on G = (V, E). The residual network Gf(V, Ef) is a graph showing residual capacities cf(u, v) = c(u, v) - f(u, v) > 0 The idea is to draw the flow network G as graph Gf to make it easier to find augmented paths, and to determine the flow increase (and decrease) the chosen path causes. for increasing flow in (u,v) two residual capacities cf() flow f() capacity c() Graph G Graph Gf 11 6 / 17 u v u v for decreasing flow in (u,v) 6

Gf Edges for Reducing Residual networks also show how flow along an edge can be reduced this is shown by having extra edges in Gf which are not in G To represent a possible decrease of a flow f(u,v) on an edge in G, the edge (v,u) is added to Gf. Its residual capacity is: cf(v, u) = f(u,v)

Residual Capacity cf() The complete definition of cf() is: for increasing flow in (u,v) for decreasing flow in (u,v) two residual capacities cf() flow f() capacity c() Graph G Graph Gf 11 6 / 17 u v u v 6

Augmenting Paths A path from s to t in Gf is an aug­menting path in G which carries flow f. The flow amount can be increased along an augmenting path by the minimum amount cf( p) = min {cf(u,v)} cf(p) is the residual or bottleneck capacity – it is the most that the flow can be increased due to one or more of p's edges being used at full capacity. (u,v) ∈ p

Example The flow network G from an earlier example. The residual network Gf with a possible augmenting path p shaded; its bottleneck capacity is cf(p) = cf(v2, v3) = 4. Edges with residual capacity equal to 0, such as (v1,v3) are not shown.

The new flow in G that results from augmenting along path p by its bottleneck capacity 4. Edges carrying no flow, such as (v3, v2), are labeled only by their capacity. The residual network Gf version of this G graph. No more augmented paths can be added, so f is the maxflow (f = 23).

6. Ford-Fulkerson (FF) Again Start with 0 flow. While there exists an augmenting path: find an augmenting path compute bottleneck capacity increase flow on that path by bottleneck capacity

FF in More Detail ford-Fulkerson(G) { foreach e ∈ E in G f(e) = 0 Gf = residual graph of G while (there exists augmenting path P) { cf(p) = min{ cf(u,v) : (u,v) ∈ P } // bottleneck capacity foreach e ∈ P { // augment the flows with cf(p) if (e ∈ E in G) f(e) = f(e) + cf(p) // forward edge: e = (u,v) else f(e') = f(e') - cf(p) // reverse edge: e' = (v,u) } update Gf

Executing of FF with a Residual Network The figures on the next 2 slides show successive iterations of the FF while-loop. The left side shows the residual network Gf with the chosen augmenting path p drawn as a shaded thick line. The right side shows the new flow f in G that results from adding the bottleneck capacity of p (cf(p)).

Gf G

The last residual network has no augmenting paths, and so the flow f shown in (e) above is a maximum flow (f = 23).

FF Running Time In the worst case, the running time of FF is O(E · |f|), where f is the maxflow in the worst case, the FF while loop will iterate f times since it's possible for the flow to increase by only I flow unit at a time (see example on the next slides) finding a path can have varying execution times, but if DFS or BFS is used to find a path, the running time is O(V + E), or O(E) is a dense graph

Worst Running Time The following example is designed so that each augmented path only increases the flow by 1 unit (i.e. the cf(p) is always 1).

The Augmenting Paths First Second

many, many more Third Fourth

199th 200th (= maxflow)

Choosing Paths is Important This case is easily avoided by a choosing a better series of augmenting paths. Only two iterations are needed.

Choosing Good Augmenting Paths Use care when selecting augmenting paths Some choices lead to exponential algorithms Clever choices lead to polynomial algorithms If capacities are not integers, some algorithm are not guaranteed to terminate Goal: choose augmenting paths so that: Can find augmenting paths efficiently Few iterations Choose augmenting paths with: Max bottleneck capacity Sufficiently large bottleneck capacity Fewest number of edges

7. Edmonds-Karp Algorithm Edmonds and Karp's implementation of Ford-Fulkerson finds an augmenting path by using a breadth-first search. the algorithm is slightly changed: a weight of 1 is assigned to every edge, not the edge's capacity It runs in O(V· E2) time this can be proved by counting the number of augmenting paths needed to find the maxflow (which is O(V· E) ) the proof requires the monotonicity lemma

Monotonicity Lemma Let d(v) = df(s, v) be the breadth-first distance from s (the source) to a vertex v in Gf each edge has a weight of 1 Lemma. During the execution of Edmonds-Karp, d(v) increases monotonically (only gets bigger).

Proof Suppose that augmenting a flow f on G produces a new flow f'. Let d′(v) = df'(s, v). We’ll show d′(v) ≥ d(v) by induction on d′(v) we are proving that increased flow makes the shortest path distance bigger for every vertex v For the base case, d′(v) = 0 implies v = s (the source), and since d(s) = 0, we have d′(v) ≥ d(v). So the base case is true.

For the inductive case, consider a breadth-first path s → …→ u → v in Gf' We must have d′(v) = d′(u) + 1, since subpaths of shortest paths are shortest paths (optimality). So, we have d′(u) ≥ d(u) by induction, because d′(v) > d′(u). (u, v) ∈ Ef' (because we assumed it; see above). But we have to consider two cases depending on whether (u, v) ∈ Ef or not.

Case 1: (u, v) ∈ Ef We have d(v) ≤ d(u) + 1 (triangle inequality) ≤ d′(u) + 1 (induction) = d′(v) (breadth-first path) and thus monotonicity of d(v) is established.

Case 2: (u, v) ∉ Ef Since (u, v) ∈ Ef' , the augmenting path p that produced f' from f must have included (v, u). Moreover, p is a breadth-first path in Gf : p = s → … → v → u → … → t Thus, we have d(v) = d(u) - 1 (breadth-first path) ≤ d′(u) - 1 (induction) = d′(v) - 2 (breadth-first path) < d′(v) thereby establishing monotonicity for this case, too.

Counting Flow Augmentations Theorem. The number of flow augmentations in the Edmonds-Karp algorithm (Ford-Fulkerson with breadth-first augmenting paths) is O(V· E). Proof. Let p be an augmenting path, and suppose that we have cf(u, v) = cf(p) for edge (u, v) ∈ p i.e. the bottleneck edge is (u, v) We say that (u, v) is critical, and it disappears from the residual graph after flow augmentation.

The first time an edge (u, v) is critical, we have d(v) = d(u) + 1, since p is a breadth-first path. We must wait until (v, u) is on an augmenting path before (u, v) can be critical again. Let d′ be the distance function when (v, u) is on an augmenting path. Then, we have d′(u) = d′(v) + 1 (breadth-first path) ≥ d(v) + 1 (monotonicity) = d(u) + 2 (breadth-first path)

Why "+2"? Example assume (u,v) is critical

assume (v,u) is critical was 5, now is 7 was 6, now is 8

Running time of Edmonds-Karp Path distances start out non-negative, never decrease, and are at most |V| - 1 long until the vertex becomes unreachable. Thus, (u, v) occurs as a critical edge at most O(V/2) times, because d(v) increases by at least 2 between occurrences simplify O(V/2) to be O(V) Since the residual graph contains O(E) edges, the number of flow augmentations is O(V· E).

Corollary. The Edmonds-Karp maximum-flow algorithm runs in O(V· E2) time. Proof. Breadth-first search runs in O(E) time (actually O(V + E) but ignore the V), so the total running time is: O(V· E · E) = O(V· E2)

Edmonds-Karp Code boolean[] marked; // true if s->v path is in residual network FlowEdge[] edgeTo; // last edge on s->v path Queue<Integer> q = new Queue<Integer>(); // use when finding an augmenting path

double edmondsKarp(FlowNetwork graph, int s, int t) { double value = 0; // flow value while (hasAugmentingPath(graph, s, t)) { double bottle = Double.POSITIVE_INFINITY; // initial bottleneck capacity for (int v = t; v != s; v = edgeTo[v].other(v)) bottle = Math.min(bottle, edgeTo[v].residualCapacityTo(v)); edgeTo[v].addResidualFlowTo(v, bottle); value += bottle; // augment flow } return value; // will be the maxflow

boolean hasAugmentingPath(FlowNetwork graph, int s, int t) { marked boolean hasAugmentingPath(FlowNetwork graph, int s, int t) { marked.clear(); // set all marks to false q.reset(s); marked[s] = true; while (!q.isEmpty()) { int v = q.remove(); for (FlowEdge e : graph.adj(v)) { int w = e.other(v); // found path from s to w? if(e.residualCapacityTo(w) > 0 && !marked[w]) { edgeTo[w] = e; // save last edge on path to w; mark w; marked[w] = true; q.add(w); } return marked[t]; // is t reachable from s in residual network?

8. Capacity Scaling Choosing an augmenting path with the highest bottleneck capacity will increase the flow by the biggest possible amount. Don't try to find an exact highest bottleneck path (too slow) Use a scaling parameter Δ instead Let Gf (Δ) be the subgraph of the residual graph consisting of only arcs with capacity of at least Δ

Capacity Scaling Code scaling-Max-Flow(G, C) { foreach e ∈ E in G f(e) = 0 Δ = smallest power of 2 greater than or equal to C Gf = residual graph of G while (Δ ≥ 1) { Gf(Δ) = Δ-residual graph while (there exists augmenting path P in Gf(Δ)) { cf(p) = min{ cf(u,v) : (u,v) ∈ P } // bottleneck capacity foreach e ∈ P { // augment flows with P if (e ∈ E) f(e) = f(e) + b // forward edge e=(u,v) else f(e') = f(e') - b // reverse edge e'=(v,u) } update Gf(Δ) } Δ = Δ / 2

Running Time The number of augmenting paths this algorithm calculates is O(E log C) the log component is due to the outer while loop repeating O(log2 C) times since initially C ≤ Δ < 2C, and Δ decreases by a factor of 2 on each iteration Total running time is O(E2 log C) since each augmentating path will be calculated in O(E) time using BFS (as before)

9. Two Flow Problems Again The maxflow problem find a flow of maximum value The mincut problem find a cut of minimum capacity Two problems that appear very different but are actually two versions of the same question. Maxflow-mincut theorem: maxflow value = mincut capacity

Relationship between Flows and Cuts The net flow across a cut (A, B) is the sum of the flows on its edges from A to B minus the sum of the flows on its edges from from B to A. If f is the current flow and let (A, B) be any cut. Then, the net flow across (A, B) equals the value of f.

Example net flow across cut = 10 + 5 + 10 = 25 flow = 25

Example maxflow = 28 mincut = 28

Computing a mincut from a maxflow It is easy to calculate the mincut (A, B) if the maxflow is known (f). The A set of the mincut = the set of vertices connected to s by an undirected path with no full forward or empty backward edges.

10. Bipartite Matching N students apply for N jobs. Each gets several offers. Is there a way to match every student to a job?

More Formally Input: undirected, bipartite graph G = (L  R, E). M  E is a matching if each node appears in at most edge in M. Max matching: find a maximum cardinality matching. Matching: 1- 2', 3 – 1', 4 – 5'

Changing Bipartite to Max Flow Direct all edges in E from L to R, and assign capacity 1 to each edge. Add source s, and capacity 1 edges from s to each node in L. Add sink t, and capacity 1 edges from each node in R to t. The resulting digraph G' = (L  R  {s, t}, E' ).

Conversion to Max Flow source sink Carol Dave Bob Eliza Alice Frank Adobe Yahoo Amazon IBM Facebook Google sink

11. Edge Disjoint Paths Given a digraph G = (V, E) and two nodes s and t, find the max number of edge-disjoint s-t paths. two paths are edge-disjoint if they have no edge in common Example: communication networks

A solution using 2 edge-disjoint paths:

Max Flow Formulation Assign unit capacity to every edge. The maximum number of edge-disjoint source-to-sink paths equals the max flow value.

Dual Problem: Network Connectivity Given a digraph G = (V, E) and two nodes s and t, find the minimum number of edges whose removal disconnects t from s. A set of edges F  E disconnects t from s if each s-t paths uses at least one edge in F removing F would make t unreachable from s Removing two edges breaks the s-t link

Theorem [Menger 1927]. The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s. utilizes the mincut

12. Making MaxFlow Faster C is the maximum capacity of any edge year method worst case running time discovered by 1955 augmenting path O(E · |f|) Ford-Fulkerson 1970 shortest augmented path O(V · E2) Edmonds-Karp 1983 dynamic trees O(E2 ·log E) Sleator-Tarjan 1985 capacity scaling O(E2 · log C) Gabow 1997 length function O(E3/2 · log E · log C) Goldberg-Rao ? E C is the maximum capacity of any edge

Analysis is Tricky Worst-case big-Oh is generally not useful for predicting or comparing real-world max flow algorithm performance. Best in practice: push-relabel method with gap relabeling: O(E3/2) (can be written as O(V3) very close to linear in E, but still an open question