Graph Algorithms: Part 1 UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 4 Tuesday, 2/23/10 Graph Algorithms: Part 1 Shortest Paths Chapters 24-25

91.404 Graph Review Elementary Graph Algorithms: Chapter 22 Minimum Spanning Trees: Chapter 23

Introductory Graph Concepts G= (V,E) Vertex Degree Self-Loops Undirected Graph No Self-Loops Adjacency is symmetric Directed Graph (digraph) Degree: in/out Self-Loops allowed B E C F D A B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts: Representations Undirected Graph Directed Graph (digraph) B E C F D A B E C F D A A B C D E F A B C D E F A B C D E F A B C D E F A BC B ACEF C AB D E E BDF F BE A BC B CEF C D D E BD F E Adjacency Matrix Adjacency List Adjacency List Adjacency Matrix This treatment follows 91.503 textbook Cormen et al. Some definitions may differ slightly from other graph literature.

Introductory Graph Concepts: Paths, Cycles length: number of edges simple: all vertices distinct Cycle: Directed Graph: <v0,v1,...,vk > forms cycle if v0=vk and k>=1 simple cycle: v1,v2..,vk also distinct self-loop is cycle of length 1 Undirected Graph: <v0,v1,...,vk > forms (simple) cycle if v0=vk and k>=3 B E C F D A path <A,B,F> B E C F D A simple cycle <E,B,F,E> most of our cycle work will be for directed graphs simple cycle <A,B,C,A>= <B,C,A,B> B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions may differ slightly from other graph literature.

Introductory Graph Concepts: Connectivity connected B E C F D A Undirected Graph: connected every pair of vertices is connected by a path one connected component connected components: equivalence classes under “is reachable from” relation Directed Graph: strongly connected every pair of vertices is reachable from each other one strongly connected component strongly connected components: equivalence classes under “mutually reachable” relation A B C 2 connected components D F E B E C F D A not strongly connected strongly connected component B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions may differ slightly from other graph literature.

Elementary Graph Algorithms: SEARCHING: DFS, BFS for unweighted directed or undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix predecessor subgraph = forest of spanning trees Breadth-First-Search (BFS): Shortest Path Distance From source to each reachable vertex Record during traversal Foundation of many “shortest path” algorithms Depth-First-Search (DFS): Encountering, finishing times “well-formed” nested (( )( ) ) structure Every edge of undirected G is either a tree edge or a back edge EdgeColor of vertex when first tested determines edge type Vertex color shows status: not yet encountered encountered, but not yet finished finished See 91.404 DFS/BFS slide show See DFS, BFS Handout for PseudoCode

Elementary Graph Algorithms: DFS, BFS Review problem: TRUE or FALSE? The tree shown below on the right can be a DFS tree for some adjacency list representation of the graph shown below on the left. A C B E D F Tree Edge Cross Edge Back Edge B E C F D A

Elementary Graph Algorithms: Topological Sort for Directed, Acyclic Graph (DAG) G=(V,E) TOPOLOGICAL-SORT(G) 1 DFS(G) computes “finishing times” for each vertex 2 as each vertex is finished, insert it onto front of list 3 return list Produces linear ordering of vertices. For edge (u,v), u is ordered before v. See also 91.404 DFS/BFS slide show source: 91.503 textbook Cormen et al.

Minimum Spanning Tree: Greedy Algorithms Invariant: Minimum weight spanning forest Becomes single tree at end Time: O(|E|lg|E|) given fast FIND-SET, UNION A B C D E F G 2 1 3 4 5 6 8 7 Produces minimum weight tree of edges that includes every vertex. Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue Invariant: Minimum weight tree Spans all vertices at end for Undirected, Connected, Weighted Graph G=(V,E) source: 91.503 textbook Cormen et al.

Minimum Spanning Trees Review problem: For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees? A B C D E F G 2 1 3 4 5 6 8 7

Shortest Paths Chapters 24 & 25 Chapter 24: Single-Source Shortest Paths (but not Section 24.4 on Difference Constraints) Chapter 25: All-Pairs Shortest Paths (but not Section 25.3 on Johnson’s Algorithms for Sparse Graphs)

BFS as a Basis for Some Shortest Path Algorithms for unweighted, undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix Source/Sink Shortest Path Problem: Given 2 vertices u, v, find the shortest path in G from u to v. Solution: BFS starting at u. Stop at v. Single-Source Shortest Paths Problem: Given a vertex u, find the shortest path in G from u to each vertex. Solution: BFS starting at u. Full BFS tree. All-Pairs Shortest Paths Problem: Find the shortest path in G from each vertex u to each vertex v. Solution: For each u: BFS starting at u; full BFS tree. Time: O(|V|(|V| + |E|)) adj list O(|V|3) adj matrix but for weighted, directed graphs… source: based on Sedgewick, Graph Algorithms

Shortest Path Applications edge weights for weighted, directed graph G=(V,E) adjacency linked list adjacency matrix Weight ~ Cost ~ Distance Road maps Airline routes Telecommunications network routing VLSI design routing source: based on Sedgewick, Graph Algorithms

Shortest Path Trees source: Sedgewick, Graph Algorithms Example from previous slide: New example with same vertices and edges but different edge weights: different edge weights Shortest Path Tree gives shortest path from root to each other vertex Shortest Path Tree is a spanning tree. Shortest path need not be unique, but shortest path total weight is unique. Here assume 0 is root. .99 .51 .38 .45 .83 .1 .41 .83 .1 .45 .21 .1 .51 .38 .36 .41 .5

Shortest Path Principles: Optimal Substructure Lemma: Any subpath of a shortest path is a shortest path. Proof: Cut-and-paste argument. Let p be a shortest path from u to v. u x y v pux pxy pyv p: Suppose there existed a shorter path p’xy from x to y. Then: forming a path from u to v with smaller weight than path p contradicts the assumption that p is a shortest path. u x y v pux p’xy pyv source: 91.503 textbook Cormen et al.

Shortest Path Principles: Relaxation “Relax” a constraint to try to improve solution Relaxation of an edge (u,v): test if shortest path to v [found so far] can be improved by going through u A B C D E F G 2 1 3 4 5 6 8 7 Example: source: 91.503 textbook Cormen et al.

Shortest Path Principles (cont.) for weighted, directed graph G=(V,E) with no negative-weight cycles shortest path weight A B C D E F G 2 1 3 4 5 6 8 7 shortest path weight estimate proof uses upper-bound property proof uses convergence property source: 91.503 textbook Cormen et al.

Single-Source Shortest Paths Chapters 24 & 25 Chapter 24: Single-Source Shortest Paths (but not Section 24.4 on Difference Constraints)

Single Source Shortest Paths Bellman-Ford for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles weights source why this upper bound? Time is in O(|V||E|) update d(v) if d(u)+w(u,v) < d(v) detect negative-weight cycle source: 91.503 textbook Cormen et al.

with no negative-weight cycles Bellman-Ford for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles Board work: Example of negative weight cycle detection. Note: Edges are relaxed in lexicographic order ; other orders are possible. source: 91.503 textbook Cormen et al.

Single-Source Shortest Paths in DAG Useful application: PERT charts for scheduling activities source: materials to accompany 91.503 textbook Cormen et al.

Single Source Shortest Paths: Dijkstra’s Algorithm Dijkstra’s algorithm solves problem efficiently for the case in which all weights are nonnegative (as in the example graph). Dijkstra’s algorithm maintains a set S of vertices whose final shortest path weights have already been determined. 1 2 3 4 6 5 10 8 It also maintains, for each vertex v not in S, an upper bound d[v] on the weight of a shortest path from source s to v. The algorithm repeatedly selects the vertex u e V – S with minimum bound d[u], inserts u into S, and relaxes all edges leaving u (determines if passing through u makes it “faster” to get to a vertex adjacent to u).

Single Source Shortest Paths: Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) Similar to weighted version of BFS. implicit DECREASE-KEY Loop invariant: At start of each iteration of while loop, d [v] = d (s,v) for all v in S. Greedy strategy: where is greedy choice being made? source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm Review problem: For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? A B C D E F G 2 1 3 4 5 6 8 7 Why can’t Dijkstra’s algorithm handle negative-weight edges?

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) Fibonacci Heap (Ch. 20) O(VlgV + E) amortized analysis PFS = Priority-First Search = generalize graph search with priority queue to determine next step. Fringe = set of edges that are candidates for being added next to shortest path tree. sources: Sedgewick, Graph Algorithms & 91.503 textbook Cormen et al.

All-Pairs Shortest Paths Chapter 25 Chapter 25: All-Pairs Shortest Paths (but not Section 25.3 on Johnson’s Algorithms for Sparse Graphs)

Transitive Closure (Matrix): Unweighted, Directed Graph Transitive Closure concepts will be useful for All-Pairs Shortest Path calculation in directed, weighted graphs G “self-loops” added for algorithmic purposes Transitive Closure Graph contains edge (u,v) if there exists a directed path in G from u to v. source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) G G2 “self-loops” added for algorithmic purposes G G2 Boolean Matrix Product: and, or replace *,+ source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) G why this upper limit? Algorithm 1: Find G, G2 , G3 ,..., G|V-1| Time: O(|V|4) G2 Algorithm 2: Find G, G2 , G4 ,..., G|V| Time: O(|V|3lg|V|) Algorithm 3: [Warshall] for i 0 to |V|-1 for s 0 to |V|-1 for t 0 to |V|-1 if G[s][i] and G[i][t] then G[s][t] 1 Time: O(|V|3) G3 G4 source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) Warshall good for dense graphs (why?) Consider 0 as maximum intermediate vertex label. Consider 3 as maximum intermediate vertex label. Consider 1 as maximum intermediate vertex label. Consider 4 as maximum intermediate vertex label. Consider 5 as maximum intermediate vertex label. Consider 2 as maximum intermediate vertex label. source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) Warshall Correctness by Induction on i: Inductive Hypothesis: ith iteration of loop sets G[s][t] to 1 iff there’s a directed path from s to t with (internal) indices at most i. Inductive Step for i+1 (sketch): 2 cases for path <s…t> internal indices at most i - covered by inductive hypothesis in prior iteration so G[s][t] already set to 1 an internal index exceeds i (= i+1) - G[s][i+1], G[i+1][t] set in a prior iteration so G[s][t] set to 1 in current iteration source: Sedgewick, Graph Algorithms

All Shortest Paths source: Sedgewick, Graph Algorithms .41 .29 .29 .45 .21 .51 .32 .38 .36 .32 .50 total path weight from 1 to 0 source: Sedgewick, Graph Algorithms

All Shortest Paths (Compact) Total distance of shortest path Shortest Paths .41 .29 .29 .45 .21 .32 .51 .38 .36 .32 .50 Entry s,t gives next vertex on shortest path from s to t (our textbook uses predecessor). source: Sedgewick, Graph Algorithms

Slides courtesy of Prof. Pecelli Algorithms for ALL shortest path pairs. If all the edge weights were non-negative, we could use Dijkstra’s algorithm: O(|V|(|V| lg |V| + |E|)), which, for a dense graph, could be O(|V|3). If we allow negative edge weights, we must use Bellman-Ford: O(|V|•|E|) from each vertex, which could be O(|V|2•|E|) = O(|V|4 ). Can we do better? Is there some advantage in looking for algorithms that would compute ALL the path lengths simultaneously? We separate the problem into two parts: dense graphs and sparse graphs. 1/18/2019

Slides courtesy of Prof. Pecelli Dense Graphs – the Floyd-Warshall Algorithm. The family of algorithms normally presented in this context is based on analogs of matrix multiplication - more precisely dynamic programming: graphs are represented as adjacency matrices rather than adjacency lists. This will lead us to the Floyd-Warshall algorithm. Johnson’s algorithm, used for sparse graphs and discussed in textbook (Section 25.3), uses adjacency lists. Note: Social Networks (the rage today) seem to be based on “sparse graphs” (|E| = O(|V|*log|V|)) even though many nodes have large numbers of edges. 1/18/2019

Slides courtesy of Prof. Pecelli Some notation. G = (V, E); |V| = n; W = (wij); wij = weight of edge (i, j); wii = 0, wij = ∞ if (i, j) not in E. Negative edges allowed; no negative cycles. D = (dij) = weight matrix for shortest paths: dij = d(i, j). P = (pij) = predecessor matrix on shortest paths: predecessor of j in some shortest path from i to j; pij = NIL if i = j or (i, j) not in E. ith row of P = shortest path tree with root i. Gp = predecessor graph; Gp,i = (Vp,i, Ep.i) = predecessor subgraph for i; Vp,i = {j in V: pij ≠ NIL}U{i}; Ep,j = {(pij, j): j in Vp,j – {j}}. 1/18/2019

Slides courtesy of Prof. Pecelli Shortest Paths & Matrix Multiplication. We need to first characterize the structure of the solutions – we try a dynamic programming approach, which will also give us a way to carry the program to success. Characterize the structure of an optimal solution Recursively define the value of an optimal solution Compute the value of an optimal solution in a bottom up fashion. Construct the optimal solution from computed information. 1/18/2019

Slides courtesy of Prof. Pecelli Structure of a shortest path. By Lemma 24.1 we know that subpaths of shortest paths are shortest paths. Let W = (wij) be the adjacency matrix of the graph. Let p be a path from i to j, p with at most m edges (no negative cycles implies p is finite). if i = j, p has weight 0 and no edges if i ≠ j, p can be decomposed into i –p’ k j, where p’ has at most m-1 edges. Lemma 24.1 gives p’ as a shortest path from i to k, d(i,j) = d(i, k) + wkj. We are now ready to set up a recursive Dynamic Programming algorithm to compute the solution. 1/18/2019

Slides courtesy of Prof. Pecelli Recursive Solution. Let lij(m) denote the min weight of any path from i to j using no more than m edges. The recursion gives: The actual shortest path lengths eventually stabilize: d(i, j) = lij(n-1) = lij(n) = lij(n+1) … (25.2) 1/18/2019

Slides courtesy of Prof. Pecelli Shortest Paths & Matrix Multiplication. We can translate this idea from (25.2) into the pseudo-code: Where the input matrices are W and L(m) and the output is L(m+1). 1/18/2019

Slides courtesy of Prof. Pecelli If we look at the operations min and +, and replace them by + and *, we see a correspondence with matrix multiplication: Each iteration of the algorithm has cost Q(n3). Because of the previous argument on termination, the total cost is Q(n4). Substitutions: 1/18/2019 similar to Transitive Closure Algorithm 1

Running Time is in Q(n4). similar to Transitive Closure Algorithm 1 1/18/2019 43

Slides courtesy of Prof. Pecelli We can speed up the running time to Q(n3log(n)) by using a well-know trick in matrix power computations We’ll see this same trick used to good effect in the implementation of RSA encryption. similar to Transitive Closure Algorithm 2 1/18/2019

Slides courtesy of Prof. Pecelli Floyd-Warshall: can we find a better dynamic programming algorithm (e.g., Q(|V|3))? – still with no negative-weight cycles. The only option open to us (short of finding a much better matrix multiplication algorithm) is to find a more efficient characterization of the structure of a shortest path – which is what allows us to use the dynamic programming framework. Def.: an intermediate vertex of a simple path p = <v1, v2, …, vl> is any vertex of p other than v1 or vl. 1/18/2019

Slides courtesy of Prof. Pecelli Observation: if V = {v1, v2, …, vn} = {1, …, n}, consider a subset {v1, v2, …, vk} for some k. For any pair of vertices i, j in V, consider all the paths from i to j whose intermediate vertices are all drawn from {1, 2, …, k}. Let p be a (simple) minimum-weight path among them. If k is not an intermediate vertex of p, then all intermediate vertices of p are in {1, 2, …, k-1}. Thus a shortest path from i to j with all intermediate vertices in {1, 2, …, k-1} is a shortest path from i to j with all intermediate vertices in {1, 2, …, k}. If k is an intermediate vertex of p, then decompose p = i –p1-> k –p2-> j. By Lemma 24.1, p1 is a shortest path from i to k with all intermediate vertices in {1, 2, …, k}. Since k is not an intermediate vertex of p1, all the intermediate vertices are in {1, 2, …, k-1}: p1 is a shortest path from i to k with intermediate vertices in {1, 2, …, k-1}; similarly p2 from k to j. 1/18/2019

Slides courtesy of Prof. Pecelli A picture that might help: 1/18/2019

Slides courtesy of Prof. Pecelli A recursive solution. Let dij(k) be the weight of a shortest path from i to j with intermediate vertices in {1, …, k}. When k = 0, the path can have no intermediate vertices, so dij(0) = wij. From the discussion on the previous slides: Since for any path all intermediate vertices are in {1, …, n}, dij(n) = d(i, j). 1/18/2019

Slides courtesy of Prof. Pecelli Running Time is in Q(n3). similar to Transitive Closure Algorithm 3 [Warshall] 1/18/2019

Shortest Path Algorithms source: Sedgewick, Graph Algorithms

Connected Dominating Set Literature Paper “On Calculating Connected Dominating Set for Efficient Routing in Ad Hoc Wireless Networks” by Wu and Li, DIAL M 1999.

Goal & Approach of Paper Goal: Efficient routing for mobile hosts All hosts have same wireless transmission range. Approach: Represent network by undirected, unweighted graph G = (V, E). 2 nodes connected if both hosts are within transmission range Find small, connected, dominating set of gateway nodes In addition, this set must include all intermediate nodes on every shortest path between 2 nodes in dominating set. All-pairs shortest paths Maintain routing table for gateway nodes. Use heuristic, due to suspected intractability of minimum dominating set problem. 1 2 3 4 5 6 7 8 9 source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”

Dominating Set 1 2 3 4 5 6 7 8 9 source: Garey & Johnson

Marking Heuristic Initialize each node’s marker to F (false) Each vertex v exchanges its open neighbor set N(v) with each of its neighbors Each vertex changes its marker to T (true) if it can determine that it has 2 (directly) unconnected neighbors. Resulting V ’ induces reduced graph G ’ = G [V ’ ]. THEOREM 1 (reworded): If G is (not completely) connected, then V ’ is a dominating set of G. THEOREM 2 (reworded): G ’ is connected. THEOREM 3 (reworded): Shortest path between any 2 nodes of G ’ does not include any non-gateway node as an intermediate node. 1 2 3 4 5 6 7 8 9 source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”

Enhanced Marking Heuristic RULES 1 & 2 try to reduce number of gateway nodes assuming ordering on vertex id’s in V ’ define closed neighbor set of v in V ’: RULE 1: Consider 2 vertices v and u in G ’ . If in G and id(v) < id(u), change marker of v to F if node v is marked. RULE 2: Given 2 marked neighbors u and w of marked vertex v in G ’, if in G and then change marker of v to F. Paper also discusses dynamic network with hosts turning on and off. Experimental simulation uses random graphs. RULE 1 RULE 2 v u v u u v w id=1 id=2 source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”