Graph Algorithms: Part 1 UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2006 Lecture 4 Tuesday, 2/21/06 Graph Algorithms: Part 1 Shortest Paths Chapters 24-25

91.404 Graph Review Elementary Graph Algorithms Minimum Spanning Trees Single-Source Shortest Paths

Introductory Graph Concepts G= (V,E) Vertex Degree Self-Loops Undirected Graph No Self-Loops Adjacency is symmetric Directed Graph (digraph) Degree: in/out Self-Loops allowed B E C F D A B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts: Representations Undirected Graph Directed Graph (digraph) B E C F D A B E C F D A A B C D E F A B C D E F A B C D E F A B C D E F A BC B ACEF C AB D E E BDF F BE A BC B CEF C D D E BD F E Adjacency Matrix Adjacency List Adjacency List Adjacency Matrix This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts: Paths, Cycles length: number of edges simple: all vertices distinct Cycle: Directed Graph: <v0,v1,...,vk > forms cycle if v0=vk and k>=1 simple cycle: v1,v2..,vk also distinct self-loop is cycle of length 1 Undirected Graph: <v0,v1,...,vk > forms (simple) cycle if v0=vk and k>=3 B E C F D A path <A,B,F> B E C F D A simple cycle <E,B,F,E> most of our cycle work will be for directed graphs simple cycle <A,B,C,A>= <B,C,A,B> B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Introductory Graph Concepts: Connectivity connected B E C F D A Undirected Graph: connected every pair of vertices is connected by a path one connected component connected components: equivalence classes under “is reachable from” relation Directed Graph: strongly connected every pair of vertices is reachable from each other one strongly connected component strongly connected components: equivalence classes under “mutually reachable” relation A B C 2 connected components D F E B E C F D A not strongly connected strongly connected component B E C F D A This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

Elementary Graph Algorithms: SEARCHING: DFS, BFS for unweighted directed or undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix predecessor subgraph = forest of spanning trees Breadth-First-Search (BFS): Shortest Path Distance From source to each reachable vertex Record during traversal Foundation of many “shortest path” algorithms Depth-First-Search (DFS): Encountering, finishing times “well-formed” nested (( )( ) ) structure Every edge of undirected G is either a tree edge or a back edge EdgeColor of vertex when first tested determines edge type Vertex color shows status: not yet encountered encountered, but not yet finished finished See 91.404 DFS/BFS slide show See DFS, BFS Handout for PseudoCode

Elementary Graph Algorithms: DFS, BFS Review problem: TRUE or FALSE? The tree shown below on the right can be a DFS tree for some adjacency list representation of the graph shown below on the left. A C B E D F Tree Edge Cross Edge Back Edge B E C F D A

Elementary Graph Algorithms: Topological Sort for Directed, Acyclic Graph (DAG) G=(V,E) TOPOLOGICAL-SORT(G) 1 DFS(G) computes “finishing times” for each vertex 2 as each vertex is finished, insert it onto front of list 3 return list Produces linear ordering of vertices. For edge (u,v), u is ordered before v. See also 91.404 DFS/BFS slide show source: 91.503 textbook Cormen et al.

Minimum Spanning Tree: Greedy Algorithms Invariant: Minimum weight spanning forest Becomes single tree at end Time: O(|E|lg|E|) given fast FIND-SET, UNION A B C D E F G 2 1 3 4 5 6 8 7 Produces minimum weight tree of edges that includes every vertex. Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue Invariant: Minimum weight tree Spans all vertices at end for Undirected, Connected, Weighted Graph G=(V,E) source: 91.503 textbook Cormen et al.

Minimum Spanning Trees Review problem: For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees? A B C D E F G 2 1 3 4 5 6 8 7

Shortest Paths Chapters 24 & 25

BFS as a Basis for Some Shortest Path Algorithms for unweighted, undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix Source/Sink Shortest Path Problem: Given 2 vertices u, v, find the shortest path in G from u to v. Solution: BFS starting at u. Stop at v. Single-Source Shortest Paths Problem: Given a vertex u, find the shortest path in G from u to each vertex. Solution: BFS starting at u. Full BFS tree. All-Pairs Shortest Paths Problem: Find the shortest path in G from each vertex u to each vertex v. Solution: For each u: BFS starting at u; full BFS tree. Time: O(|V|(|V| + |E|)) adj list O(|V|3) adj matrix but for weighted, directed graphs… source: based on Sedgewick, Graph Algorithms

Shortest Path Applications for weighted, directed graph G=(V,E) Weight ~ Cost ~ Distance Road maps Airline routes Telecommunications network routing VLSI design routing source: based on Sedgewick, Graph Algorithms

Transitive Closure (Matrix): Unweighted, Directed Graph “self-loops” added for algorithmic purposes Transitive Closure concepts will be useful for All-Pairs Shortest Path calculation in directed, weighted graphs G Transitive Closure Graph contains edge (u,v) if there exists a directed path in G from u to v. source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) G G2 “self-loops” added for algorithmic purposes G G2 Boolean Matrix Product: and, or replace *,+ source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) G why this upper limit? Algorithm 1: Find G, G2 , G3 ,..., G|V-1| Time: O(|V|4) G2 Algorithm 2: Find G, G2 , G4 ,..., G|V| Time: O(|V|3lg|V|) Algorithm 3: [Warshall] for i 0 to |V|-1 for s 0 to |V|-1 for t 0 to |V|-1 if G[s][i] and G[i][t] then G[s][t] 1 Time: O(|V|3) G3 G4 source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) Warshall good for dense graphs source: Sedgewick, Graph Algorithms

Transitive Closure (Matrix) Warshall Correctness by Induction on i: Inductive Hypothesis: ith iteration of loop sets G[s][t] to 1 iff there’s a directed path from s to t with (internal) indices at most i. Inductive Step for i+1 (sketch): 2 cases for path <s…t> internal indices at most i - covered by inductive hypothesis in prior iteration so G[s][t] already set to 1 an internal index exceeds i (= i+1) - G[s][i+1], G[i+1][t] set in a prior iteration so G[s][t] set to 1 in current iteration source: Sedgewick, Graph Algorithms

Shortest Path Trees Shortest Path Tree gives shortest path from root to each other vertex shortest path need not be unique .99 .51 .38 .45 .83 .1 .41 .83 .1 .45 .21 .1 .51 .38 .36 .41 .5 source: Sedgewick, Graph Algorithms

All Shortest Paths source: Sedgewick, Graph Algorithms .41 .29 .29 .45 .21 .51 .32 .38 .36 .32 .50 source: Sedgewick, Graph Algorithms

Shortest Path Trees Shortest Path Tree is a spanning tree. 3 as root for reverse graph .41 .29 .29 .41 .29 .45 .21 .21 .32 .51 .32 .36 .38 .36 .32 .50 st = Spanning Tree reverse edges have same weight as forward ones predecessor vertex in tree Shortest Path Tree is a spanning tree. prelude to compact representation, except that uses next vertex, not predecessor source: Sedgewick, Graph Algorithms

All Shortest Paths (Compact) Total distance of shortest path Shortest Paths .41 .29 .29 .45 .21 .32 .51 .38 .36 .32 .50 Entry s,t gives next vertex on shortest path from s to t. source: Sedgewick, Graph Algorithms

All Shortest Paths In a Network Shortest Path Trees for reverse graph source: Sedgewick, Graph Algorithms

Shortest Path Principles: Relaxation “Relax” a constraint to try to improve solution Relaxation of an Edge (u,v): test if shortest path to v [found so far] can be improved by going through u A B C D E F G 2 1 3 4 5 6 8 7

Single Source Shortest Paths Bellman-Ford for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles weights source why this upper bound? Time is in O(|V||E|) update d(v) if d(u)+w(u,v) < d(v) detect negative-weight cycle source: 91.503 textbook Cormen et al.

with no negative-weight cycles Bellman-Ford for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles source: 91.503 textbook Cormen et al.

Single Source Shortest Paths: Dijkstra’s Algorithm Dijkstra’s algorithm solves problem efficiently for the case in which all weights are nonnegative (as in the example graph). Dijkstra’s algorithm maintains a set S of vertices whose final shortest path weights have already been determined. 1 2 3 4 6 5 10 8 It also maintains, for each vertex v not in S, an upper bound d[v] on the weight of a shortest path from source s to v. The algorithm repeatedly selects the vertex u e V – S with minimum bound d[u], inserts u into S, and relaxes all edges leaving u (determines if passing through u makes it “faster” to get to a vertex adjacent to u).

Single Source Shortest Paths: Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) implicit DECREASE-KEY source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) shortest path weight A B C D E F G 2 1 3 4 5 6 8 7 shortest path weight estimate source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm Review problem: For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? A B C D E F G 2 1 3 4 5 6 8 7 Why can’t Dijkstra’s algorithm handle negative-weight edges?

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) PFS = Priority-First Search = generalize graph search with priority queue to determine next step source: Sedgewick, Graph Algorithms

All-Pairs Shortest Paths for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles similar to Transitive Closure Algorithm 1 Time: O(|V|4) Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

All-Pairs Shortest Paths similar to Transitive Closure Algorithm 2 Time: O(|V|3lg|V|) Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

All-Pairs Shortest Paths similar to Transitive Closure Algorithm 3 [Warshall] Can have negative-weight edges Time: O(|V|3) How can output be used to detect a negative-weight cycle? Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

Food for thought… What does the following matrix (the nxn form of it) used in shortest-path algorithms correspond to in regular matrix multiplication? Note: D here is replaced by L in new edition

Shortest Path Algorithms source: Sedgewick, Graph Algorithms