Data Structures & Algorithms Digraphs and DAGs Richard Newman based on book by R. Sedgewick and slides by S. Sahni 1 1 1 1

Digraphs Edges are directed Number of possible undirected graphs is huge 2V(V+1)/2 Number of possible directed graphs is … huger(?) 2V^2 2 2 2 2

Digraphs Graph enumeration V Undir Graphs Digraphs 2 8 16 3 64 512 4 1,024 65,536 5 32,768 33,554,432 6 2,097,152 68,719,476,736 7 268,435,456 562,949,953,421,312 wow Wow wow wow 3 3 3 3

Digraphs Defn. 19.1: A digraph is a set of nodes V and a set of distinct directed edges E, each from one node to another node in V. (self-loop allowed) Defn. 19.2: A directed path in a digraph is a list of nodes for which there is an edge from each node to its successor. A node t is reachable from node s iff there is a d.p. from s to t. 4 4 4 4

Digraphs Defn. 19.3: A directed acyclic graph (DAG) is a digraph with no directed cycles (tours). A node with only out- edges is a source; a node with only in-edges is a sink. Defn. 19.4: A digraph is strongly connected iff every node is reachable from every node. 5 5 5 5

DAGs DAGs can be used to model many real-life problems Scheduling Precedence Pre-requisite structures Causality Etc. 6 6 6 6

Digraphs Prop. 19.1: A digraph that is not strongly connected comprises a set of strongly connected components, which are maximal strongly connected subgraphs, and a set of directed edges that go from one component to another. 7 7 7 7

DAGs Connected components 0-7-4-5-3 2 2-6 1 2 2 1 1 7 7 5 5 6 6 4 4 3 0-7-4-5-3 2 2-6 1 2 2 1 1 7 7 5 5 6 6 4 4 3 3 8 8 8 8

Digraphs Prop. 19.2: Given a digraph D, define another digraph K(D) with one node corresponding to each strongly connected component of D, and an edge from u to v iff there is one or more edge from the component corresponding to u to the component corresponding to v. K(D) is a DAG called the kernel DAG of D. 9 9 9 9

DAGs DAG Components 0: 0-7-4-5-3 1: 2-6 2: 1 Kernel DAG Component 0 1 1 7 7 5 5 6 6 4 4 3 3 1 2 10 10 10 10

Digraphs In undirected graph, we just say two nodes are connected if there is a path between them In a digraph, node t is reachable from node s if there is a directed path from s to t. In a digraph, s and t are strongly connected if they are mutually reachable. 11 11 11 11

Digraphs Classify edges in DFS Tree – recursive calls Back – to ancestor (including parent!) Down – to visited descendent Cross – neither ancestor nor descendent (cousins) 12 12 12 12

DFS in Digraphs Stack: 572 576 57(2) 541 54 55(6 cross) 53(0 back) 5(4 back) (5 down) down 2 7 2 2 6 4 back 1 1 1 7 7 5 5 cross 5 6 6 4 4 3 3 3 1 2 3 4 5 6 7 pre 4 1 7 5 6 2 3 post 7 2 1 3 5 4 6 13 13 13 13

DFS Algorithms Cycle Detection If we find a back edge, it represents a cycle – including link to parent! Cross edges don’t make cycles! Reachability Start from one, DFS until find other (or complete DFS) Weak connectivity If DFS finds all the nodes, then yes! 14 14 14 14

DFS Algorithms Convert digraph to DAG Remove back edges! Use to generate large DAGs from large digraphs Note that DFS in a digraph only gives reachability from the start node, not from all nodes 15 15 15 15

Transitive Closure Defn. 19.5: The transitive closure of a digraph D is a digraph T with the same vertices but with an edge from s to t in T iff t is reachable from s in D. 2 2 2 1 7 5 1 1 7 7 5 5 6 4 3 6 6 4 4 3 3 16 16 16 16

Transitive Closure Can also view (and compute) transitive closure by Boolean matrix multiplication Use logical AND as x Use logical OR as + Ai represents (any) path of length i A 0 1 2 3 4 5 6 7 0 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1 0 3 0 0 0 0 1 0 0 0 4 0 0 0 0 0 1 1 0 5 1 0 0 1 0 0 0 0 6 0 0 1 0 0 0 0 0 7 0 1 0 0 1 0 0 0 A2 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 1 1 0 0 0 0 0 0 1 0 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 17 17 17 17

Transitive Closure Can also view (and compute) transitive closure by Boolean matrix multiplication Use logical AND as x Use logical OR as + Ai represents (any) path of length i A<3 0 1 2 3 4 5 6 7 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 1 0 3 0 0 0 0 1 1 1 0 4 1 0 1 1 0 1 1 0 5 1 0 1 1 1 1 0 1 6 0 0 1 0 0 0 1 0 7 0 1 0 0 1 1 1 0 A3 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 0 0 1 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1 0 0 0 1 0 0 0 0 0 1 1 1 1 1 0 1 0 18 18 18 18

Transitive Closure Can also view (and compute) transitive closure by Boolean matrix multiplication Use logical AND as x Use logical OR as + Ai represents (any) path of length i A<4 0 1 2 3 4 5 6 7 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 1 0 3 1 0 1 1 1 1 1 0 4 1 0 1 1 1 1 1 1 5 1 1 1 1 1 1 1 1 6 0 0 1 0 0 0 1 0 7 1 1 1 1 1 1 1 0 A4 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 0 0 1 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 0 0 0 0 0 0 1 0 0 0 1 0 1 1 1 1 19 19 19 19

Transitive Closure Can also view (and compute) transitive closure by Boolean matrix multiplication Use logical AND as x Use logical OR as + Ai represents (any) path of length i A<5 0 1 2 3 4 5 6 7 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 1 0 3 1 0 1 1 1 1 1 1 4 1 1 1 1 1 1 1 1 5 1 1 1 1 1 1 1 1 6 0 0 1 0 0 0 1 0 7 1 1 1 1 1 1 1 1 A5 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 0 0 1 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0 0 0 0 1 1 1 1 1 1 1 0 20 20 20 20

Transitive Closure Can also view (and compute) transitive closure by Boolean matrix multiplication Use logical AND as x Use logical OR as + Ai represents (any) path of length i A<6 0 1 2 3 4 5 6 7 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 1 0 3 1 1 1 1 1 1 1 1 4 1 1 1 1 1 1 1 1 5 1 1 1 1 1 1 1 1 6 0 0 1 0 0 0 1 0 7 1 1 1 1 1 1 1 1 Keep on multiplying and adding until… … reach fixed point Matrix does not change 21 21 21 21

Transitive Closure Prop. 19.5: We can compute the transitive closure of a digraph by adding self-loops, then computing AV, taking time V4. Self-loops allow path to be of any length up to exponent Must reach fixed point by V – why? Efficient approach: A, A2, A4, A8, … successive squaring Takes lg V matrix multiplies, each V3 Total time is V3 lg V 22 22 22 22

Transitive Closure Even faster way! Warshall’s algorithm: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1; 23 23 23 23

Transitive Closure Correctness by induction on i: Base: After first iteration, s-t or s-0-t After second iteration, s-t, s-0-t, s-1-t, s-0-1-t, s-1-0-t. IH: After ith iteration – all paths w/o inner nodes > i Inductive step: path from s to t w/o i+1 (already there) or path via i+1 (tested by if statement) for (every intermediate node i) for (every source s) for (every destination t) if (s reaches i & i reaches t) s reaches t; 24 24 24 24

Transitive Closure Prop. 19.7: Warshall’s algorithm computes the transitive closure of a digraph in time V3. Obvious from structure of Warshall’s algorithm – three nested loops of V each: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1; 25 25 25 25

Transitive Closure Prop. 19.8: We can support constant-time reachability testing for a digraph with V nodes using space O(V2) and preprocessing time O(V3). Can improve Warshall’s algorithm: for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] && A[i][t]) A[s][t] = 1; 26 26 26 26

Transitive Closure We can improve Warshall’s algorithm by moving the test of A[s][i] out of the inner loop, avoiding innermost loop when s cannot reach i. for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) if (A[s][i]) for (t = 0; t < V; ++t) if (A[i][t]) A[s][t] = 1; 27 27 27 27

Shortest Path We can modify Warshall’s algorithm to compute shortest path, if A[][] contains the length of the minimum path from s to t (initialized with 1 for an edge and sentinel value V for no edge). for (i = 0; i < V; ++i) for (s = 0; s < V; ++s) for (t = 0; t < V; ++t) if (A[s][i] + A[i][t] < A[s][t]) A[s][t] = A[s][i] + A[i][t]; 28 28 28 28

Reduction Prop. 19.9: We can use any transitive- closure algorithm to compute the product of two Boolean matrices with at most a constant factor difference in running time. Prf: Construct a 3v x 3v matrix using A, B, and VxV identity matrix I. TC is square. 2 I A 0 I A AB 0 I B = 0 I B 0 0 I 0 0 I 29 29 29 29

Reduction What this means is that if we can perform transitive closure faster, then we can compute Boolean matrix products faster. Likewise, a faster Boolean matrix multiply algorithm will speed up our TC algorithm. Note that we can compute TC faster for sparse graphs – time O(V(E+V)) 30 30 30 30

Topological Sort Relabel: Given a DAG, relabel its nodes such that every directed edge points from a lower-numbered node to a higher- numbered node. Rearrange: Given a DAG, rearrange its nodes on a horizontal line such that all the directed edges point from left to right. Key = turn partial order into total order that is consistent with the partial order 31 31 31 31

Topological Sort Relabel: 2 2 7 2 1 7 5 1 1 2 7 7 1 5 5 5 6 4 3 6 6 6 2 7 2 1 7 5 1 1 2 7 7 1 5 5 5 6 4 3 6 6 6 4 4 3 3 Rearrange: 3 7 1 4 5 6 2 32 32 32 32

Reverse Topological Sort Relabel: Given a DAG, relabel its nodes such that every directed edge points from a higher-numbered node to a lower- numbered node. Rearrange: Given a DAG, rearrange its nodes on a horizontal line such that all the directed edges point from right to left. Just reverse the regular topological sort 33 33 33 33

Topological Sort Prop. 19.11: Postorder numbering in DFS yields a reverse topological sort for any DAG. It is easy to turn reverse topological sort into a regular topological sort 34 34 34 34

Topological Sort Prop. 19.12: Every DAG has at least one source and at least one sink. Turn this into a topological sort algorithm: Make indegree[V] vector initialized to 0 Scan through DAG (visiting every edge) and increment indegree[i] each time an edge to node i is found. Scan through indegree[] and enqueue all nodes with indegree zero (the sources). 35 35 35 35

Topological Sort Prop. 19.12: Every DAG has at least one source and at least one sink. Set currentID = 0. While the queue is non-empty, remove a node x and label it currentID currentID++ Decrement indegree[j] for all edges from node x to node j If indegree[j] == 0, enqueue node j 36 36 36 36

Topological Sort 2 2 7 2 1 7 5 1 3 1 7 2 7 5 5 5 6 4 3 6 6 6 4 4 3 1 indegree: 1 2 1 1 2 3 2 1 1 1 Source Queue: 3 7 1 4 5 6 2 37 37 37 37

Topological Sort Application Finding longest path from each node, longest path in DAG “Critical path” for scheduling What is the most urgent thing to do? Reverse topological sort DAG In RTS order, for each node v, longest path from v, LP[v] = 1 + max{LP[x] | (v,x) in E} Guaranteed that LP[x] is known by time it is needed. Example of… Dynamic Programming! 38 38 38 38

Topological Sort 2 2 7 1 3 1 7 2 7 5 5 5 6 6 6 4 4 1 3 Longest path: 5 1 3 1 7 2 7 5 5 5 6 6 6 4 4 1 3 Longest path: 5 4 4 1 3 1 2 1 39 39 39 39

Transitive Closure Redux Reachability from each node in DAG Reverse topological sort DAG Row vector Reach[v] – initially self and successors In RTS order, for each node v Reach[v] = OR {Reach[x] | (v,x) in E} Guaranteed that Reach[x] is known by time it is needed. Another example of… Dynamic Programming! 40 40 40 40

Transitive Closure Redux Matrix approach takes time O(VE) Direct recursive DFS approach: No back edges (no cycles) Tree edges – recursive call Cross edges – do OR, no call Down edges – skip, no call Prop. 19.13: Using DFS and DP, can compute TC of a DAG in time O(V2+VX) where X is the number of cross edges 41 41 41 41

Strongly Connected Components Kosaraju’s method Do DFS on reversed digraph Compute post-order numbering Do DFS of original graph using reverse of postorder computed on reverse graph Time and space are both linear in V+E! 42 42 42 42

Recap Digraphs Strong connectivity Connected components Reachability Digraph kernel Transitive closure Shortest paths (special case) Reduction (from Boolean matrix multiply) Topological Sort 43 43 43 43