G RAPH A LGORITHMS Dr. Tanzima Hashem Assistant Professor CSE, BUET

G RAPHS A graph G = (V, E) V = set of vertices E = set of edges = subset of V  V Thus |E| = O(|V| 2 )

G RAPH V ARIATIONS Variations: A connected graph has a path from every vertex to every other In an undirected graph: Edge (u,v) = edge (v,u) No self-loops In a directed graph: Edge (u,v) goes from vertex u to vertex v, notated u  v Self loops are allowed.

G RAPH V ARIATIONS More variations: A weighted graph associates weights with either the edges or the vertices E.g., a road map: edges might be weighted w/ distance A multigraph allows multiple edges between the same vertices E.g., the call graph in a program (a function can get called from multiple points in another function)

G RAPHS We will typically express running times in terms of |E| and |V| (often dropping the |’s) If |E|  |V| 2 the graph is dense If |E|  |V| the graph is sparse If you know you are dealing with dense or sparse graphs, different data structures may make sense

R EPRESENTING G RAPHS Assume V = {1, 2, …, n } An adjacency matrix represents the graph as a n x n matrix A: A[ i, j ] = 1 if edge ( i, j )  E (or weight of edge) = 0 if edge ( i, j )  E

G RAPHS : A DJACENCY M ATRIX Example: a d bc A ?? 4

G RAPHS : A DJACENCY M ATRIX Example: a d bc A

G RAPHS : A DJACENCY M ATRIX Space:  ( V 2 ). Not memory efficient for large graphs. Time: to list all vertices adjacent to u :  ( V ). Time: to determine if ( u, v)  E :  ( 1 ).

G RAPHS : A DJACENCY M ATRIX The adjacency matrix is a dense representation Usually too much storage for large graphs But can be very efficient for small graphs Most large interesting graphs are sparse E.g., planar graphs, in which no edges cross, have |E| = O(|V|) by Euler’s formula For this reason the adjacency list is often a more appropriate respresentation

G RAPHS : A DJACENCY L IST Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: Adj[1] = {2,3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3} Variation: can also keep a list of edges coming into vertex

G RAPHS : A DJACENCY L IST For directed graphs: Sum of lengths of all adj. lists is  out-degree( v ) = | E | v  V Total storage:  ( V + E ) For undirected graphs: Sum of lengths of all adj. lists is  degree( v ) = 2| E | v  V Total storage:  ( V + E ) No. of edges leaving v No. of edges incident on v. Edge ( u, v ) is incident on vertices u and v.

G RAPH D EFINITIONS Path Sequence of nodes n 1, n 2, … n k Edge exists between each pair of nodes n i, n i+1 Example A, B, C is a path

G RAPH D EFINITIONS Path Sequence of nodes n 1, n 2, … n k Edge exists between each pair of nodes n i, n i+1 Example A, B, C is a path A, E, D is not a path

G RAPH D EFINITIONS Cycle Path that ends back at starting node Example A, E, A

G RAPH D EFINITIONS Cycle Path that ends back at starting node Example A, E, A A, B, C, D, E, A Simple path No cycles in path Acyclic graph No cycles in graph

G RAPH S EARCHING Given: a graph G = (V, E), directed or undirected Goal: methodically explore every vertex and every edge Ultimately: build a tree on the graph Pick a vertex as the root Choose certain edges to produce a tree Note: might also build a forest if graph is not connected

B READTH -F IRST S EARCH “Explore” a graph, turning it into a tree One vertex at a time Expand frontier of explored vertices across the breadth of the frontier Builds a tree over the graph Pick a source vertex to be the root Find (“discover”) its children, then their children, etc.

B READTH -F IRST S EARCH Input: Graph G = ( V, E ), either directed or undirected, and source vertex s  V. Output: d [ v ] = distance (smallest # of edges, or shortest path) from s to v, for all v  V. d [ v ] =  if v is not reachable from s.  [ v ] = u such that ( u, v ) is last edge on shortest path s v. u is v ’s predecessor. Builds breadth-first tree with root s that contains all reachable vertices.

B READTH -F IRST S EARCH Associate vertex “colors” to guide the algorithm White vertices have not been discovered All vertices start out white Grey vertices are discovered but not fully explored They may be adjacent to white vertices Black vertices are discovered and fully explored They are adjacent only to black and gray vertices Explore vertices by scanning adjacency list of grey vertices

BFS(G,s) 1.for each vertex u in V[G] – {s} 2 color [ u ]  white 3 d [ u ]   4  [ u ]  nil 5color[ s ]  gray 6d[ s ]  0 7  [ s ]  nil 8 Q   9enqueue( Q,s) 10 while Q   11u  dequeue(Q) 12 for each v in Adj[ u ] 13 if color[ v ] = white 14color[ v ]  gray 15 d [ v ]  d [ u ]  [ v ]  u 17enqueue( Q, v ) 18color[ u ]  black BFS(G,s) 1.for each vertex u in V[G] – {s} 2 color [ u ]  white 3 d [ u ]   4  [ u ]  nil 5color[ s ]  gray 6d[ s ]  0 7  [ s ]  nil 8 Q   9enqueue( Q,s) 10 while Q   11u  dequeue(Q) 12 for each v in Adj[ u ] 13 if color[ v ] = white 14color[ v ]  gray 15 d [ v ]  d [ u ]  [ v ]  u 17enqueue( Q, v ) 18color[ u ]  black white: undiscovered gray: discovered black: finished Q : a queue of discovered vertices color[ v ]: color of v d[ v ]: distance from s to v  [ u ]: predecessor of v initialization access source s

B READTH -F IRST S EARCH : E XAMPLE         rstu vwxy

  0      rstu vwxy s Q:

B READTH -F IRST S EARCH : E XAMPLE 1  0 1     rstu vwxy w Q: r

B READTH -F IRST S EARCH : E XAMPLE 1    rstu vwxy r Q: tx

B READTH -F IRST S EARCH : E XAMPLE   rstu vwxy Q: txv

B READTH -F IRST S EARCH : E XAMPLE  rstu vwxy Q: xvu

B READTH -F IRST S EARCH : E XAMPLE rstu vwxy Q: vuy

B READTH -F IRST S EARCH : E XAMPLE rstu vwxy Q: uy

B READTH -F IRST S EARCH : E XAMPLE rstu vwxy Q: y

B READTH -F IRST S EARCH : E XAMPLE rstu vwxy Q: Ø

A NALYSIS OF BFS Initialization takes O (| V| ). Traversal Loop After initialization, each vertex is enqueued and dequeued at most once, and each operation takes O (1). So, total time for queuing is O (| V| ). The adjacency list of each vertex is scanned at most once. The total time spent in scanning adjacency lists is O (| E| ). Summing up over all vertices => total running time of BFS is O (| V| + |E| )

B READTH - FIRST T REE For a graph G = ( V, E ) with source s, the predecessor subgraph of G is G  = ( V , E  ) where V  ={ v  V :  [ v ]  n il }  { s } E  ={(  [ v ], v )  E : v  V  - { s }} The predecessor subgraph G  is a breadth-first tree if: V  consists of the vertices reachable from s and for all v  V , there is a unique simple path from s to v in G  that is also a shortest path from s to v in G. The edges in E  are called tree edges. | E  | = | V  | - 1.

D EPTH - FIRST S EARCH (DFS) Explore edges out of the most recently discovered vertex v. When all edges of v have been explored, backtrack to explore other edges leaving the vertex from which v was discovered (its predecessor ). “Search as deep as possible first.” Continue until all vertices reachable from the original source are discovered. If any undiscovered vertices remain, then one of them is chosen as a new source and search is repeated from that source.

D EPTH - FIRST S EARCH Input: G = ( V, E ), directed or undirected. No source vertex given! Output: 2 timestamps on each vertex. d [ v ] = discovery time ( v turns from white to gray) f [ v ] = finishing time ( v turns from gray to black)  [ v ] : predecessor of v = u, such that v was discovered during the scan of u ’s adjacency list. Depth-first forest

D EPTH - FIRST S EARCH Coloring scheme for vertices as BFS. A vertex is “discovered” the first time it is encountered during the search. A vertex is “finished” if it is a leaf node or all vertices adjacent to it have been finished. White before discovery, gray while processing and black when finished processing GRAY WHITE BLACK 02Vd[u]f[u] 1 ≤ d[u] < f [u] ≤ 2 |V|

DFS( G ) 1. for each vertex u  V [ G ] 2. do color [ u ]  white 3.  [ u ]  NIL 4. time  0 5. for each vertex u  V [ G ] 6. do if color [ u ] = white 7. then DFS-Visit( u ) DFS( G ) 1. for each vertex u  V [ G ] 2. do color [ u ]  white 3.  [ u ]  NIL 4. time  0 5. for each vertex u  V [ G ] 6. do if color [ u ] = white 7. then DFS-Visit( u ) Uses a global timestamp time. DFS-Visit( u ) 1. color [ u ]  GRAY // White vertex u has been discovered 2. time  time d [ u ]  time 4. for each v  Adj [ u ] 5. do if color [ v ] = WHITE 6. then  [ v ]  u 7. DFS-Visit( v ) 8. color [ u ]  BLACK // Blacken u ; it is finished. 9. f [ u ]  time  time + 1 DFS-Visit( u ) 1. color [ u ]  GRAY // White vertex u has been discovered 2. time  time d [ u ]  time 4. for each v  Adj [ u ] 5. do if color [ v ] = WHITE 6. then  [ v ]  u 7. DFS-Visit( v ) 8. color [ u ]  BLACK // Blacken u ; it is finished. 9. f [ u ]  time  time + 1 P SEUDOCODE

DFS E XAMPLE

1 | | | | | | | | d f

DFS E XAMPLE 1 | | | | | | 2 | | d f

DFS E XAMPLE 1 | | | | |3 | 2 | | d f

DFS E XAMPLE 1 | | | | |3 | 4 2 | | d f

DFS E XAMPLE 1 | | | |5 |3 | 4 2 | | d f

DFS E XAMPLE 1 | | | |5 | 63 | 4 2 | | d f

DFS E XAMPLE 1 |8 | | |5 | 63 | 4 2 | 7 | d f

DFS E XAMPLE 1 |8 | | |5 | 63 | 4 2 | 7 | d f

DFS E XAMPLE 1 |8 | | |5 | 63 | 4 2 | 79 | d f

DFS E XAMPLE 1 |8 | | |5 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |8 |11 | |5 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |128 |11 | |5 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |128 |1113| |5 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |128 |1113| 14|5 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |128 |1113| 14|155 | 63 | 4 2 | 79 |10 d f

DFS E XAMPLE 1 |128 |1113|16 14|155 | 63 | 4 2 | 79 |10 d f

A NALYSIS OF DFS Loops on lines 1-3 & 5-7 take  ( V ) time, excluding time to execute DFS-Visit. DFS-Visit is called once for each white vertex v  V when it’s painted gray the first time. Lines 4-7 of DFS-Visit is executed |Adj[ v ]| times. The total cost of executing DFS-Visit is  v  V |Adj[ v ]| =  ( E ) Total running time of DFS is  (| V| + |E| ).

D EPTH -F IRST T REES Predecessor subgraph defined slightly different from that of BFS. The predecessor subgraph of DFS is G  = ( V, E  ) where E  ={(  [ v ], v ) : v  V and  [ v ]  nil }. How does it differ from that of BFS? The predecessor subgraph G  forms a depth-first forest composed of several depth-first trees. The edges in E  are called tree edges.

T IME -S TAMP S TRUCTURE IN DFS There is also a nice structure to the time stamps, which is referred to as Parenthesis Structure. Theorem 22.7 For all u, v, exactly one of the following holds: 1. d [ u ] < f [ u ] < d [ v ] < f [ v ] or d [ v ] < f [ v ] < d [ u ] < f [ u ] and neither u nor v is a descendant of the other. 2. d [ u ] < d [ v ] < f [ v ] < f [ u ] and v is a descendant of u. 3. d [ v ] < d [ u ] < f [ u ] < f [ v ] and u is a descendant of v. Theorem 22.7 For all u, v, exactly one of the following holds: 1. d [ u ] < f [ u ] < d [ v ] < f [ v ] or d [ v ] < f [ v ] < d [ u ] < f [ u ] and neither u nor v is a descendant of the other. 2. d [ u ] < d [ v ] < f [ v ] < f [ u ] and v is a descendant of u. 3. d [ v ] < d [ u ] < f [ u ] < f [ v ] and u is a descendant of v.

T IME -S TAMP S TRUCTURE IN DFS  So d [ u ] < d [ v ] < f [ u ] < f [ v ] cannot happen.  Like parentheses:  OK: ( ) [ ] ( [ ] ) [ ( ) ]  Not OK: ( [ ) ] [ ( ] ) Corollary v is a proper descendant of u if and only if d [ u ] < d [ v ] < f [ v ] < f [ u ].

59 S OME A PPLICATIONS OF BFS AND DFS BFS To find the shortest path from a vertex s to a vertex v in an unweighted graph To find the length of such a path Find the bipartiteness of a graph. DFS To find a path from a vertex s to a vertex v. To find the length of such a path. To find out if a graph contains cycles

A PPLICATION OF DFS: D ETECTING C YCLE FOR D IRECTED G RAPH DFS_visit( u ) color( u ) ← GRAY d[ u ] ← time ← time + 1 for each v adjacent to u do if color[ v ] ← GRAY then return "cycle exists" else if color[ v ] ← WHITE then predecessor[ v ] ← u DFS_visit( v ) color[u] ← BLACK f[u] ← time ← time + 1

A PPLICATION OF DFS: D ETECTING C YCLE FOR U NDIRECTED G RAPH DFS_visit( u ) color( u ) ← GRAY d[ u ] ← time ← time + 1 for each v adjacent to u do if color[ v ] ← GRAY and  [ u ] ≠ v then return "cycle exists" else if color[ v ] ← WHITE then predecessor[ v ] ← u DFS_visit( v ) color[u] ← BLACK f[u] ← time ← time + 1

T OPOLOGICAL S ORT Want to “sort” a directed acyclic graph (DAG). B E D C A C E D A B Think of original DAG as a partial order. Want a total order that extends this partial order.

T OPOLOGICAL S ORT Performed on a DAG. Linear ordering of the vertices of G such that if ( u, v )  E, then u appears somewhere before v. Topological-Sort ( G ) 1.call DFS ( G ) to compute finishing times f [ v ] for all v  V 2.as each vertex is finished, insert it onto the front of a linked list 3. return the linked list of vertices Topological-Sort ( G ) 1.call DFS ( G ) to compute finishing times f [ v ] for all v  V 2.as each vertex is finished, insert it onto the front of a linked list 3. return the linked list of vertices Time:  ( V + E ).

E XAMPLE Linked List: A B D C E 1/

E XAMPLE Linked List: A B D C E 1/ 2/

E XAMPLE Linked List: A B D C E 1/ 2/3 E

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/ 6/

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/ 6/7 C

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/8 6/7 C 5/8 B

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/8 6/7 C 5/8 B 9/

E XAMPLE Linked List: A B D C E 1/4 2/3 E 1/4 D 5/8 6/7 C 5/8 B 9/10 A

P RECEDENCE E XAMPLE Tasks that have to be done to eat breakfast: get glass, pour juice, get bowl, pour cereal, pour milk, get spoon, eat. Certain events must happen in a certain order (ex: get bowl before pouring milk) For other events, it doesn't matter (ex: get bowl and get spoon)

P RECEDENCE E XAMPLE get glass pour juice get bowl pour cereal pour milk get spoon eat breakfast Order: glass, juice, bowl, cereal, milk, spoon, eat.

P RECEDENCE E XAMPLE Topological Sort eat juice glass milk cereal bowl spoon consider reverse order of finishing times: spoon, bowl, cereal, milk, glass, juice, eat

P RECEDENCE E XAMPLE What if we started with juice? eat juice glass milk cereal bowl spoon consider reverse order of finishing times: spoon, bowl, cereal, milk, glass, juice, eat

W HY A CYCLIC ? Why must directed graph by acyclic for the topological sort problem? Otherwise, no way to order events linearly without violating a precedence constraint.

C ORRECTNESS P ROOF Show if ( u, v )  E, then f [ v ] < f [ u ]. When we explore ( u, v ), what are their colors? Note, u is gray – we are exploring it Is v gray? No, because then v would be an ancestor of u.  ( u, v ) is a back edge.  a cycle (dag has no back edges). Is v white? Then v becomes descendant of u. By parenthesis theorem, d [ u ] < d [ v ] < f [ v ] < f [ u ]. Is v black? Then v is already finished. Since we’re exploring ( u, v ), we have not yet finished u. Therefore, f [ v ] < f [ u ].

S TRONGLY C ONNECTED C OMPONENTS G is strongly connected if every pair ( u, v ) of vertices in G is reachable from one another. A strongly connected component ( SCC ) of G is a maximal set of vertices C  V such that for all u, v  C, both u v and v u exist.

C OMPONENT G RAPH G SCC = ( V SCC, E SCC ). V SCC has one vertex for each SCC in G. E SCC has an edge if there’s an edge between the corresponding SCC’s in G. G SCC for the example considered:

G SCC IS A DAG Proof: Suppose there is a path v v in G. Then there are paths u u v and v v u in G. Therefore, u and v are reachable from each other, so they are not in separate SCC’s. Lemma Let C and C be distinct SCC’s in G, let u, v  C, u, v  C, and suppose there is a path u u in G. Then there cannot also be a path v v in G. Lemma Let C and C be distinct SCC’s in G, let u, v  C, u, v  C, and suppose there is a path u u in G. Then there cannot also be a path v v in G.

T RANSPOSE OF A D IRECTED G RAPH G T = transpose of directed G. G T = ( V, E T ), E T = {( u, v ) : ( v, u )  E }. G T is G with all edges reversed. Can create G T in Θ ( V + E ) time if using adjacency lists. G and G T have the same SCC’s. ( u and v are reachable from each other in G if and only if reachable from each other in G T.)

SCC E XAMPLE h fae g cbd four SCCs

H OW C AN DFS H ELP ? Suppose we run DFS on the directed graph. All vertices in the same SCC are in the same DFS tree. But there might be several different SCCs in the same DFS tree. Example: start DFS from vertex h in previous graph

M AIN I DEA OF SCC A LGORITHM DFS tells us which vertices are reachable from the roots of the individual trees Also need information in the "other direction": is the root reachable from its descendants? Run DFS again on the "transpose" graph (reverse the directions of the edges)

A LGORITHM TO DETERMINE SCC S SCC ( G ) 1. call DFS ( G ) to compute finishing times f [ u ] for all u 2. compute G T 3. call DFS ( G T ), but in the main loop, consider vertices in order of decreasing f [ u ] (as computed in first DFS) 4. output the vertices in each tree of the depth-first forest formed in second DFS as a separate SCC SCC ( G ) 1. call DFS ( G ) to compute finishing times f [ u ] for all u 2. compute G T 3. call DFS ( G T ), but in the main loop, consider vertices in order of decreasing f [ u ] (as computed in first DFS) 4. output the vertices in each tree of the depth-first forest formed in second DFS as a separate SCC Time:  ( V + E ).

E XAMPLE abcd efgh 1/ 2/3/465/7 8/11/ 12/ 13/ f4f4 h6h6 g7g7 d9d9 c 10 a 14 e 15 b 16 DFS on the initial graph G DFS on G T: start at b: visit a, e start at c: visit d start at g: visit f start at h Strongly connected components: C 1 = {a, b, e}, C 2 = {c, d}, C 3 = {f, g}, C 4 = {h} abcd efgh

C OMPONENT G RAPH The component graph G SCC = (V SCC, E SCC ): V SCC = { v 1, v 2, …, v k }, where v i corresponds to each strongly connected component C i There is an edge ( v i, v j )  E SCC if G contains a directed edge ( x, y ) for some x  C i and y  C j The component graph is a DAG abcd efgh a b e c d f gh

N OTATIONS Extend notation for d (starting time) and f (finishing time) to sets of vertices U  V: d(U) = min u  U { d[u] } (earliest discovery time) f(U) = max u  U { f[u] } (latest finishing time) abcd efgh 1/ 2/3/465/7 8/11/ 12/ 13/ C1C1 C2C2 C3C3 C4C4 d(C 1 ) f(C 1 ) d(C 2 ) f(C 2 ) d(C 3 ) f(C 3 ) d(C 4 ) f(C 4 ) =11 =16 =1 =10 =2 =7 =5 =6

SCC S AND DFS FINISHING TIMES Proof: Case 1: d ( C ) < d ( C ) Let x be the first vertex discovered in C. At time d [ x ], all vertices in C and C are white. Thus, there exist paths of white vertices from x to all vertices in C and C. By the white-path theorem, all vertices in C and C are descendants of x in depth-first tree. By the parenthesis theorem, f [ x ] = f ( C ) > f ( C ). Lemma Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E such that u  C and v  C. Then f ( C ) > f ( C ). Lemma Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E such that u  C and v  C. Then f ( C ) > f ( C ). C C u v x

SCC S AND DFS FINISHING TIMES Proof: Case 2: d ( C ) > d ( C ) Let y be the first vertex discovered in C. At time d [ y ], all vertices in C are white and there is a white path from y to each vertex in C  all vertices in C become descendants of y. Again, f [ y ] = f ( C ). At time d [ y ], all vertices in C are also white. By earlier lemma, since there is an edge ( u, v), we cannot have a path from C to C. So no vertex in C is reachable from y. Therefore, at time f [ y ], all vertices in C are still white. Therefore, for all w  C, f [ w ] > f [ y ], which implies that f ( C ) > f ( C ). Lemma Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E such that u  C and v  C. Then f ( C ) > f ( C ). Lemma Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E such that u  C and v  C. Then f ( C ) > f ( C ). C C u v y x

SCC S AND DFS FINISHING TIMES Proof: ( u, v)  E T  (v, u )  E. Since SCC’s of G and G T are the same, f ( C ) > f ( C ), by Lemma Corollary Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E T, where u  C and v  C. Then f ( C ) < f ( C ). Corollary Let C and C be distinct SCC’s in G = ( V, E ). Suppose there is an edge ( u, v)  E T, where u  C and v  C. Then f ( C ) < f ( C ).

C ORRECTNESS OF SCC When we do the second DFS, on G T, we start with a component C such that f(C) is maximum (b, in our case) We start from b and visit all vertices in C 1 From corollary: f(C) > f(C’) in G for all C  C’  there are no edges from C to any other SCCs in G T  DFS will visit only vertices in C 1  The depth-first tree rooted at b contains exactly the vertices of C 1 C1C1 C2C2 C3C3 C4C4 abcd efgh f4f4 h6h6 g7g7 d9d9 c 10 a 14 e 15 b 16

C ORRECTNESS OF SCC The next root chosen in the second DFS is in SCC C 2 such that f(C) is maximum over all SCC’s other than C 1 DFS visits all vertices in C 2 the only edges out of C 2 go to C 1, which we’ve already visited  The only tree edges will be to vertices in C 2 Each time we choose a new root it can reach only: vertices in its own component vertices in components already visited C1C1 C2C2 C3C3 C4C4 abcd efgh f4f4 h6h6 g7g7 d9d9 c 10 a 14 e 15 b 16

R UNNING T IME OF SCC A LGORITHM Step 1: O(V+E) to run DFS Step 2: O(V+E) to construct transpose graph, assuming adjacency list rep. Step 3: O(V+E) to run DFS again Step 4: O(V) to output result Total: O(V+E)

T HE E ND