1. I/O-Efficient Graph Algorithms Norbert Zeh Duke University EEF Summer School on Massive Data Sets Århus, Denmark June 26 – July 1, 2002

2. Motivation For theoreticians: Graph problems are neat, often difficult, hence interesting For practitioners: Massive graphs arise in GIS, web modelling,... Problems in computational geometry can be expressed as graph problems Many abstract problems best viewed as graph problems Extreme: Pointer-based data structures = graphs with extra information at their nodes

3. Outline Fundamental graph problems List ranking Algorithms for trees Euler tour Tree labelling Graph searching BFS/DFS Connectivity Connected components Minimum spanning tree Single source shortest paths

4. Outline Techniques and data structures Graph contraction Time-forward processing Tournament tree Buffered repository tree Lower bounds List ranking Connectivity Planar graphs

5. Introduction and “Simple” Problems List ranking Euler tour Tree labelling Evaluating directed acyclic graphs Greedy graph algorithms

6. List Ranking 123456 315231349111415

7. Why Is List Ranking Non-Trivial? 123456789101112131415161591326101434781112151615913261014347811121516159132610143478111215161256910131434781112151612569101314347811121516  The internal memory algorithm spends  (N) I/Os in the worst case.

8. An Efficient List Ranking Algorithm Assume an independent set of size at least N/3 can be found efficiently (in O(sort(N)) I/Os). 3152313174341115 315231349111415

9. An Efficient List Ranking Algorithm Compressing L: Sort elements in L \ I Sort elements in I by their successor pointers Scan the two lists to update the label of succ(v), for every element v  I The I/O-complexity of this procedure is Theorem: A list of size N can be ranked in O(sort(N)) I/Os.

10. The Euler Tour Technique Goal: Given a tree T, represent it by a list L so that certain computations on T can be performed by ranking L. r

11. The Euler Tour Technique Theorem: Given the adjacency lists of the vertices in T, an Euler tour can be constructed in O(scan(N)) I/Os. Let {v,w 1 },…,{v,w r } be the edges incident to v Then succ((w i,v)) = (v,w i+1 )) v w4w4 w3w3 w2w2 w1w1

12. Rooting a Tree Choosing a vertex r as the root of a tree T defines parent-child relationships between adjacent nodes Rooting tree T = computing for every edge {v,w} who is the parent and who is the child v = p(w) if and only if rank((v,w)) < rank((w,v)) Theorem: A tree can be rooted in O(sort(N)) I/Os.

13. Computing a Preorder Numbering Theorem: A preorder numbering of a rooted tree T can be computed in O(sort(N)) I/Os. 0 1 2 34 5 6 78 9 1 0 1 1 1 0 1 1 0 0 0 1 0 0 0 0 1 1 1 8 2 3 4 4 5 6 4 3 8 7 8 5 7 9 9 8 preorder#(v) = rank((p(v),v))

14. Computing Subtree Sizes Theorem: The nodes of T can be labelled with their subtree sizes in O(sort(N)) I/Os. 10 8 3 11 1 3 11 1 1 16 2 3 5 6 8 10 7 4 11 14 9 18 17 13 12 15

15. Evaluating a Directed Acyclic Graph More general: Given a labelling , compute a labelling  so that  (v) is computed from  (v) and  (u 1 ),…,  (u r ), where u 1,…,u r are v’s in-neighbors 0 1 0 0 1 01 0 0 0 0 10 1 1 0 0 1 0 10 1 1 00 1 0

16. Q: 0 1 0 1 2 3 4 5 6 7 8 10 9 11 12 Time-Forward Processing Assume nodes are given in topologically sorted order. 00 0 1 1 1 0 0 0 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 00 ØUse priority queue Q to send data along the edges. (6,1,0)(4,2,1) (5,2,1) (6,1,0)(4,2,1) (4,3,0) (5,2,1) (5,3,0) (6,1,0) (5,2,1) (5,3,0) (6,1,0)(5,2,1) (5,3,0) (6,1,0) (7,4,0) (8,4,0) (6,1,0) (7,4,0) (8,4,0)(6,1,0) (6,5,1) (7,4,0) (7,5,1) (8,4,0) (8,5,1) (7,4,0) (7,5,1) (8,4,0) (8,5,1)(7,4,0) (7,5,1) (8,4,0) (8,5,1) (10,6,0) (8,4,0) (8,5,1) (10,6,0)(8,4,0) (8,5,1) (9,7,1) (10,6,0) (10,7,1) (9,7,1) (10,6,0) (10,7,1)(9,7,1) (9,8,0) (10,6,0) (10,7,1) (10,6,0) (10,7,1)(10,6,0) (10,7,1) (11,9,1) (12,9,1) (11,9,1) (12,9,1)(11,9,1) (11,10,0) (12,9,1) (12,10,0) (12,9,1) (12,10,0)

17. Time-Forward Processing Analysis: Vertex set + adjacency lists scanned ØO(scan(|V| + |E|)) I/Os Priority queue: Every edge inserted into and deleted from Q exactly once ØO(|E|) priority queue operations ØO(sort(|E|)) I/Os

18. Time-Forward Processing Analysis: Vertex set + adjacency lists scanned ØO(scan(|V| + |E|)) I/Os Priority queue: Every edge inserted into and deleted from Q exactly once ØO(|E|) priority queue operations ØO(sort(|E|)) I/Os Theorem: A directed acyclic graph G = (V,E) can be evaluated in O(sort(|V| + |E|)) I/Os.

19. Maximal Independent Set (MIS) Algorithm G REEDY MIS: 1. I  0 2. for every vertex v  G do 3. if no neighbor of v is in I then 4. Add v to I 5. end if 6. end for

20. Maximal Independent Set (MIS) Algorithm G REEDY MIS: 1. I  0 2. for every vertex v  G do 3. if no neighbor of v is in I then 4. Add v to I 5. end if 6. end for Observation: It suffices to consider all neighbors of v which have been visited in a previous iteration.

21. Maximal Independent Set (MIS) 1 2 3 4 5 6 7 89 10 11 1 2 3 4 5 6

22. 1 2 3 4 5 6 7 89 10 11 1 1 2 2 33 4 4 5 5 66 7 8 789 9 10 11 Maximal Independent Set (MIS) Theorem: A maximal independent set of a graph G = (V,E) can be computed in O(sort(|V|+|E|)) I/Os.

23. Large Independent Set of a List Corollary: An independent set of size at least N/3 for a list L of size N can be found in O(sort(N)) I/Os. Every vertex in an MIS I prevents two other vertices from being in I: ØEvery MIS has size at least N/3.

24. Graph Connectivity Connected components Minimum spanning tree

25. Connectivity A Semi-External Algorithm

26. Analysis: Scan vertex set to load vertices into main memory Scan edge set to carry out algorithm O(scan(|V| + |E|)) I/Os Theorem: The connected components of a graph can be computed in O(scan(|V| + |E|)) I/Os, provided that |V|  M.

27. Connectivity The General Case Idea: If |V|  M Use semi-external algorithm If |V| > M Identify simple connected subgraphs of G Contract these subgraphs to obtain graph G’ = (V’,E’) with |V’|  c|V|, c < 1 Recursively compute connected components of G’ Obtain labelling of connected components of G from labelling of components of G’

28. A B C D E Connectivity The General Case a b c d e f g h i j k l m n A B C D E 1 1 2 2 2 1 1 1 1 1 1 2 2 2 2 2 2 2 2

29. Main steps: Find smallest neighbors (easy) Compute connected components of graph H induced by selected edges Contract each component into a single vertex (easy) Call the procedure recursively Copy label of every vertex v  G’ to all vertices in G represented by v (easy)

30. Connectivity The General Case Every connected component of H has size at least 2 Ø|V’|  |V|/2 Ø recursive calls Theorem: The connected components of a graph G = (V,E) can be computed in I/Os.

31. Connectivity The General Case Later: BFS in O(|V| + sort(|E|)) I/Os ØCan be used to identify connected components When |V| = |E|/B, algorithm takes O(sort(|E|)) I/Os Can stop recursion after recursive calls Theorem: The connected components of a graph G = (V,E) can be computed in I/Os.

32. a b c d e f g h i j k abc d j h k g e f i Biconnectivity Theorem: The biconnected components of a graph G = (V,E) can be computed in I/Os. a b c d e f g h i j k

33. Minimum Spanning Tree (MST) Observation: Connectivity algorithm can be augmented to produce a spanning tree of G. a b c d e f g h i j k l m n A B C D E

34. Minimum Spanning Tree (MST) To obtain a minimum spanning tree: Choose edge of minimum weight incident to v Some book-keeping: The weight of an edge e in the compressed graph = the min weight of all edges represented by e When “e is added” to T, add in fact this minimum edge 4 1 5 3 v a b c d

35. Minimum Spanning Tree (MST) a b c d e f g h i j k l m n A B C D E Theorem: A MST of a graph G = (V,E) can be computed in I/Os.

36. A Fast MST Algorithm Idea: Assume MST can be computed in O(|V| + sort(|E|)) I/Os Again recursion can be stopped after iterations Prim’s algorithm:

37. A Fast MST Algorithm Maintain superset of blue edges in priority queue Q When edge {v,w} of minimum weight is retrieved, test whether v,w are both in T Yes  discard edge No  Add edge to MST and add all edges incident to w to Q, except {v,w} (assuming that w  T) Problem: How to test whether v,w  T.

38. A Fast MST Algorithm If v,w  T, but {v,w}  T, then both v and w have inserted edge {v,w} into Q ØThere are two copies of {v,w} in Q They are consecutive ØPerform two D ELETE M IN operations If {v,w} = {y,z}, discard both Otherwise, add {v,w} to T and re-insert {y,z} v w

39. A Fast MST Algorithm Analysis: O(|V| + scan(|E|)) I/Os for retrieving adjacency lists O(sort(|E|)) I/Os for priority queue operations Theorem: A MST of a graph G = (V,E) can be found in O(|V| + sort(|E|)) I/Os. Corollary: A MST of a graph G = (V,E) can be found in I/Os.

40. Graph Contraction and Sparse Graphs A graph G = (V,E) is sparse if for any graph H obtainable from G through a series of edge contractions, |E(H)| = O(|V(H)|). For a sparse graph, the number of vertices and edges in G reduces by a constant factor in each iteration of the connectivity and MST algorithms. Theorem: The connected components or a MST of a sparse graph with N vertices can be computed in O(sort(N)) I/Os.

41. Three Techniques for Graph Algorithms Time-forward processing: Express graph problems as evaluation problems of DAGs Graph contraction: Reduce the size of G while maintaining the properties of interest Solve problem recursively on compressed graph Construct solution for G from solution for compressed graph Bootstrapping: Switch to generally less efficient algorithm as soon as (part of the) input is small enough