1. Algorithm Course Dr. Aref Rashad February 20131 Algorithms Course..... Dr. Aref Rashad Part: 5 Graph Algorithms

2. Review of Graphs Minimum Spanning Trees Prim and Kruskal Greedy Algorithms Shortest Path Djikstra Greedy Algorithm Dynamic Programming Outline

3. A graph is a pair (V, E), where  V is a set of nodes, called vertices  E is a collection of pairs of vertices, called edges  Thus |E| = O(|V| 2 ) Graphs

4. GraphNodesEdges Transportation networks street intersections highways communicationcomputersfiber optic cables World Wide Webweb pagesHyperlinks socialpeopleRelationships food webspeciespredator-prey software systemsfunctionsPrecedence SchedulingtasksConstraints circuitsgatesWires Graph Applications

5. Graphs We will typically express running times in terms of |E| and |V| If |E|  |V| 2 the graph is dense If |E|  |V| the graph is sparse If you know you are dealing with dense or sparse graphs, different data structures may make sense

6. A connected graph has a path from every vertex to every other Connected Graphs UnConnected Graphs

7. Undirected graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = |V|, m = |E|. V = { 1, 2, 3, 4, 5, 6, 7, 8 } E = { 1-2, 1-3, 2-3, 2-4, 2-5, 3-5, 3-7, 3-8, 4-5, 5-6 } n = 8 m = 11

8. (u, v) is edge from u to v, denoted as u  v Directed graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = |V|, m = |E|.

9. Bipartite Graph A Bipartite graph is a graph that has its set of vertices positioned into two subsets, one of size m and the other of size n. Some or all possible edges from one subset to the other are present, but there are no edges between the vertices of the same subset. A complete graph is a graph in which all possible edges are present. Complete Graph

10. Cycles Def. A cycle is a path v 1, v 2, …, v k-1, v k in which v 1 = v k, k > 2, and the first k-1 nodes are all distinct. cycle C = 1-2-4-5-3-1

11. Trees Def. An undirected graph is a tree if it is connected and does not contain a cycle. Theorem. Let G be an undirected graph on n nodes. Any two of the following statements imply the third. G is connected. G does not contain a cycle. G has n-1 edges.

12. (free) tree - connected graph without cycles forest - collection of trees Trees

13. Representation of Graphs Two standard ways. Adjacency Lists. Adjacency Matrix. a dc b a b c d b a d dc c ab ac a dc b 12 34 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0

14. Graphs: Adjacency Matrix 1 24 3 a d bc A1234 1 2 3 ?? 4

15. Graphs: Adjacency Matrix Example: 1 24 3 a d bc A1234 10110 20010 30000 40010 Required storage of the adjacency matrix … O(V 2 )

16. The adjacency matrix is a dense representation – Usually too much storage for large graphs – But can be very efficient for small graphs Most large interesting graphs are sparse – For this reason the adjacency list is often a more appropriate representation; Adjacency lists take O(V+E) storage Graphs: Adjacency Matrix

17. Graphs: Adjacency List Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: – Adj[1] = {2,3} – Adj[2] = {3} – Adj[3] = {} – Adj[4] = {3} Variation: can also keep a list of edges coming into vertex 1 24 3

18. Graph-searching Algorithms Searching a graph: – Systematically follow the edges of a graph to visit the vertices of the graph. Used to discover the structure of a graph. Standard graph-searching algorithms. – Breadth-first Search (BFS). – Depth-first Search (DFS).

19. Breadth First Search Application1: Given the following state space (tree search), give the sequence of visited nodes when using BFS (assume that the nodeO is the goal state): A BCED FGHIJ KL O M N

20. Breadth First Search A, A BCED FGHIJ KL O M N

21. Breadth First Search A, B, A BCED FGHIJ KL O M N

22. Breadth First Search A, B,C A BCED FGHIJ KL O M N

23. A, B,C,D A BCED FGHIJ KL O M N Breadth First Search

24. A, B,C,D,E A BCED FGHIJ KL O M N Breadth First Search

25. A, B,C,D,E, F, A BCED FGHIJ KL O M N Breadth First Search

26. A, B,C,D,E, F,G A BCED FGHIJ KL O M N Breadth First Search

27. A, B,C,D,E, F,G,H A BCED FGHIJ KL O M N

28. A, B,C,D,E, F,G,H,I A BCED FGHIJ KL O M N Breadth First Search

29. A, B,C,D,E, F,G,H,I,J, A BCED FGHIJ KL O M N Breadth First Search

30. A, B,C,D,E, F,G,H,I,J, K, A BCED FGHIJ KL O M N Breadth First Search

31. A, B,C,D,E, F,G,H,I,J, K,L A BCED FGHIJ KL O M N Breadth First Search

32. A, B,C,D,E, F,G,H,I,J, K,L, M, A BCED FGHIJ KL O M N Breadth First Search

33. A, B,C,D,E, F,G,H,I,J, K,L, M,N, A BCED FGHIJ KL O M N Breadth First Search

34. A, B,C,D,E, F,G,H,I,J, K,L, M,N, Goal state: O A BCED FGHIJ KL O M N Breadth First Search

35. The returned solution is the sequence of operators in the path: A, B, G, L, O A BCED FGHIJ KL O M N Breadth First Search

36. February 2013 Algorithms Course..... Dr. Aref Rashad 36 Starting Breadth First Search For any state s that we’ve labeled, we’ll remember: previous(s) as the previous state on a shortest path from START state to s. On the k th iteration of the algorithm we’ll begin with V k defined as the set of those states for which the shortest path from the start costs exactly k steps Then, during that iteration, we’ll compute V k+1, defined as the set of those states for which the shortest path from the start costs exactly k+1 steps We begin with k = 0, V 0 = {START} and we’ll define, previous(START) = NULL Then we’ll add in things one step from the START into V 1. And we’ll keep going.

37. February 2013 Algorithms Course..... Dr. Aref Rashad 37 Breadth First Details It is fine for there to be more than one goal state. It is fine for there to be more than one start state. This algorithm works forwards from the start. Any algorithm which works forwards from the start is said to be forward chaining. You can also work backwards from the goal. This algorithm is very similar to Dijkstra’s algorithm. Any algorithm which works backwards from the goal is said to be backward chaining. Notice that BFS finds the shortest path in terms of number of transitions. It does not find the least-cost path.

38. February 2013 Algorithms Course..... Dr. Aref Rashad 38

39. Depth First Search (DFS) Given the following state space (tree search), give the sequence of visited nodes when using DFS (assume that the nodeO is the goal state): A BCED FGHIJ KL O M N

40. A, A BCED FGHIJ KL O M N Depth First Search (DFS)

41. A,B, A BCED FGHIJ KL O M N Depth First Search (DFS)

42. A,B,F, A BCED FGHIJ KL O M N Depth First Search (DFS)

43. A,B,F, G, A BCED FGHIJ KL O M N Depth First Search (DFS)

44. A,B,F, G,K, A BCED FGHIJ KL O M N Depth First Search (DFS)

45. A,B,F, G,K, L, A BCED FGHIJ KL O M N Depth First Search (DFS)

46. A,B,F, G,K, L, O: Goal State A BCED FGHIJ KL O M N Depth First Search (DFS)

47. The returned solution is the sequence of operators in the path: A, B, G, L, O A BCED FGHIJ KL O M N Depth First Search (DFS)

48. February 2013 Algorithms Course..... Dr. Aref Rashad 48

49. February 2013 Algorithms Course..... Dr. Aref Rashad 49

50. Minimum Spanning Tree Prim’s Algorithm Kruskal’s Algorithm Two Greedy Algorithms:

51. The Scenario Construct a telephone network… We’ve got to connect many cities together – Each city must be connected – We want to minimize the total cable used

52. Required: Connect all nodes at minimum cost. 1 3 2 Minimum Spanning Tree

53. Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 2 3 2 1 3 Minimum Spanning Tree

54. Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 2 3 2 1 3 S.T. = 3S.T. = 5S.T. = 4 Minimum Spanning Tree

55. Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 3 2 1 3 2 1 3 2 M.S.T. = 3S.T. = 5S.T. = 4 Minimum Spanning Tree

56. Required: Connect all nodes at minimum cost. Cost is sum of edge weights Can start at any node Unique solution. Two algorithms – Prim’s – Kruskal’s Minimum Spanning Tree

57. Finding a MST Principal greedy methods: algorithms by Prim and Kruskal Prim Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree Intermediary solution is a subtree Kruskal Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far Intermediary solution is a spanning forest

58. Prim’s Algorithm Start with any vertex. All its edges are candidates. Stop looking when all vertices are picked Otherwise repeat… – Pick minimum edge (no cycles) and connect the adjacent vertex – Add all edges from that new vertex to our “candidates”

59. Prim’s Algorithm Pick any vertex and add it to “vertices” list Loop Exit if (“vertices” contains all the vertices in graph) Select shortest, unmarked edge coming from “vertices” list If (shortest edge does NOT create a cycle) then Add that edge to your “edges” Add the adjoining vertex to the “vertices” list End if Mark that edge as having been considered End loop

60. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 20 9 17 11 13 21 24 Example

61. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 Start 20 21 24

62. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

63. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

64. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

65. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

66. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

67. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

68. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24 NO!

69. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24 LOOP

70. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

71. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

72. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

73. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

74. 4 2 7 5 6 9 3 10 9 11 Prim’s: MST = 66

75. What will be the running time? A: Depends on queue binary heap: O((n+m)lg n) = O(E lg V) Fibonacci heap: O(V lg V + E) Prim’s Algorithm There are n=|V| ExtractMin calls and m=|E| DecreaseKey calls.

76. An Alternate Algorithm Kruskal’s Algorithm

77. Kruskal’s Algorithm Sort edges in graph in increasing order Select the shortest edge Loop Exit if all edges examined Select next shortest edge (if no cycle created) Mark this edge as examined Endloop This guarantees an MST, but as it is built, edges do not have to be connected

78. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24 Example

79. Sort the Edges 2, 3, 4, 5, 6, 7, 8, 9, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 24

80. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

81. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

82. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

83. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

84. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

85. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

86. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24 Still NO!

87. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

88. Or

89. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

90. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

91. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

92. 4 2 7 5 18 6 9 3 15 14 19 8 10 12 9 17 11 13 20 21 24

93. 4 2 7 5 6 9 3 10 9 11 Kruskal’s: MST = 66

94. 4 2 7 5 6 9 3 10 9 11 Prim’s: MST = 66

95. the algorithm maintains a forest of trees an edge is accepted it if connects vertices of distinct trees we need a data structure that maintains a partition, i.e.,a collection of disjoint sets, with the following Operations: Kruskal’s Algorithm

96. Using binary heaps: O(E lg V). Initialization – O(V). Building initial queue – O(V). V Extract-Min’s – O(V lgV). E Decrease-Key’s – O(E lg V). Using Fibonacci heaps: O(E + V lg V). Prim’s Algorithm: Time Complexity

97. (initialization): O(V) (sorting): O(E lg E) (set-operation): O(E log E) Total: O(E log E) Kruskal’s Algorithm: Time Complexity

98. Summary Minimum spanning trees – Connect all vertices – With no cycles – And minimize the total edge cost Prim’s and Kruskal’s algorithm – Generate minimum spanning trees – Give same total cost, but may give different trees (if graph has edges with same weight)