Algorithm Course Dr. Aref Rashad Part: 5 Graph Algorithms February 2013 Algorithms Course ..... Dr. Aref Rashad
Outline Review of Graphs Minimum Spanning Trees Merge Sort 6/1/2019 9:17 AM Outline Review of Graphs Minimum Spanning Trees Prim and Kruskal Greedy Algorithms Shortest Path Djikstra Greedy Algorithm Dynamic Programming
Graphs A graph is a pair (V, E), where Merge Sort 6/1/2019 9:17 AM Graphs A graph is a pair (V, E), where V is a set of nodes, called vertices E is a collection of pairs of vertices, called edges Thus |E| = O(|V|2)
Graph Applications Graph Nodes Edges Transportation networks Merge Sort 6/1/2019 9:17 AM Graph Applications Graph Nodes Edges Transportation networks street intersections highways communication computers fiber optic cables World Wide Web web pages Hyperlinks social people Relationships food web species predator-prey software systems functions Precedence Scheduling tasks Constraints circuits gates Wires
Graphs We will typically express running times in terms of |E| and |V| Merge Sort 6/1/2019 9:17 AM Graphs We will typically express running times in terms of |E| and |V| If |E| |V|2 the graph is dense If |E| |V| the graph is sparse If you know you are dealing with dense or sparse graphs, different data structures may make sense
A connected graph has a path from every vertex to every other Merge Sort 6/1/2019 9:17 AM UnConnected Graphs Connected Graphs A connected graph has a path from every vertex to every other
Undirected graph. G = (V, E) V = nodes. Merge Sort 6/1/2019 9:17 AM Undirected graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = |V|, m = |E|. V = { 1, 2, 3, 4, 5, 6, 7, 8 } E = { 1-2, 1-3, 2-3, 2-4, 2-5, 3-5, 3-7, 3-8, 4-5, 5-6 } n = 8 m = 11
E = edges between pairs of nodes. Merge Sort 6/1/2019 9:17 AM Directed graph. G = (V, E) V = nodes. E = edges between pairs of nodes. Captures pairwise relationship between objects. Graph size parameters: n = |V|, m = |E|. (u, v) is edge from u to v, denoted as u v
Complete Graph Bipartite Graph Merge Sort 6/1/2019 9:17 AM Complete Graph Bipartite Graph A Bipartite graph is a graph that has its set of vertices positioned into two subsets, one of size m and the other of size n. Some or all possible edges from one subset to the other are present, but there are no edges between the vertices of the same subset. A complete graph is a graph in which all possible edges are present.
Merge Sort 6/1/2019 9:17 AM Cycles Def. A cycle is a path v1, v2, …, vk-1, vk in which v1 = vk, k > 2, and the first k-1 nodes are all distinct. cycle C = 1-2-4-5-3-1
Merge Sort 6/1/2019 9:17 AM Trees Def. An undirected graph is a tree if it is connected and does not contain a cycle. Theorem. Let G be an undirected graph on n nodes. Any two of the following statements imply the third. G is connected. G does not contain a cycle. G has n-1 edges.
Trees (free) tree - connected graph without cycles Merge Sort 6/1/2019 9:17 AM Trees (free) tree - connected graph without cycles • forest - collection of trees
Representation of Graphs Two standard ways. Adjacency Lists. Adjacency Matrix. a d c b a d c b 1 2 3 4 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0
Graphs: Adjacency Matrix Merge Sort 6/1/2019 9:17 AM Graphs: Adjacency Matrix A 1 2 3 4 ?? 1 a 2 d 4 b c 3
Graphs: Adjacency Matrix Merge Sort 6/1/2019 9:17 AM Graphs: Adjacency Matrix Example: A 1 2 3 4 1 a 2 d 4 b c 3 Required storage of the adjacency matrix … O(V2)
Graphs: Adjacency Matrix Merge Sort 6/1/2019 9:17 AM Graphs: Adjacency Matrix The adjacency matrix is a dense representation Usually too much storage for large graphs But can be very efficient for small graphs Most large interesting graphs are sparse For this reason the adjacency list is often a more appropriate representation; Adjacency lists take O(V+E) storage
Graphs: Adjacency List Merge Sort 6/1/2019 9:17 AM Graphs: Adjacency List Adjacency list: for each vertex v V, store a list of vertices adjacent to v Example: Adj[1] = {2,3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3} Variation: can also keep a list of edges coming into vertex 1 2 4 3
Graph-searching Algorithms Merge Sort 6/1/2019 9:17 AM Graph-searching Algorithms Searching a graph: Systematically follow the edges of a graph to visit the vertices of the graph. Used to discover the structure of a graph. Standard graph-searching algorithms. Breadth-first Search (BFS). Depth-first Search (DFS).
Breadth First Search Application1: Given the following state space (tree search), give the sequence of visited nodes when using BFS (assume that the nodeO is the goal state): A B C E D F G H I J K L O M N
Breadth First Search A, A B C E D F G H I J K L O M N
Breadth First Search A, B, A B C E D F G H I J K L O M N
Breadth First Search A, B,C A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F, A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G,H A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G,H,I A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, K, A B C E D F G H I J K L O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, K,L A B C E D F G H I J K O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, K,L, M, A B C E D F G H I O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, K,L, M,N, A B C E D F G H O M N
Breadth First Search A, B,C,D,E, F,G,H,I,J, K,L, M,N, Goal state: O A
Breadth First Search The returned solution is the sequence of operators in the path: A, B, G, L, O A B C E D F G H I J K L O M N
Algorithms Course ..... Dr. Aref Rashad Starting Breadth First Search For any state s that we’ve labeled, we’ll remember: •previous(s) as the previous state on a shortest path from START state to s. On the kth iteration of the algorithm we’ll begin with Vk defined as the set of those states for which the shortest path from the start costs exactly k steps Then, during that iteration, we’ll compute Vk+1, defined as the set of those states for which the shortest path from the start costs exactly k+1 steps We begin with k = 0, V0 = {START} and we’ll define, previous(START) = NULL Then we’ll add in things one step from the START into V1. And we’ll keep going. February 2013 Algorithms Course ..... Dr. Aref Rashad
Algorithms Course ..... Dr. Aref Rashad Breadth First Details • It is fine for there to be more than one goal state. • It is fine for there to be more than one start state. • This algorithm works forwards from the start. Any algorithm which works forwards from the start is said to be forward chaining. • You can also work backwards from the goal. This algorithm is very similar to Dijkstra’s algorithm. • Any algorithm which works backwards from the goal is said to be backward chaining. Notice that BFS finds the shortest path in terms of number of transitions. It does not find the least-cost path. February 2013 Algorithms Course ..... Dr. Aref Rashad
Algorithms Course ..... Dr. Aref Rashad February 2013 Algorithms Course ..... Dr. Aref Rashad
Depth First Search (DFS) Given the following state space (tree search), give the sequence of visited nodes when using DFS (assume that the nodeO is the goal state): A B C E D F G H I J K L O M N
Depth First Search (DFS) B C E D F G H I J K L O M N
Depth First Search (DFS) A,B, A B C E D F G H I J K L O M N
Depth First Search (DFS) A,B,F, A B C E D F G H I J K L O M N
Depth First Search (DFS) A,B,F, G, A B C E D F G H I J K L O M N
Depth First Search (DFS) A,B,F, G,K, A B C E D F G H I J K L O M N
Depth First Search (DFS) A,B,F, G,K, L, A B C E D F G H I J K L O M N
Depth First Search (DFS) A,B,F, G,K, L, O: Goal State A B C E D F G H I J K L O M N
Depth First Search (DFS) The returned solution is the sequence of operators in the path: A, B, G, L, O A B C E D F G H I J K L O M N
Algorithms Course ..... Dr. Aref Rashad February 2013 Algorithms Course ..... Dr. Aref Rashad
Algorithms Course ..... Dr. Aref Rashad February 2013 Algorithms Course ..... Dr. Aref Rashad
Merge Sort Prim’s Algorithm Kruskal’s Algorithm 6/1/2019 9:17 AM Minimum Spanning Tree Two Greedy Algorithms: Prim’s Algorithm Kruskal’s Algorithm
The Scenario Construct a telephone network… Merge Sort 6/1/2019 9:17 AM The Scenario Construct a telephone network… We’ve got to connect many cities together Each city must be connected We want to minimize the total cable used
Required: Connect all nodes at minimum cost. Merge Sort 6/1/2019 9:17 AM Minimum Spanning Tree Required: Connect all nodes at minimum cost. 1 3 2
Required: Connect all nodes at minimum cost. Merge Sort 6/1/2019 9:17 AM Minimum Spanning Tree Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 1 3 3 2 2
Minimum Spanning Tree Required: Connect all nodes at minimum cost. Merge Sort 6/1/2019 9:17 AM Minimum Spanning Tree Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 1 3 3 2 2 S.T. = 3 S.T. = 5 S.T. = 4
Minimum Spanning Tree Required: Connect all nodes at minimum cost. Merge Sort 6/1/2019 9:17 AM Minimum Spanning Tree Required: Connect all nodes at minimum cost. Cost is sum of edge weights 1 1 1 3 3 3 2 2 2 M.S.T. = 3 S.T. = 5 S.T. = 4
Required: Connect all nodes at minimum cost. Merge Sort 6/1/2019 9:17 AM Minimum Spanning Tree Required: Connect all nodes at minimum cost. Cost is sum of edge weights Can start at any node Unique solution. Two algorithms Prim’s Kruskal’s
Finding a MST Principal greedy methods: algorithms by Prim and Kruskal Merge Sort 6/1/2019 9:17 AM Finding a MST Principal greedy methods: algorithms by Prim and Kruskal Prim Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree Intermediary solution is a subtree Kruskal Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far Intermediary solution is a spanning forest
All its edges are candidates. Merge Sort 6/1/2019 9:17 AM Prim’s Algorithm Start with any vertex. All its edges are candidates. Stop looking when all vertices are picked Otherwise repeat… Pick minimum edge (no cycles) and connect the adjacent vertex Add all edges from that new vertex to our “candidates”
Prim’s Algorithm Pick any vertex and add it to “vertices” list Loop Merge Sort 6/1/2019 9:17 AM Prim’s Algorithm Pick any vertex and add it to “vertices” list Loop Exit if (“vertices” contains all the vertices in graph) Select shortest, unmarked edge coming from “vertices” list If (shortest edge does NOT create a cycle) then Add that edge to your “edges” Add the adjoining vertex to the “vertices” list End if Mark that edge as having been considered End loop
Merge Sort 6/1/2019 9:17 AM Example 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM Start 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 NO! 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 LOOP 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 Prim’s: MST = 66 3 9 2 7 6 10 9 11
Merge Sort 6/1/2019 9:17 AM Prim’s Algorithm There are n=|V| ExtractMin calls and m=|E| DecreaseKey calls. What will be the running time? A: Depends on queue binary heap: O((n+m)lg n) = O(E lg V) Fibonacci heap: O(V lg V + E)
An Alternate Algorithm Kruskal’s Algorithm Merge Sort 6/1/2019 9:17 AM An Alternate Algorithm Kruskal’s Algorithm
Kruskal’s Algorithm Sort edges in graph in increasing order Merge Sort 6/1/2019 9:17 AM Kruskal’s Algorithm Sort edges in graph in increasing order Select the shortest edge Loop Exit if all edges examined Select next shortest edge (if no cycle created) Mark this edge as examined Endloop This guarantees an MST, but as it is built, edges do not have to be connected
Merge Sort 6/1/2019 9:17 AM Example 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM Sort the Edges 2, 3, 4, 5, 6, 7, 8, 9, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 24
Merge Sort 6/1/2019 9:17 AM 4 2 7 5 18 6 9 3 15 14 19 8 10 12 17 11 13 20 21 24
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 Still NO! 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM Or
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 18 2 7 14 15 12 21 24 13 20 8 6 10 9 17 19 11
Merge Sort 6/1/2019 9:17 AM 4 5 3 9 2 7 Kruskal’s: MST = 66 6 10 9 11
Merge Sort 6/1/2019 9:17 AM 4 5 Prim’s: MST = 66 3 9 2 7 6 10 9 11
Kruskal’s Algorithm the algorithm maintains a forest of trees Merge Sort 6/1/2019 9:17 AM Kruskal’s Algorithm the algorithm maintains a forest of trees • an edge is accepted it if connects vertices of distinct trees • we need a data structure that maintains a partition, i.e.,a collection of disjoint sets, with the following Operations:
Prim’s Algorithm: Time Complexity Using binary heaps: O(E lg V). Initialization – O(V). Building initial queue – O(V). V Extract-Min’s – O(V lgV). E Decrease-Key’s – O(E lg V). Using Fibonacci heaps: O(E + V lg V).
Kruskal’s Algorithm: Time Complexity (initialization): O(V) (sorting): O(E lg E) (set-operation): O(E log E) Total: O(E log E)
Algorithms Course ..... Dr. Aref Rashad February 2013 Algorithms Course ..... Dr. Aref Rashad
Minimum spanning trees Merge Sort 6/1/2019 9:17 AM Summary Minimum spanning trees Connect all vertices With no cycles And minimize the total edge cost Prim’s and Kruskal’s algorithm Generate minimum spanning trees Give same total cost, but may give different trees (if graph has edges with same weight)