Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 12. Graph Algorithms

Prof. Amr Goneid, AUC2 Graph Algorithms

Prof. Amr Goneid, AUC3 Graph Algorithms Graph Traversal: DFS and BFS A Simple Graph Class Minimum Cost Spanning Trees (MST) Single Source Shortest Paths All Pairs Shortest Paths Transitive Closure Graph Coloring Topological Sorting The Clique Problem

Prof. Amr Goneid, AUC4 1. Graph Traversal Depth-First Search (DFS) Exhaustive Search Algorithm Visits every node in the graph by following node connections in depth. Recursive algorithm. An Array val[v] records the order in which vertices are visited. Initialized to “unseen”. Any edge to a vertex that has not been seen is followed via the recursive call.

Prof. Amr Goneid, AUC5 DFS Algorithm // Assume No. of vertices = v and order = 0 initially // val[1..v] is an array recording the visit order void DFS() { // Initialize all to unseen for each vertex k set val[k] = unseen; // Follow Nodes in Depth for each vertex k if (val[k] == unseen) Visit(k); }

Prof. Amr Goneid, AUC6 DFS Algorithm (continued) //Assume an adjacency matrix a[1..v][1..v] Visit(k) { val[k] = ++order; for each vertex (t), t =1..v if (vertices (t) and (k) are adjacent) if (val[t] == unseen) Visit(t); }

Prof. Amr Goneid, AUC7 Example A BCGF DE 1 VAL[K] ABCDEFG 1 start

Prof. Amr Goneid, AUC8 Example A BCGF DE 1 2 VAL[K] ABCDEFG 12

Prof. Amr Goneid, AUC9 Example A BCGF DE VAL[K] ABCDEFG 123

Prof. Amr Goneid, AUC10 Example A BCGF DE VAL[K] ABCDEFG

Prof. Amr Goneid, AUC11 Example A BCGF DE VAL[K] ABCDEFG

Prof. Amr Goneid, AUC12 Example A BCGF DE VAL[K] ABCDEFG

Prof. Amr Goneid, AUC13 Example A BCGF DE VAL[K] ABCDEFG

Prof. Amr Goneid, AUC14 Exercises Show that the complexity of DFS for adjacency matrix representation is T(V) = O(V 2 ) Show that the complexity of DFS for adjacency list representation is T(V) = O(V+E) Show how DFS can be used to: 1. Determine number of connected components in a graph. 2. Test if a graph has a cycle. 3. Find the number of edges in a graph. Explore the non-recursive version of the DFS using a stack. What will happen if you use a queue instead of the stack?

Prof. Amr Goneid, AUC15 Non-Recursive DFS // Assume hold = node discovered but not yet visited // Val[1..v ] is set to “unseen” initially, order = 0 // Assume a stack, initially empty DFS(k) { push (k) on top of stack; while stack is not empty { pop stack top into (k); val[k] = ++order; for vertices t = v..1 // Scan from right to left if (vertices (t) and (k) are adjacent)) if (val[t] == unseen) { push (t) on top of stack; val[t] = hold;} }

Prof. Amr Goneid, AUC16 Breadth-First Search (BFS) // Replacing the stack by a queue, gives the BFS algorithm // Also an Exhaustive Search Algorithm // Assume a queue, intially empty BFS(k) { add (k) to end of queue; while queue is not empty { remove queue front into (k); val[k] = ++order; for vertices t = 1..v // Scan from left to right if (vertices (t) and (k) are adjacent)) if (val[t] == unseen) { add (t) to end of queue; val[t] = hold;} }

Prof. Amr Goneid, AUC17 Example BFS A BCGF DE ABCDEFG

Prof. Amr Goneid, AUC18 DFS and BFS of a Tree BFS Complexity: T(V) = O(V 2 ) for adjacency matrix representation. T(V) = O(V+E) for adjacency list representation. BFS Applications Same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges For a tree structure: DFS is equivalent to Pre-order traversal BFS is equivalent to Level-order traversal

Prof. Amr Goneid, AUC19 Pre-Order Traversal DFS is also an Unbound Backtracking algorithm

Prof. Amr Goneid, AUC20 DFS Demo BFS Demo orica/animations/search.html kundan/dsal/GraphAppl.html Graph Traversal Demos

Prof. Amr Goneid, AUC21 Exercise Model the shown maze asa graph. Show how DFS can be used to find the exit. in out

Prof. Amr Goneid, AUC22 2. A Simple Graph Class To represent a weighted undirected graph with a maximum of V max vertices and E max = V max (V max -1)/2 edges. The verices are numbered 0,1,...V-1. The graph is assumed to be on a text file in the form of an adjacency matrix. The weights on the edges are assumed to be positive integers with zero weight indicating the absence of an edge. When loaded from the text file, the weights are stored in a 2-D array (AdjMatrix) representing the adjacency matrix. Another array (edges) stores the non-zero edges in the graph. An edge (u,v,w) existing between nodes (u) and (v) with weight (w) is modeled as a class (Edge).

Prof. Amr Goneid, AUC23 Edge Class // File: Edge.h // Definition of Edge class #ifndef EDGE_H #define EDGE_H typedef int weightType;// weights are positive integers class Edge { public: int u,v;weightType w; bool operator < (const Edge &e) { return (w < e.w); } bool operator <= (const Edge &e) { return (w <= e.w); } }; // end of class Edge declaration #endif // EDGE_H

Prof. Amr Goneid, AUC24 Graph Class // File: Graphs.h // Graph library header file #ifndef GRAPHS_H #define GRAPHS_H #include #include "Edge.h" using namespace std; const int Vmax = 50; // Maximum number of vertices const int Emax = Vmax*(Vmax-1)/2; // Maximum number of edges

Prof. Amr Goneid, AUC25 Graph Class class Graphs { public: Graphs();// Constructor ~Graphs();// Destructor // Map vertex number to a name (character) char Vname(const int s) const; void getGraph(string fname);// Get Graph from text File (fname) void dispGraph( ) const;// Display Ajacency Matrix int No_of_Verices( ) const;// Get number of vertices (V) int No_of_Edges( ) const;// Get Number of Non-zero edges (E) void dispEdges( ) const;// Display Graph edges void DFS( );// Depth First Search Traversal (DFS)

Prof. Amr Goneid, AUC26 Graph Class private: int V, E;// No.of vertices (V) and edges (E) weightType AdjMatrix[Vmax][Vmax];// Adjacency Matrix Edge edges[Emax];// Array of non-zero edges int order;// Order of Visit of a node in the DFS int val[Vmax];// Array holding order of traversal void getEdges();// Get edges from adjacency matrix void printEdge(Edge e) const;// Output an edge (e) void visit(int k);// Node Visit Function for DFS }; #endif // GRAPHS_H #include "Graphs.cpp"

Prof. Amr Goneid, AUC27 Graph Class The CSCI 321 web site contains an implementation of the simple graph class “Graphs”

Prof. Amr Goneid, AUC28 3. Minimum Cost Spanning Tree (MST) Consider houses A..F connected by muddy roads with the distances indicated. We want to pave some roads such that: We can reach a house from any other house via paved roads. The cost of paving is minimum. This problem is an example of finding a Minimum Spanning Tree (MST) E G A B C F D

Prof. Amr Goneid, AUC29 Minimum Spanning Tree (MST) Cost: For a weighted graph, the cost of a spanning tree is the sum of the weights of the edges in that tree. Minimum Spanning tree: A spanning tree of minimum cost For the shown graph, the minimum cost is 22 E G A B C F D

Prof. Amr Goneid, AUC30 Kruskal’s Algorithm for MST The algorithm was written by Joseph Kruskal in 1956 A Greedy Algorithm: Builds the MST edge by edge into a set of edges (T). At a given stage, chooses an edge that results in minimum increase in the sum of costs included so far in (T). The set (T) might not be a tree at all stages of the algorithm, but it can be completed into a tree iff there are no cycles in (T).

Prof. Amr Goneid, AUC31 Kruskal’s Algorithm for MST Builds up forests, then joins them in a single tree. Constraints: - The graph must be connected. - Uses exactly V-1 edges. - Excludes edges that form a cycle.

Prof. Amr Goneid, AUC32 Abstract Algorithm Form Set E of edges in increasing order of costs. Set MST T = empty Repeat Select an edge (e) from top. Delete edge from E set. Add edge (e) to (T) if it does not form a cycle, otherwise, reject. Until we have V-1 successful edges.

Prof. Amr Goneid, AUC33 Example E G A B C F D Edge uvw accept 1EF2 2AC3 3CE3 4AE4 5CD4 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC34 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 5CD4 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC35 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 5CD4 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC36 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 5CD4 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC37 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 NO 5CD4 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC38 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 NO 5CD4yes 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC39 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 NO 5CD4yes 6EG4 7DF5 8BD6

Prof. Amr Goneid, AUC40 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 NO 5CD4yes 6EG4 7DF5 No 8BD6

Prof. Amr Goneid, AUC41 Example E G A B C F D Edge uvw accept 1EF2yes 2AC3 3CE3 4AE4 NO 5CD4yes 6EG4 7DF5 NO 8BD6yes

Prof. Amr Goneid, AUC42 How to test for Cycles? Simple Union- Simple Find Given a set (1,2,..,n} initially partitioned into n disjoint sets (each element is its own parent): Find (i): return the sub-set that (i) is in (i.e. return the parent set of i). Union (k,j): combine the two sub-sets that (j) and (k) are in (make k the child of j) Union builds up a tree, while find will search for the root of the tree. Cost is O(log n)

Prof. Amr Goneid, AUC43 How to test for Cycles? Represent vertices as Disjoint Sets Initially, each node is its own parent (-1) ABCDEFG ABCDEFG Let the parent set of u be p(u)

Prof. Amr Goneid, AUC44 How to test for Cycles? When edge (E,F) is selected, we find that p(E)  p(F). So, we make a union between p(E) and p(F), i.e., p(F) becomes the child of p(E). Same for edge (A,C) A BCDEFG ABCDEFG Parent table after selecting (E,F) then (A,C)

Prof. Amr Goneid, AUC45 How to test for Cycles? When (C,E) is selected, we find that P(C)  P(E). This means that the edge will be accepted. Select (A,E), Find will give P(A) = P(E) (cycle, reject). A BC D EFG ABCDEFG Parent table after accepting (E,F) then (A,C), then (C,E), then rejecting (A,E)

Prof. Amr Goneid, AUC46 How to test for Cycles? When (C,D) is selected, we find that P(C)  P(D). This means that the edge will be accepted. Select (E,G), Find will give P(E)  P(G) (accept). A BC D EF G ABCDEFG Parent table after 5 th accepted edge

Prof. Amr Goneid, AUC47 How to test for Cycles? When (D,F) is selected, we find that P(D) = P(F). This means that the edge will be rejected. Select (B,D), Find will give P(B)  P(D) (accept). A BC D EF G ABCDEFG Parent table after 6 th accepted edge (Final MST)

Prof. Amr Goneid, AUC48 Kruskal’s Algorithm 1. Insert edges with weights into a minimum heap 2. Put each vertex in a separate set 3. i = 0 ; 4. While ((i < V-1) && (heap not empty)) { 5. Remove edge (u,v) from heap 6. Find set (j) of connected vertices having (u) 7. Find set (k) of connected vertices having (v) 8. if ( j != k ) { i++; MST [i].u = u; MST [i].v = v; MST [i].w = w; 9. Make a Union between set (j) and set (k); } }

Prof. Amr Goneid, AUC49 Kruskal’s Algorithm Demo animations/mst.html Kruskal's algorithm at work on a graph of distances between 128 North American cities. Almost imperceptively at first, short edges get added all around the continent, slowly building forests until the tree is completed. MST (Kruskal) Demo1 MST (Kruskal) Demo2

Prof. Amr Goneid, AUC50 Analysis of Kruskal’s Algorithm Line(1): generation of the minimum heap O(E log E) Line(2): O(V) Line(3): O(1) Line(5): O(V log E) = O(V log V) since E = O(V 2 ) Lines(6),(7): O(V log V) Lines(8),(9): O(V) Total: O(V) + O(V log V) + O(E log E) The algorithm is dominated by the generation of the minimum heap. Hence, T(n) = O(E log V)

Prof. Amr Goneid, AUC51 Prim’s Algorithm for MST Prim’s algorithm for Minimum Spanning Trees. The algorithm was developed in 1930 by Czech mathematician Vojtěch Jarník Prim’s algorithm and later independently by computer scientist Robert C. Prim in 1957 and rediscovered by Edsger Dijkstra in Therefore it is also sometimes called the DJP algorithm, the Jarník algorithm, or the Prim– Jarník algorithm.

Prof. Amr Goneid, AUC52 Prim’s Algorithm Choose any starting vertex. At any stage, the set of selected edges form a tree If A is the set of edges selected so far, then A is a tree. The next edge (u,v) is a minimum cost edge not in A such that A  {(u,v)} is also a tree.

Prof. Amr Goneid, AUC53 Prim’s Algorithm // Given a connected graph, a set of vertices V, and a tree T set T = {empty}, V = {first vertex}, done = false while (T contains less than V-1 edges and not done) { choose an edge (u,v) with (u) in V and (v) not in V such that (u,v) is of least cost ; if there is no such edge then done = true else { V ← V  {v}, T ← T  {(u,v)} }; }

Example c d b a c d b a c d b a c d b a c d b a Prof. Amr Goneid, AUC

55 Prim’s Algorithm Explore the differences between Prim’s Algorithm and Kruskal’s Algorithm Make an analysis of the complexity of Prim’s Algorithm and show that it is O(E log V) Prim's Algorithm Demo

Prof. Amr Goneid, AUC56 4. Single Source Shortest Paths (General) In a graph G(V,E), find shortest paths from a single source vertex (S) to all other vertices. Edsger Dijkstra published an algorithm to solve this problem in 1959.

Prof. Amr Goneid, AUC57 Shortest Paths: Dijkstra’s Algorithm Dijkstra’s Algorithm for V vertices: Uses three arrays: - Distance[i]: holds distance from (S) to vertex (i). - Processed[i]: to flag processed vertices. - Via[i]: holds index of vertex from which we can directly reach vertex (i).

Prof. Amr Goneid, AUC58 Initialization Distance[i]: = 0if S = i = W si if (S, i) are adjacent =  otherwise Processed[i] = yes if i = S, No otherwise Via[i] = S if (i, S) are adjacent, 0 otherwise

Prof. Amr Goneid, AUC59 Method S z y x j i Already Processed Distance[j] Distance[i] Wij not yet processed closest to S adjacent to j

Prof. Amr Goneid, AUC60 Dijkstra’s Algorithm (Greedy Algorithm) Repeat Find j = index of unprocessed node closest to (S) Mark (j) as now processed For each node (i) not yet processed: if (i) is adjacent to (j) then { new_distance = Distance[j] + W ij if new_distance < Distance[i] then { Distance[i] = new_distance ; Via[i] = j ; } } Until all vertices are processed

Prof. Amr Goneid, AUC61 InitialSource = A Example A E C B D A 0 A A 0 No yes 20  Dist Processed Via A B C D E

Prof. Amr Goneid, AUC62 j = B Example A E C B D A 0 A A 0 No yes 20  Dist Processed Via A B C D E

Prof. Amr Goneid, AUC63 j = Bi = D Example A E C B D A B A A 0 No yes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC64 j = E Example A E C B D A B A A 0 yesNo yes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC65 j = Ei = C Example A E C B D A B E A 0 yesNo yes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC66 j = C Example A E C B D A B E A 0 yesNoyes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC67 j = Ci = D Example A E C B D A B E A 0 yesNoyes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC68 j = D Example A E C B D A B E A 0 yes Dist Processed Via A B C D E

Prof. Amr Goneid, AUC69 Final Example A E C B D A B E A 0 yes Dist Processed Via A B C D E A →B A→E →C A→B→D A→E

Prof. Amr Goneid, AUC70 How to print the path? /* The function Vname(k) maps a vertex number to a name (e.g a character A, B,.. etc). Given the via[ ] array resulting from Dijkstra’s algorithm, the following recursive function prints the vertices on the shortest path from source (s) to destination (i). */ void Graphs::printPath(int s, int i) const { if (i == s) cout << Vname(s); else {printPath(s,via[i]); cout << Vname(i);} }

Prof. Amr Goneid, AUC71 Demo binatorica/animations/dijkstra.html Shortest Paths Demo1 Shortest Paths Demo2

Prof. Amr Goneid, AUC72 Analysis The repeat loop will be done V-1 times Finding the node (j) closest to (S) and not yet processed will cost O(V) Updating distances to all nodes adjacent to (j) and not yet processed will cost O(V) Hence, Dijkstra’s algorithm has a complexity of O(V 2 )

Prof. Amr Goneid, AUC73 5. All Pairs Shortest Paths (Floyd’s Algorithm: Dynamic Programming) Given a weighted directed graph G = (V,E) with n vertices. Find for each pair of vertices (i,j)  V the cost of the shortest path from i to j. Obviously, we can solve this problem by running Dijkstra’s algorithm (n) times, once for each node as a source vertex. The complexity would be O(n 3 ).

Prof. Amr Goneid, AUC74 Floyd’s Algorithm An alternate algorithm (Floyd’s Algorithm) uses dynamic programming and also has a complexity of O(n 3 ) The algorithm was developed by Robert Floyd (1962) Because it has a similarity with Warshall’s algorithm, it is sometimes called Floyd-Warshall algorithm.

Prof. Amr Goneid, AUC75 A 0 (i,j) = Cost Matrix Inf Inf 40 Inf 35 Inf Inf Inf 20 Inf 10 Inf 0 A 5 (i,j) Final Solution Example A E C B D

Prof. Amr Goneid, AUC76 All Pairs Shortest Paths Let A be an (nxn) matrix such that A k (i,j) is the length of the shortest path from i to j with internal nodes numbered ≤ k. Hence, the final solution is A n (i,j). A dynamic programming approach would first set A 0 (i,j) then computes A 1 (i,j), A 2 (i,j), etc. For this purpose, let:

Prof. Amr Goneid, AUC77 All Pairs Shortest Paths A shortest path from i to j with internal vertices no higher than k either goes through vertex k or it does not. k j i vertices numbered at most k-1

Prof. Amr Goneid, AUC78 All Pairs Shortest Paths If it does not pass through k, then there will be no change from the previous cost, i.e., A k (i,j) = A k-1 (i,j) If it goes through k then A k (i,j) = A k-1 (i,k) + A k-1 (k,j) On the k-th iteration, the algorithm determines shortest paths between every pair of vertices i, j that use only vertices among 1,…,k as intermediate.

Prof. Amr Goneid, AUC79 All Pairs Shortest Paths Hence, A k (i,j) = min { A k-1 (i,j), A k-1 (i,k) + A k-1 (k,j) }, k ≥ 1 i j k A (k-1) (i,j) A (k-1) (i,k) A (k-1) (k,j)

Prof. Amr Goneid, AUC80 All Pairs Shortest Paths Prove that A k (k,j) = A k-1 (k,j) Proof: A k (k,j) = min { A k-1 (k,j), A k-1 (k,k) + A k-1 (k,j) } Hence, A k (k,j) = A k-1 (k,j) Similarly, we can prove that A k (i,k) = A k-1 (i,k) Hence, the k th row and k th column do not change and we can do the computation in-place using the same matrix zero the same

Prof. Amr Goneid, AUC81 All Pairs Shortest Paths (Algorithm) void AllPaths (float cost[ ][N], float A[ ][N], int p[ ][N], int n) // cost[1:n][1:n] is the cost adjacency matrix of a graph with n // vertices; // cost[i][i] = 0.0, cost[i][j] = w ij for i  j and (i,j)  E, // and cost[i][j] =  if (i,j)  E // A[i][j] is the cost of a shortest path from vertex i to vertex j. // p[i][j] holds the indices of the nodes in the shortest paths. { for (int i=1; i<=n; i++) for (int j=1; j<=n; j++) { A[i][j] = cost[i][j]; // Copy cost into A to get A 0 (i,j) p[i][j] = 0; }

Prof. Amr Goneid, AUC82 All Pairs Shortest Paths (Algorithm) for (int k=1; k<=n; k++) for (i=1; i<=n; i++) for (j=1; j<=n; j++) if (A[i, k] + A[k, j] < A[i, j]) { A[i, j] = A[i, k] + A[k, j]; p[i][j] = k; } Analysis: It is easy to see that the complexity of Floyd’s algorithm is  (n 3 )

Prof. Amr Goneid, AUC83 A 0 (i,j) = Cost Matrix Inf Inf 40 Inf 35 Inf Inf Inf 20 Inf 10 Inf 0 A 1 (i,j) Example A E C B D Inf Inf Inf Inf 0

Prof. Amr Goneid, AUC84 A 2 (i,j) A 3 (i,j) Example A E C B D

Prof. Amr Goneid, AUC85 A 4 (i,j) A 5 (i,j) Final Solution Example A E C B D

Prof. Amr Goneid, AUC86 All Pairs Shortest Paths (Listing nodes on shortest paths) for (i=1; i<=n; i++) for (j=1; j<=n; j++) if (A[i][j] <  ) { cout << i <<” ”; between(i, j); cout << i <<” ”; } void between (int i, int j) { k = p[i, j]; if (k > 0) { between(i, k); cout << k << ” ”; between(k, j); } }

Prof. Amr Goneid, AUC87 6. Transitive Closure ( Warshall’s Algorithm: Dynamic Programming ) Problem: Given a directed graph G = (V,E), find for each pair of vertices i,j  V whether there is a path from i to j. Solution: make the cost of all edges 1, and run Floyd’s algorithm. If on termination A[i, j]  , then there is a path from i to j.

Prof. Amr Goneid, AUC88 Warshall’s Algorithm A better solution: use Boolean values to obtain the Transitive Closure (Reachability Matrix) of G. The transitive closure is a matrix A * such that A * (i,j) = 1 iff there is a directed path from vertex i to vertex j. The algorithm has been developed by Stephen Warshall (1962).

Prof. Amr Goneid, AUC89 Transitive Closure

Prof. Amr Goneid, AUC90 Warshall’s Algorithm Constructs transitive closure T as the last matrix in the sequence of n-by-n matrices A 0, …, A k, …, A n where A k (i,j) = 1 iff there is nontrivial path from i to j with only the first k vertices allowed as intermediate Note that A 0 = (adjacency matrix), A n = T (transitive closure) Main idea: a path exists between two vertices i, j, iff there is an edge from i to j; or there is a path from i to j going through vertex 1; or there is a path from i to j going through vertex 1 and/or 2; or there is a path from i to j going through vertex 1, 2, and/or 3; or... there is a path from i to j going through any of the other vertices

Prof. Amr Goneid, AUC91 Warshall’s Algorithm In the k th stage determine if a path exists between two vertices i, j using just vertices among 1,…,k i j k k th stage

Prof. Amr Goneid, AUC92 Warshall’s Algorithm Set A matrix to all false; for (int i=1; i<=n; i++) { for (int j=1; j<=n; j++) if ((i,j)  E) A[i][j] = true; A[i][i] = true; } for (int k=1; k<=n; k++) for (i=1; i<=n; i++) for (j=1; j<=n; j++) A[i][j] = A[i][j] || (A[i][k] && A[k][j]); The Complexity is also O(n 3 )

Prof. Amr Goneid, AUC93 Warshall’s Algorithm Example A A A A A

6a. Single-Source All Shortest Paths, General Weights (Bellman-Ford Algorithm) Dijkstra’s Algorithm computes Single –source all shortest paths in a directed or undirected graph with weights W ij > 0 Floyd’s Algorithm computes All-Pairs shortest paths in a directed or undirected graph with general weights (positive or negative), provided that there is no cycle with a negative length. Bellman-Ford Algorithm computes Single –source all shortest paths in a directed or undirected graph with general weights, provided that there is no cycle with a negative length. Prof. Amr Goneid, AUC94

Bellman-Ford Algorithm Named after Richard Bellman and Lester Ford Jr. (1956,1958) Cannot work if the graph contains a negative cycle reachable from the source, since any path can be made cheaper by more walk through the cycle. Like Floyd’s and Warshal’s algorithms, it is a DP algorithm based on the Principle of Optimality. Prof. Amr Goneid, AUC95

Bellman-Ford Algorithm Dijkstra's algorithm greedily selects the minimum- weight node that has not yet been processed, and relaxes the minimum distance on all of its outgoing edges. Complexity is O(V 2 ) and a use of a heap makes it O(V log V). The Bellman–Ford algorithm relaxes all the edges, doing this |V|-1 times (when there are no cycles of negative length, there is a shortest path between any two vertices that has at most |V|-1 edges). Complexity is O(VE). Prof. Amr Goneid, AUC96

Bellman-Ford Algorithm Given that W si is the initial cost from source (s) to nodes (i), a distance array is initially set at D 1 si = W si for i = 1…|V|, Consider a vertex (u != s) and u has at least one incoming edge, all vertices (i) such that edge (i,u) is in the graph are candidates to reach (u). Hence, there are two distances to compete: D su and min i {D si + W iu } Hence, the update rule is: Prof. Amr Goneid, AUC97

Bellman-Ford Algorithm DP Update Rule (Recurrence Relation): D 1 su = W su D k su = min {D k-1 su, min i {D k-1 si + W iu }, k = 2,3,…, |V|-1 Prof. Amr Goneid, AUC98 S u i i i D su D si W ui

Bellman-Ford Algorithm Algorithm: for i = 1 to |V| D si = W si for k = 2 to |V|-1 for (each u != s, u has at least one incoming edge) for (each edge (i, u) in the graph) // Relax if (D su > D si + W iu ) D su = D si + W iu // one more iteration: if there is a shorter path anywhere, then we have a negative length cycle Prof. Amr Goneid, AUC99

7. Graph Coloring Problem (1) m-Colorability Decision: Can we use (m) colors only to color a graph such that no two adjacent nodes have the same color? If yes, what are the possible coloring schemes? Example: Prof. Amr Goneid, AUC100 B A D C A C B D

Backtracking Permutation Tree Solution 1 : G R G R Solution 2 : R G R G Gain = 1-15/31 = 51.6% Prof. Amr Goneid, AUC G GR G G R R

Prof. Amr Goneid, AUC102 Graph Coloring Problem (2) (Backtracking) The problem: Given m colors represented by integers 1.. m, and a graph of n vertices. Find all n-tuples (x 1,x 2,..x n ), where x k is the color of node (k) such that no two adjacent nodes have the same color. Pre: The graph is represented by the boolean adjacency matrix G[1:n] [1:n] x[k] is assigned zeros.

Prof. Amr Goneid, AUC103 NextColor void NextColor(int k, int n) { do { x[k] = (x[k] + 1) % (m+1); // Next Highest Color if ( !x[k]) return; // No more colors available for (int j = 1; j <= n; j++) if ( G[k] [j] && x[k] == x[j] ) break; if ( j == n+1) return; // New color found } while (1); // Otherwise, try to find another color }

Prof. Amr Goneid, AUC104 m-Coloring Algorithm void mColoring(int k, int n) { do { NextColor(k,n); // Assign to x[k] // a legal color if( !x[k] ) break ; // No new color possible if (k == n) output vector x[1:n]; else mColoring(k+1, n); } while(1); }

Prof. Amr Goneid, AUC105 Example Find all 3-colorings of the shown 4-node graph

Prof. Amr Goneid, AUC106 Solutions withTwo Colors Only 6 Tuples

Prof. Amr Goneid, AUC107 Solutions with Exactly 3 Colors 12 Tuples: 1 st

Prof. Amr Goneid, AUC108 Solutions with Exactly 3 Colors 12 Tuples: 2 nd

8. Topological Sorting A Topological Sorting of a Directed Acyclic Graph (DAG) is a linear ordering of its vertices such that, for every edge (u,v), u comes before v in the ordering. For instance, the vertices of the graph may represent tasks to be performed, and the edges may represent constraints that one task must be performed before another. In this application, a topological sorting is just a valid sequence for the tasks. A topological ordering is possible if and only if the graph has no directed cycles, that is, if it is a directed acyclic graph (DAG).. Prof. Amr Goneid, AUC109

Example Consider the following DAG representing the pre- requisite structure of five courses. A valid topological ordering would be: C1, C2, C3, C4, C5 Also: C2, C1, C3, C4, C5 Prof. Amr Goneid, AUC110

An Algorithm (exclude & Conquer) L ← Empty list that will contain the sorted elements S ← Set of all nodes with no incoming edges while S is non-empty do remove a node n from S insert n into L for each node m with an incoming edge e from n remove edge e from the graph if m has no other incoming edges then insert m into S return L (a topologically sorted order) Prof. Amr Goneid, AUC111

An Algorithm (exclude & Conquer) Prof. Amr Goneid, AUC112

9. The Clique Problem (a) Complete and Sub Graphs The complete graph on n vertices is K n = (V,E) where V = {1, 2,..., n}, and every pair of vertices in V is joined by an edge.The number of edges in K n is maximum, i.e.,│E│= n(n-1)/2 A subgraph of a graph G = (V,E) is a graph B =(U, F) such that U ⊆ V and F ⊆ E. Prof. Amr Goneid, AUC113

(b) The Clique Prof. Amr Goneid, AUC114 A clique in a graph is complete subgraph of three or more nodes. The clique size (k) is the number of nodes in the subgraph.

The Clique Finding a clique is in a class of NP-complete problems, along with other hard problems such as Knapsack packing, graph coloring, and famous TSP (Traveling salesman problem). Up until now, no better than an exponential algorithm for such problems is known. widely seen in numerous fields of science and engineering applications, even business application. Prof. Amr Goneid, AUC115

The Clique Example (Map Coloring) Each state is to be colored differently from other neighbors. The number of different colors used represents the number of cities that you can travel directly from one to another, which is equivalent to a clique in a graph. Note that B, C, F, and J are adjacent to every other state in the sub- province, forming a 4-clique in the graph. Prof. Amr Goneid, AUC116