1 Graphs: shortest paths & Minimum Spanning Tree(MST) Fundamental Data Structures and Algorithms Ananda Guna April 8, 2003

2 Announcements  Homework #5 is due Tuesday April 15th.  Quiz #3 feedback is enabled.  Final Exam is Tuesday May 8 th at 8AM

Recap

4 Dijkstra’s algorithm  S = {1}  for i = 2 to n do D[i] = C[1,i] if there is an edge from 1 to i, infinity otherwise  for i = 1 to n-1 { choose a vertex w in V-S such that D[w] is min add w to S (where S is the set of visited nodes) for each vertex v in V-S do D[v] = min(D[v], D[w]+c[w,v]) } Where |V| = n

5 Features of Dijkstra’s Algorithm A greedy algorithm “Visits” every vertex only once, when it becomes the vertex with minimal distance amongst those still in the priority queue Distances may be revised multiple times: current values represent ‘best guess’ based on our observations so far Once a vertex is visited we are guaranteed to have found the shortest path to that vertex…. why?

6 Correctness (via contradiction) D(x) must represent a shortest path to x, and D(x)  D shortest (u). However, Dijkstra’s always visits the vertex with the smallest distance next, so we can’t possibly visit u before we visit x u x s visited unvisited Prove D(u) represent the shortest path to u (visited node) Assume u is the first vertex visited such that D(u) is not a shortest path (thus the true shortest path to u must pass through some unvisited vertex) Let x represent the first unvisited vertex on the true shortest path to u

7 Quiz break  Would it be better to use an adjacency list or an adjacency matrix for Dijkstra’s algorithm?  What is the running time of Dijkstra’s algorithm, in terms of |V| and |E| in each case?

8 Complexity of Dijkstra  Adjacency matrix version Dijkstra finds shortest path from one vertex to all others in O(|V| 2 ) time  If |E| is small compared to |V| 2, use a priority queue to organize the vertices in V-S, where V is the set of all vertices and S is the set that has already been explored  So total of |E| updates each at a cost of O(log |V|)  So total time is O(|E| log|V|)

Negative Weighted Single-Source Shortest Path Algorithm (Bellman-Ford Algorithm)

10 The Bellman-Ford algorithm (see Weiss, Section 14.4) Returns a boolean: TRUE if and only if there is no negative- weight cycle reachable from the source: a simple cycle, where v 0 =v k and FALSE otherwise If it returns TRUE, it also produces the shortest paths

11 Example  For each edge (u,v), let's denote its length by C(u,v))  Let d[i][v] = distance from start to v using the shortest path out of all those that use i or fewer edges, or infinity if you can't get there with <= i edges.

12 Example ctd..  How can we fill out the rows?  1050  15   V i

13 Example ctd..  Can we get i th row from i-1 th row?  for v != start, d[v][i] = MIN d[x][i-1] + len(x,v) x->v  We know minimum path to come to x using < i nodes.So for all x that can reach v, find the minimum such sum (in blue) among all x  Assume d[start][i] = 0 for all i

14 Completing the table  1050  15   d[v][i] = MIN d[x][i-1] + len(x,v) x->v

15 Key features If the graph contains no negative-weight cycles reachable from the source vertex, after |V| - 1 iterations all distance estimates represent shortest paths…why?

16 Correctness By induction: D(s) = 0 after initialization Assume D(v i-1 ) is a shortest path after iteration (i-1) Since edge (v i-1,v i ) is updated on the i th pass, D(v i ) must then reflect the shortest path to v i. Since we perform |V| - 1 iterations, D(v i ) for all reachable vertices v i must now represent shortest paths The algorithm will return true because on the |V| th iteration, no distances will change Case 1: Graph G=(V,E) doesn’t contain any negative- weight cycles reachable from the source vertex s Consider a shortest path p =, which must have k  |V| - 1 edges

17 Correctness Proof by contradiction: Assume the algorithm returns TRUE Thus, D(v i-1 ) + weight(v i-1, v i )  D(v i ) for i = 1,…,k Summing the inequalities for the cycle: leads to a contradiction since the first sums on each side are equal (each vertex appears exactly once) and the sum of weights must be less than 0. Case 2: Graph G=(V,E) contains a negative-weight cycle reachable from the source vertex s

18 Performance Initialization: O(|V|) Path update and cycle check: |V| calls checking |E| edges, O(|VE|) Overall cost: O(| VE |)

The All Pairs Shortest Path Algorithm (Floyd’s Algorithm)

20 Finding all pairs shortest paths  Assume G=(V,E) is a graph such that c[v,w]  0, where C is the matrix of edge costs.  Find for each pair (v,w), the shortest path from v to w. That is, find the matrix of shortest paths  Certainly this is a generalization of Dijkstra’s.  Note: For later discussions assume |V| = n and |E| = m

21 Floyd’s Algorithm  A[i][j] = C(i,j) if there is an edge (i,j)  A[i][j] = infinity(inf) if there is no edge (i,j) Graph “adjacency” matrix A is the shortest path matrix that uses 1 or fewer edges

22 Floyd ctd..  To find shortest paths that uses 2 or fewer edges find A 2, where multiplication defined as min of sums instead sum of products  That is (A 2 ) ij = min{ A ik + A kj | k =1..n}  This operation is O(n 3 )  Using A 2 you can find A 4 and then A 8 and so on  Therefore to find A n we need log n operations  Therefore this algorithm is O(log n* n 3 )  We will consider another algorithm next

23 Floyd-Warshall Algorithm  This algorithm uses nxn matrix A to compute the lengths of the shortest paths using a dynamic programming technique.  Let A[i,j] = c[i,j] for all i,j & ij  If (i,j) is not an edge, set A[i,j]=infinity and A[i,i]=0  A k [i,j] = min (A k-1 [i,j], A k-1 [i,k]+ A k-1 [k,j]) Where A k is the matrix after k-th iteration and path from i to j does not pass through a vertex higher than k

24 Example – Floyd-Warshall Algorithm Find the all pairs shortest path matrix  A k [i,j] = min (A k-1 [i,j], A k-1 [i,k]+ A k-1 [k,j]) Where A k is the matrix after k-th iteration and path from i to j does not pass through a vertex higher than k

25 Floyd-Warshall Implementation  initialize A[i,j] = C[i,j]  initialize all A[i,i] = 0  for k from 1 to n for i from 1 to n for j from 1 to n if (A[i,j] > A[i,k]+A[k,j]) A[i,j] = A[i,k]+A[k,j];  The complexity of this algorithm is O(n 3 )

26 Questions  Question: What is the asymptotic run time of Dijkstra (adjacency matrix version)?  O(n 2 )  Question: What is the asymptotic running time of Floyd-Warshall?

27 Minimum Spanning Trees (some material adapted from slides by Peter Lee)

28 Problem: Laying Telephone Wire Central office

29 Wiring: Naïve Approach Central office Expensive!

30 Wiring: Better Approach Central office Minimize the total length of wire connecting the customers

31 Minimum Spanning Tree (MST) (see Weiss, Section )  it is a tree (i.e., it is acyclic)  it covers all the vertices V  contains |V| - 1 edges  the total cost associated with tree edges is the minimum among all possible spanning trees  not necessarily unique A minimum spanning tree is a subgraph of an undirected weighted graph G, such that

32 Applications of MST  Any time you want to visit all vertices in a graph at minimum cost (e.g., wire routing on printed circuit boards, sewer pipe layout, road planning…)  Internet content distribution  $$$, also a hot research topic  Idea: publisher produces web pages, content distribution network replicates web pages to many locations so consumers can access at higher speed  MST may not be good enough!  content distribution on minimum cost tree may take a long time!

33 How Can We Generate a MST? a c e d b a c e d b

34 Prim’s Algorithm  Let V ={1,2,..,n} and U be the set of vertices that makes the MST and T be the MST  Initially : U = {1} and T =   while (U  V) let (u,v) be the lowest cost edge such that u U and v  V-U T = T  {(u,v)} U = U  {v}

35 Prim’s Algorithm implementation Initialization a. Pick a vertex r to be the root b. Set D(r) = 0, parent(r) = null c. For all vertices v  V, v  r, set D(v) =  d. Insert all vertices into priority queue P, using distances as the keys a c e d b eabcd 0  VertexParent e -

36 Prim’s Algorithm While P is not empty: 1. Select the next vertex u to add to the tree u = P.deleteMin() 2. Update the weight of each vertex w adjacent to u which is not in the tree (i.e., w  P) If weight(u,w) < D(w), a. parent(w) = u b. D(w) = weight(u,w) c. Update the priority queue to reflect new distance for w

37 Prim’s algorithm a c e d b dbca 455  VertexParent e- be ce de The MST initially consists of the vertex e, and we update the distances and parent for its adjacent vertices

38 Prim’s algorithm a c e d b acb 245 VertexParent e- be c d de a d

39 Prim’s algorithm a c e d b cb 45 VertexParent e- be c d de a d

40 Prim’s algorithm a c e d b b 5 VertexParent e- be c d de a d

41 Prim’s algorithm VertexParent e- be c d de a d a c e d b The final minimum spanning tree

42 Prim’s Algorithm Invariant  At each step, we add the edge (u,v) s.t. the weight of (u,v) is minimum among all edges where u is in the tree and v is not in the tree  Each step maintains a minimum spanning tree of the vertices that have been included thus far  When all vertices have been included, we have a MST for the graph!

43 Running time of Prim’s algorithm Initialization of priority queue (array): O(| V |) Update loop: | V | calls Choosing vertex with minimum cost edge: O(| V |) Updating distance values of unconnected vertices: each edge is considered only once during entire execution, for a total of O(| E |) updates Overall cost: O(| E | + | V | 2 )

44 Another Approach – Kruskal’s a c e d b Create a forest of trees from the vertices Repeatedly merge trees by adding “safe edges” until only one tree remains A “safe edge” is an edge of minimum weight which does not create a cycle forest: {a}, {b}, {c}, {d}, {e}

45 Kruskal’s algorithm Initialization a. Create a set for each vertex v  V b. Initialize the set of “safe edges” A comprising the MST to the empty set c. Sort edges by increasing weight a c e d b {a}, {b}, {c}, {d}, {e} A =  E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)}

46 Kruskal’s algorithm For each edge (u,v)  E in increasing order while more than one set remains: If u and v, belong to different sets a. A = A  {(u,v)} b. merge the sets containing u and v Return A  Use Union-Find algorithm to efficiently determine if u and v belong to different sets

47 Kruskal’s algorithm E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)} Forest {a}, {b}, {c}, {d}, {e} {a,d}, {b}, {c}, {e} {a,d,c}, {b}, {e} {a,d,c,e}, {b} {a,d,c,e,b} A  {(a,d)} {(a,d), (c,d)} {(a,d), (c,d), (d,e)} {(a,d), (c,d), (d,e), (b,e)} a c e d b

48  After each iteration, every tree in the forest is a MST of the vertices it connects  Algorithm terminates when all vertices are connected into one tree Kruskal’s Algorithm Invariant

49 Greedy Approach  Like Dijkstra’s algorithm, both Prim’s and Kruskal’s algorithms are greedy algorithms  The greedy approach works for the MST problem; however, it does not work for many other problems!

50 Thursday  P vs NP  Models of Hard Problems  Work on Homework 5