David Luebke 1 11/21/2016 CS 332: Algorithms Minimum Spanning Tree Shortest Paths

David Luebke 2 11/21/2016 Review: Getting Dressed UnderwearSocks ShoesPants Belt Shirt Watch Tie Jacket SocksUnderwearPantsShoesWatchShirtBeltTieJacket

David Luebke 3 11/21/2016 Review: Topological Sort Algorithm Topological-Sort() { Run DFS When a vertex is finished, output it Vertices are output in reverse topological order } l Time: O(V+E)

David Luebke 4 11/21/2016 Review: Minimum Spanning Tree l Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight

David Luebke 5 11/21/2016 Review: Minimum Spanning Tree l Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight

David Luebke 6 11/21/2016 Review: Minimum Spanning Tree l MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees l If T is MST of G, and A  T is a subtree of T, and (u,v) is the min-weight edge connecting A to V-A, then (u,v)  T

David Luebke 7 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); Run on example graph

David Luebke 8 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);     Run on example graph

David Luebke 9 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0    Pick a start vertex r r

David Luebke 10 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0    Red vertices have been removed from Q u

David Luebke 11 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0  3  Red arrows indicate parent pointers u

David Luebke 12 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  0  3  u

David Luebke 13 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  0  3  u

David Luebke 14 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  08  3  u

David Luebke 15 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 10  08  3  u

David Luebke 16 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 10  08  3  u

David Luebke 17 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102  08  3  u

David Luebke 18 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102   u

David Luebke 19 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102   u

David Luebke 20 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  u

David Luebke 21 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 22 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 23 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 24 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 25 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 26 11/21/2016 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); u

David Luebke 27 11/21/2016 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What is the hidden cost in this code?

David Luebke 28 11/21/2016 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v));

David Luebke 29 11/21/2016 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v)); How often is ExtractMin() called? How often is DecreaseKey() called?

David Luebke 30 11/21/2016 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What will be the running time? A: Depends on queue binary heap: O(E lg V) Fibonacci heap: O(V lg V + E)

David Luebke 31 11/21/2016 Single-Source Shortest Path l Problem: given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v n “Shortest-path” = minimum weight n Weight of path is sum of edges n E.g., a road map: what is the shortest path from Chapel Hill to Charlottesville?

David Luebke 32 11/21/2016 Shortest Path Properties l Again, we have optimal substructure: the shortest path consists of shortest subpaths: n Proof: suppose some subpath is not a shortest path u There must then exist a shorter subpath u Could substitute the shorter subpath for a shorter path u But then overall path is not shortest path. Contradiction

David Luebke 33 11/21/2016 Shortest Path Properties l Define  (u,v) to be the weight of the shortest path from u to v l Shortest paths satisfy the triangle inequality:  (u,v)   (u,x) +  (x,v) l “Proof”: x u v This path is no longer than any other path

David Luebke 34 11/21/2016 Shortest Path Properties l In graphs with negative weight cycles, some shortest paths will not exist (Why?): < 0

David Luebke 35 11/21/2016 Relaxation l A key technique in shortest path algorithms is relaxation n Idea: for all v, maintain upper bound d[v] on  (s,v) Relax(u,v,w) { if (d[v] > d[u]+w) then d[v]=d[u]+w; } Relax

David Luebke 36 11/21/2016 Bellman-Ford Algorithm BellmanFord() for each v  V d[v] =  ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v)  E Relax(u,v, w(u,v)); for each edge (u,v)  E if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w Initialize d[], which will converge to shortest-path value  Relaxation: Make |V|-1 passes, relaxing each edge Test for solution Under what condition do we get a solution?

David Luebke 37 11/21/2016 Bellman-Ford Algorithm BellmanFord() for each v  V d[v] =  ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v)  E Relax(u,v, w(u,v)); for each edge (u,v)  E if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w What will be the running time?

David Luebke 38 11/21/2016 Bellman-Ford Algorithm BellmanFord() for each v  V d[v] =  ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v)  E Relax(u,v, w(u,v)); for each edge (u,v)  E if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w What will be the running time? A: O(VE)

David Luebke 39 11/21/2016 Bellman-Ford Algorithm BellmanFord() for each v  V d[v] =  ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v)  E Relax(u,v, w(u,v)); for each edge (u,v)  E if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B E DC A Ex: work on board s

David Luebke 40 11/21/2016 Bellman-Ford l Note that order in which edges are processed affects how quickly it converges l Correctness: show d[v] =  (s,v) after |V|-1 passes n Lemma: d[v]   (s,v) always u Initially true u Let v be first vertex for which d[v] <  (s,v) u Let u be the vertex that caused d[v] to change: d[v] = d[u] + w(u,v) u Then d[v]<  (s,v)  (s,v)   (s,u) + w(u,v)(Why?)  (s,u) + w(u,v)  d[u] + w(u,v)(Why?) u So d[v] < d[u] + w(u,v). Contradiction.

David Luebke 41 11/21/2016 Bellman-Ford l Prove: after |V|-1 passes, all d values correct n Consider shortest path from s to v: s  v 1  v 2  v 3  v 4  v u Initially, d[s] = 0 is correct, and doesn’t change (Why?) u After 1 pass through edges, d[v 1 ] is correct (Why?) and doesn’t change u After 2 passes, d[v 2 ] is correct and doesn’t change u … u Terminates in |V| - 1 passes: (Why?) u What if it doesn’t?

David Luebke 42 11/21/2016 DAG Shortest Paths l Problem: finding shortest paths in DAG n Bellman-Ford takes O(VE) time. n How can we do better? n Idea: use topological sort u If were lucky and processes vertices on each shortest path from left to right, would be done in one pass u Every path in a dag is subsequence of topologically sorted vertex order, so processing verts in that order, we will do each path in forward order (will never relax edges out of vert before doing all edges into vert). u Thus: just one pass. What will be the running time?

David Luebke 43 11/21/2016 Dijkstra’s Algorithm l If no negative edge weights, we can beat BF l Similar to breadth-first search n Grow a tree gradually, advancing from vertices taken from a queue l Also similar to Prim’s algorithm for MST n Use a priority queue keyed on d[v]

David Luebke 44 11/21/2016 Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S  {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); Relaxation Step Note: this is really a call to Q->DecreaseKey() B C DA Ex: run the algorithm

David Luebke 45 11/21/2016 Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S  {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); How many times is ExtractMin() called? How many times is DecraseKey() called? What will be the total running time?

David Luebke 46 11/21/2016 Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S  {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); How many times is ExtractMin() called? How many times is DecraseKey() called? A: O(E lg V) using binary heap for Q Can acheive O(V lg V + E) with Fibonacci heaps

David Luebke 47 11/21/2016 Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] =  ; d[s] = 0; S =  ; Q = V; while (Q   ) u = ExtractMin(Q); S = S  {u}; for each v  u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); Correctness: we must show that when u is removed from Q, it has already converged

David Luebke 48 11/21/2016 Correctness Of Dijkstra's Algorithm l Note that d[v]   (s,v)  v l Let u be first vertex picked s.t.  shorter path than d[u]  d[u] >  (s,u) l Let y be first vertex  V-S on actual shortest path from s  u  d[y] =  (s,y) n Because d[x] is set correctly for y's predecessor x  S on the shortest path, and n When we put x into S, we relaxed (x,y), giving d[y] the correct value s x y u p2p2 p2p2

David Luebke 49 11/21/2016 Correctness Of Dijkstra's Algorithm l Note that d[v]   (s,v)  v l Let u be first vertex picked s.t.  shorter path than d[u]  d[u] >  (s,u) l Let y be first vertex  V-S on actual shortest path from s  u  d[y] =  (s,y) l d[u]>  (s,u) =  (s,y) +  (y,u) (Why?) = d[y] +  (y,u)  d[y]But if d[u] > d[y], wouldn't have chosen u. Contradiction. s x y u p2p2 p2p2

David Luebke 50 11/21/2016 The End

David Luebke 51 11/21/2016 Exercise 1 Feedback l First, my apologies… n Harder than I thought n Too late to help with midterm l Proof by substitution: n T(n) = T(n/2 +  n) + n n Most people assumed it was O(n lg n)…why? u Resembled proof from class: T(n) = 2T(n/2 + 17) + n u The correct intuition: n/2 dominates  n term, so it resembles T(n) = T(n/2) + n, which is O(n) by m.t. n Still, if it’s O(n) it’s O(n lg n), right?

David Luebke 52 11/21/2016 Exercise 1: Feedback l So, prove by substitution that T(n) = T(n/2 +  n) + n = O(n lg n) l Assume T(n)  cn lg n l Then T(n)  c(n/2 +  n) lg (n/2 +  n)  c(n/2 +  n) lg (n/2 + n)  c(n/2 + n) lg (3n/2)  …