CSE 421 Algorithms Richard Anderson Dijkstra’s algorithm

Single Source Shortest Path Problem Given a graph and a start vertex s –Determine distance of every vertex from s –Identify shortest paths to each vertex Express concisely as a “shortest paths tree” Each vertex has a pointer to a predecessor on shortest path s v x u s v x u 1 3 3

Construct Shortest Path Tree from s a b c s e g f d a b c s e g f d

Warmup If P is a shortest path from s to v, and if t is on the path P, the segment from s to t is a shortest path between s and t WHY? s t v

Dijkstra’s Algorithm S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], d[v] + c(v, w)) s u v z y x Assume all edges have non-negative cost

Simulate Dijkstra’s algorithm (strarting from s) on the graph Round Vertex Added sabcd bd c a s

Dijkstra’s Algorithm as a greedy algorithm Elements committed to the solution by order of minimum distance

Correctness Proof Elements in S have the correct label Key to proof: when v is added to S, it has the correct distance label. s y v x u

Proof Let v be a vertex in V-S with minimum d[v] Let P v be a path of length d[v], with an edge (u,v) Let P be some other path to v. Suppose P first leaves S on the edge (x, y) –P = P sx + c(x,y) + P yv –Len(P sx ) + c(x,y) >= d[y] –Len(P yv ) >= 0 –Len(P) >= d[y] + 0 >= d[v] s y v x u

Negative Cost Edges Draw a small example a negative cost edge and show that Dijkstra’s algorithm fails on this example

Bottleneck Shortest Path Define the bottleneck distance for a path to be the maximum cost edge along the path s v x u

Compute the bottleneck shortest paths a b c s e g f d a b c s e g f d

How do you adapt Dijkstra’s algorithm to handle bottleneck distances Does the correctness proof still apply?

Who was Dijkstra? What were his major contributions?

Edsger Wybe Dijkstra was one of the most influential members of computing science's founding generation. Among the domains in which his scientific contributions are fundamental are –algorithm design –programming languages –program design –operating systems –distributed processing –formal specification and verification –design of mathematical arguments

Shortest Paths Negative Cost Edges –Dijkstra’s algorithm assumes positive cost edges –For some applications, negative cost edges make sense –Shortest path not well defined if a graph has a negative cost cycle a b c s e g f

Negative Cost Edge Preview Topological Sort can be used for solving the shortest path problem in directed acyclic graphs Bellman-Ford algorithm finds shortest paths in a graph with negative cost edges (or reports the existence of a negative cost cycle).

Dijkstra’s Algorithm Implementation and Runtime S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], d[v] + c(v, w)) s u v z y x Edge costs are assumed to be non-negative a b HEAP OPERATIONS n Extract Mins m Heap Updates

Bottleneck Shortest Path Define the bottleneck distance for a path to be the maximum cost edge along the path s v x u

Compute the bottleneck shortest paths a b c s e g f d a b c s e g f d

Dijkstra’s Algorithm for Bottleneck Shortest Paths S = {}; d[s] = negative infinity; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], max(d[v], c(v, w))) s u v z y x a b

Minimum Spanning Tree Introduce Problem Demonstrate three different greedy algorithms Provide proofs that the algorithms work

Minimum Spanning Tree a b c s e g f t u v

Greedy Algorithms for Minimum Spanning Tree Extend a tree by including the cheapest out going edge Add the cheapest edge that joins disjoint components Delete the most expensive edge that does not disconnect the graph a bc d e

Greedy Algorithm 1 Prim’s Algorithm Extend a tree by including the cheapest out going edge t a e c g f b s u v Construct the MST with Prim’s algorithm starting from vertex a Label the edges in order of insertion

Greedy Algorithm 2 Kruskal’s Algorithm Add the cheapest edge that joins disjoint components t a e c g f b s u v Construct the MST with Kruskal’s algorithm Label the edges in order of insertion

Greedy Algorithm 3 Reverse-Delete Algorithm Delete the most expensive edge that does not disconnect the graph t a e c g f b s u v Construct the MST with the reverse- delete algorithm Label the edges in order of removal

Why do the greedy algorithms work? For simplicity, assume all edge costs are distinct Let S be a subset of V, and suppose e = (u, v) is the minimum cost edge of E, with u in S and v in V-S e is in every minimum spanning tree

Proof Suppose T is a spanning tree that does not contain e Add e to T, this creates a cycle The cycle must have some edge e 1 = (u 1, v 1 ) with u 1 in S and v 1 in V-S T 1 = T – {e 1 } + {e} is a spanning tree with lower cost Hence, T is not a minimum spanning tree

Optimality Proofs Prim’s Algorithm computes a MST Kruskal’s Algorithm computes a MST

Reverse-Delete Algorithm Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

Dealing with the assumption of no equal weight edges Force the edge weights to be distinct –Add small quantities to the weights –Give a tie breaking rule for equal weight edges

Dijkstra’s Algorithm for Minimum Spanning Trees S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], c(v, w)) s u v z y x a b

Minimum Spanning Tree a b c s e g f t u v Undirected Graph G=(V,E) with edge weights

Greedy Algorithms for Minimum Spanning Tree [Prim] Extend a tree by including the cheapest out going edge [Kruskal] Add the cheapest edge that joins disjoint components [ReverseDelete] Delete the most expensive edge that does not disconnect the graph a bc d e

Why do the greedy algorithms work? For simplicity, assume all edge costs are distinct

Edge inclusion lemma Let S be a subset of V, and suppose e = (u, v) is the minimum cost edge of E, with u in S and v in V-S e is in every minimum spanning tree of G –Or equivalently, if e is not in T, then T is not a minimum spanning tree SV - S e

Proof Suppose T is a spanning tree that does not contain e Add e to T, this creates a cycle The cycle must have some edge e 1 = (u 1, v 1 ) with u 1 in S and v 1 in V-S T 1 = T – {e 1 } + {e} is a spanning tree with lower cost Hence, T is not a minimum spanning tree SV - S e e is the minimum cost edge between S and V-S

Optimality Proofs Prim’s Algorithm computes a MST Kruskal’s Algorithm computes a MST Show that when an edge is added to the MST by Prim or Kruskal, the edge is the minimum cost edge between S and V-S for some set S.

Prim’s Algorithm S = { }; T = { }; while S != V choose the minimum cost edge e = (u,v), with u in S, and v in V-S add e to T add v to S

Prove Prim’s algorithm computes an MST Show an edge e is in the MST when it is added to T

Kruskal’s Algorithm Let C = {{v 1 }, {v 2 },..., {v n }}; T = { } while |C| > 1 Let e = (u, v) with u in C i and v in C j be the minimum cost edge joining distinct sets in C Replace C i and C j by C i U C j Add e to T

Prove Kruskal’s algorithm computes an MST Show an edge e is in the MST when it is added to T

Reverse-Delete Algorithm Lemma: The most expensive edge on a cycle is never in a minimum spanning tree

Dealing with the assumption of no equal weight edges Force the edge weights to be distinct –Add small quantities to the weights –Give a tie breaking rule for equal weight edges

Application: Clustering Given a collection of points in an r- dimensional space, and an integer K, divide the points into K sets that are closest together

Distance clustering Divide the data set into K subsets to maximize the distance between any pair of sets –dist (S 1, S 2 ) = min {dist(x, y) | x in S 1, y in S 2 }

Divide into 2 clusters

Divide into 3 clusters

Divide into 4 clusters

Distance Clustering Algorithm Let C = {{v 1 }, {v 2 },..., {v n }}; T = { } while |C| > K Let e = (u, v) with u in C i and v in C j be the minimum cost edge joining distinct sets in C Replace C i and C j by C i U C j

K-clustering