Chapter 23 Minimum Spanning Trees Kruskal Prim [Source: Cormen et al. textbook except where noted] I joined the UMass Lowell Computer Science faculty this summer. This collection of slides is intended to familiarize the reader/viewer with my field of research (Computational Geometry), summarize my previous research results in this field and outline my plan for Computational Geometry research at UMass Lowell.

Minimum Spanning Trees This refers to a number of problems (many of practical importance) that can be reduced to the following abstraction: let G = (V, E) be a connected, undirected graph. You are also given a function w : E  R (usually non-negative, but not always so required). You are then asked to find an acyclic subset T Í E that connects all the vertices and whose total weight is minimized. The acyclicity of T implies that T is a tree; the requirement that it connect all the vertices implies that it is a spanning tree; the minimization implies that it is a minimum such object.

Minimum Spanning Trees MST Example

Minimum Spanning Trees Algorithms We will construct two algorithms as variants of a “generic” one. Both will be greedy algorithms (requiring that we show all necessary greedy properties are satisfied). We start with a loop invariant: let A be a set of edges; prior to each iteration, A is a subset of some minimum spanning tree. At each step we determine an edge (u, v) (a safe edge) that we can add to A without violating the invariant.

Minimum Spanning Trees The Generic Algorithm

Minimum Spanning Trees The Generic Algorithm How do we find a safe edge? It exists, because of the assumption on A (part of a minimum spanning tree)… Def.: a cut (S, V – S) of an undirected graph G = (V, E) is a partition of V. Def.: an edge (u, v) crosses the cut (S, V – S) if one of its endpoints is in S and the other in V. Def.: a cut respects a set A of edges if no edge in A crosses the cut. Def.: An edge is a light edge crossing a cut if its weight is minimum among all edges crossing the cut. Uniqueness is not required. This can be generalized to properties other than crossing.

Minimum Spanning Trees The Generic Algorithm

Minimum Spanning Trees The Generic Algorithm Theorem 23.1. Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let (S, V – S) be any cut of G that respects A, and let (u, v) be a light edge crossing (S, V – S). Then edge (u, v) is safe for A. Proof: let T be an MST, A Í T, A ≠ T. Assume T does not contain the light edge (u, v). If it does, we are done. If not, we construct another MST T’ that contains both A and (u, v). T {(u, v)} must contain a cycle, with edges on a simple path p from u to v in T. u and v are on opposite sides of the cut (S, V – S), and at least one edge in T lies on p and crosses the cut.

Minimum Spanning Trees The Generic Algorithm Proof (cont.) Let (x, y) be any such edge – it cannot be in A, because the cut respects A. Since (x, y) is on the unique simple path from u to v in T, removing (x, y) breaks T into two components. Adding (u, v) reconnects them to form a new spanning tree T’ = (T – {(x, y)}) {(u, v)}. But T’ is also an MST: since (u, v) is a light edge crossing (S, V – S) and (x, y) also crosses the cut, w(u, v) ≤ w(x, y) and w(T’) = w(T) – w(x, y) + w(u, v) ≤ w(T). The minimality of T implies w(T) ≤ w(T’), so T’ must be minimal, also. Is (u, v) safe? Since A Í T and (x, y) Ï A, we have A Í T’. Thus A {(u, v)} Í T’. Since T’ is an MST, (u, v) is safe for A.

Minimum Spanning Trees The Generic Algorithm Corollary 23.2. Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let C = (VC, EC) be a connected component (tree) in the forest GA = (V, A). If (u, v) is a light edge connecting C to some other component in GA, then (u, v) is safe for A. Proof: the cut (VC, V – VC) respects A, and (u, v) is a light edge for this cut. Therefore (u, v) is safe for A.

Minimum Spanning Trees The Generic Algorithm We now move to realizations of the generic algorithm. We choose two distinct ones, one based on a sorted (non-decreasing) list of edges and the set operations Make-Set, Find-Set and Union; the other based on priority queues of edges and the operation Extract-Min.

Minimum Spanning Trees Kruskal’s Algorithm

Minimum Spanning Trees Kruskal’s Algorithm This algorithm uses the disjoint-set datatype introduced in Ch. 21. The user-accessible operations are Make-Set(x), Union(x, y) and Find-Set(x), where x and y stand for either singletons or sets of which they are representative elements. We can show (easily) that a sequence of m Make-Set, Union and Find-Set operations (with an easy implementation), n of which are Make-Set takes O(m lg n) time. A more complex implementation (with union-by-rank and path compression) and analysis gives a bound O(m•a(n)), where a(n) ≤ 4 for any value of n relevant to our universe.

Minimum Spanning Trees Kruskal’s Algorithm Line 1 takes O(1); lines 2 and 3 take O(n), but we will account for it later; line 4 takes O(|E| lg |E|); the for loop (ll. 5-8) performs O(|E|) Find-Set and Union operations, which, with the |V| Make-Set operations take a total of O((|V| + |E|)a(|V|)) time. The assumption that G is connected implies |E| ≥ |V| - 1, so that the disjoint-set operations take O(|E|a(|V|)). Since a(|V|) = O(lg |V|) = O(lg |E|), the total time of Kruskal’s algorithm is O(|E| lg |E|) and |E| < |V|2 reduces this to O(|E| lg |V|).

Minimum Spanning Trees Kruskal’s Algorithm We observe that the set A is a forest, whose vertices are all those of the given graph. A safe edge is added to A: it is always a least-weight edge that connects two distinct components.

Minimum Spanning Trees Prim’s Algorithm

Minimum Spanning Trees Prim’s Algorithm The edges in the set A always make up a single tree. The tree starts from an arbitrary root vertex r and grows until it spans all vertices in V. Each step adds to the tree A a light edge that connects A to an isolated vertex (one on which no edge of A is incident) By Cor. 23.2, this process this adds only edges that are safe for A, so that, at the end, A forms an MST. The strategy is greedy, since at each step it adds to the tree an edge of minimum weight.

Minimum Spanning Trees Prim’s Algorithm: Invariant Prior to each iteration of the while loop (ll. 6-11): A = {(v, v.p) : v Î V – {r} – Q} The vertices already placed into the minimum spanning tree are those in V – Q For all vertices v Î Q, if v.p ≠ NIL, then v.key < ¥ and v.key is the weight of a light edge (v, v.p) connecting v to some vertex already placed into the MST.

Minimum Spanning Trees Prim’s Algorithm: Maintenance L. 7 extracts a vertex u in Q incident on a light edge that crosses the cut (V – Q, Q) (except for the first iteration, which extracts r). Removing u from Q adds it to V – Q of vertices in the tree, adding (u, u.p) to A. The for loop updates the key and p attributes of each vertex adjacent to u, but not in the tree, thus maintaining the third part of the loop invariant.

Minimum Spanning Trees Prim’s Algorithm: Time This will depend on the implementation of the priority queue. But; Creating a priority queue is O(|V|). Extracting elements is O(lg |V|) for each extraction. Decrease-Key operations, necessary because the keys of elements in the queue can change, will also take time O(lg |V|). Since this can happen for each edge we examine, total time can be O(|E| lg |V|).

Minimum Spanning Tree: Greedy Algorithms Invariant: Minimum weight spanning forest Becomes single tree at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Produces minimum weight tree of edges that includes every vertex. A B C D E F G 2 1 3 4 5 6 8 7 Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue Invariant: Minimum weight tree Spans all vertices at end for Undirected, Connected, Weighted Graph G=(V,E) source: 91.503 textbook Cormen et al.

Minimum Spanning Trees Review problem: For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees? A B C D E F G 2 1 3 4 5 6 8 7

Chapter 24 Shortest Paths Dijkstra [Source: Cormen et al. textbook except where noted] I joined the UMass Lowell Computer Science faculty this summer. This collection of slides is intended to familiarize the reader/viewer with my field of research (Computational Geometry), summarize my previous research results in this field and outline my plan for Computational Geometry research at UMass Lowell.

BFS as a Basis for Shortest Path Algorithms for unweighted, undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix Source/Sink Shortest Path Problem: Given 2 vertices u, v, find the shortest path in G from u to v. Solution: BFS starting at u. Stop at v. Single-Source Shortest Paths Problem: Given a vertex u, find the shortest path in G from u to each vertex. Solution: BFS starting at u. Full BFS tree. All-Pairs Shortest Paths Problem: Find the shortest path in G from each vertex u to each vertex v. Solution: For each u: BFS starting at u; full BFS tree. Time: O(|V|(|V| + |E|)) adj list O(|V|3) adj matrix source: based on Sedgewick, Graph Algorithms

Shortest Path Applications for weighted, directed graph G=(V,E) Weight ~ Cost ~ Distance Road maps Airline routes Telecommunications network routing VLSI design routing source: based on Sedgewick, Graph Algorithms

Shortest Path Trees Shortest Path Tree gives shortest path from root to each other vertex source: Sedgewick, Graph Algorithms

Shortest Path Principles: Relaxation Graph Initialization We start with a weighted, directed graph with a weight function w defined on its edges. We will generally consider a distinguished vertex, called the source, and usually denoted by s.

Shortest Path Principles: Relaxation “Relax” a constraint to try to improve solution “Rubber band” analogy [Sedgewick] Relaxation of an Edge (u,v): test if shortest path to v [found so far] can be improved by going through u A B C D E F G 2 1 3 4 5 6 8 7

Shortest Path Principles: Relaxation

Shortest Path Principles: Bellman-Ford

Dag-Shortest-Paths

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) See separate ShortestPath 91.404 slide show A B C D E F G 2 1 3 4 5 6 8 7 source: 91.503 textbook Cormen et al.

Single Source Shortest Paths Dijkstra’s Algorithm Review problem: For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? A B C D E F G 2 1 3 4 5 6 8 7 Why can’t Dijkstra’s algorithm handle negative-weight edges?

Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative) weighted, directed graph G=(V,E) source: Sedgewick, Graph Algorithms

Shortest Path Algorithms source: Sedgewick, Graph Algorithms