Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos and Alexandra Stefan University of Texas at Arlington These slides are based on “Algorithms in C” by R. Sedgewick

Weighted Graphs Each edge has a weight. 1 7 2 5 3 4 6 10 20 30 15 18 25 Each edge has a weight. Example: a transportation network (roads, railroads, subway). The weight of each road can be: The length. The expected time to traverse. The expected cost to build. Example: in a computer network, the weight of each edge (direct link) can be: Latency. Expected cost to build.

Spanning Tree Minimum-cost spanning tree (MST): Connects all vertices of the graph (spanning tree). Has the smallest total weight of edges. A spanning tree is a tree that connects all vertices of the graph. The weight of a spanning tree is the sum of weight of its edges. 1 7 2 5 3 4 6 10 20 30 15 18 25 20 6 20 30 2 10 1 15 7 30 3 15 25 10 5 4 18 Weight:20+15+30+20+30+25+18 = 158 Weight:10+20+15+30+20+10+15 = 120

Minimum-Cost Spanning Tree (MST) 1 7 2 5 3 4 6 10 20 30 15 18 25 Minimum-Cost Spanning Tree (MST) Assume that the graph is: connected undirected edges can have negative weights. Warning: later in the course (when we discuss Dijkstra's algorithm) we will make the opposite assumptions: Allow directed graphs. Not allow negative weights. 1 7 2 5 3 4 6 10 20 30 15

Prim's Algorithm - Overview Start from an tree that contains a single vertex. Keep growing that tree, by adding at each step the ‘shortest’ edge connecting a vertex in the tree to a vertex outside the tree. We will cover the CLRS version. Sedgewick gives other implementations

Example 1 7 2 5 3 4 6 10 20 30 15 18 25 Start by adding a vertex, r, (here r = 0) to the MST (minimum-cost spanning tree). Repeat until all vertices have been added to the tree: From all edges connecting vertices from the current tree to vertices outside the current tree, select the smallest edge. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.

Example 2. Repeat until all vertices have been added to the tree: 6 2 a) Select the smallest edge from all edges connecting vertices from the current tree to vertices outside the tree. b) Add to the tree that edge, and non-tree vertex of it. 1 7 2 5 3 4 6 20 Red - current MST Purple - potential edges Blue – unprocessed edges and vertices. 20 30 10 15 30 15 25 10 18

Example 2. Repeat until all vertices have been added to the tree: a) Select the smallest edge from all edges connecting vertices from the current tree to vertices outside the tree. b) Add to the tree that edge, and non-tree vertex of it. 1 7 2 5 3 4 6 20 1 7 2 5 3 4 6 20 20 30 20 30 10 10 15 15 30 30 15 25 15 25 10 10 18 18

Example 2. Repeat until all vertices have been added to the tree: a) Select the smallest edge from all edges connecting vertices from the current tree to vertices outside the tree. b) Add to the tree that edge, and non-tree vertex of it. 1 7 2 5 3 4 6 20 20 30 10 15 30 15 25 10 18

Example Note that (5,4) with weight 18 is not picked because it connects vertices that are already in the tree. 1 7 2 5 3 4 6 20 1 7 2 5 3 4 6 20 20 30 20 30 10 10 15 15 30 30 15 25 25 10 15 10 18 18

Example 1 7 2 5 3 4 6 20 1 7 2 5 3 4 6 20 20 30 20 30 10 10 15 15 30 30 15 25 25 10 15 10 18 18

Example 2 NOTE: This graph has some new edges added and weights changed. Transition to implementation: 1. Note that for each vertex that is not in the MST, we need to know what is the shortest edge that connects it to the MST. 2. As we add one vertex to the tree, update its neighbors if: They are not in the tree This vertex ‘brings them closer’ 1 7 2 5 3 4 6 20 4 20 30 10 15 18 9 30 15 25 5 12

Example 2 Transition to implementation: 6 2 1 7 3 5 4 1. Note that for each vertex that is not in the MST, we need to know what is the shortest edge that connects it to the MST. 2. As we add one vertex to the tree, update its neighbors if: They are not in the tree This vertex ‘brings them closer’ 1 7 2 5 3 4 6 20 4 20 30 10 15 18 9 30 Note that as we added 5, for vertex 3 We record that edge (5,3, 15) is the ‘shortest’ edge to connect 3 to the MST ( not (0,3,18) ). Convention: (u, v, weight(u,w)) 15 25 5 12

Example 2 Transition to implementation: 6 2 1 7 3 5 4 1. Note that for each vertex that is not in the MST, we need to know what is the shortest edge that connects it to the MST. 2. As we add one vertex to the tree, update its neighbors if: They are not in the tree This vertex ‘brings them closer’ 1 7 2 5 3 4 6 20 4 20 30 10 15 18 9 30 Note that as we added 4, edge (4, 3, 25) is not a better connection for 3 and so we still keep edge (5,3,15) for vertex 3). Similar to vertex 3 before, for vertex 7 we mark edge (4,7,10) as the best (instead of (0,7,15) ). 15 25 5 12

Example 2 1 7 2 5 3 4 6 20 1 7 2 5 3 4 6 20 4 4 20 30 20 30 10 10 15 15 18 9 18 9 30 30 15 25 15 25 5 5 12 12 Now (7,2,9) is recorded for vertex 2. Now (2,6,4) is recorded for vertex 6.

1 7 2 5 3 4 6 20 4 20 30 10 15 18 9 30 15 25 5 12 1 7 2 5 3 4 6 20 1 7 2 5 3 4 6 20 4 4 20 30 20 30 10 10 15 15 18 9 18 9 30 30 15 25 15 25 5 5 12 12

Prim’s Algorithm (CLRS) Q – is a priority queue Time complexity:

Prim’s Algorithm (CLRS) Q – is a priority queue Time complexity: O(V + VlgV + E lgV) = O(ElgV) connected graph => |E| ≥ (|V|-1) O(V) O(V) (build heap) O(V) O(V*lgV) O(lgV) Lines 6 & 8 together: O(E) (touch each edge twice) O(E*lgV) Decrease-priority: O(lgV)

Prim’s Algorithm (CLRS) See if v is in Q. ANS: Find index of v in Q Note difference between v and node in heap corresponding to v. See heap slides : Index Heap Example

Kruskal's Algorithm

Kruskal's Algorithm: Overview Kruskal's algorithm works with a forest (a set of trees). Initially, each vertex in the graph is its own tree. Keep merging trees together, until end up with a single tree.

Kruskal's Algorithm: Overview Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. The abstract description is simple, but we need to think carefully about how exactly to implement these steps as it affects the runtime. Sort edges: Y/N? Put edges in a min-heap?

Kruskal’s Algorithm Example 2 graph with edge (3,1, 11) added. Algorithm (based on a forest of trees): Initially, each vertex in the graph is its own tree. Keep merging trees together, until end up with a single tree. Order u, v, weight 1st 2nd … Last 1 7 2 5 3 4 6 20 4 20 30 10 18 15 9 11 30 25 15 5 12

Extra Page 1 7 2 5 3 4 6 20 4 20 30 10 18 15 9 11 30 25 15 5

Kruskal's Algorithm: Worked Out Example 1 Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. (work-out example) 1 7 2 5 3 4 6 10 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 1 7 2 5 3 4 6 10 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 10 1 7 2 5 3 4 6 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 10 1 7 2 5 3 4 6 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 10 1 7 2 5 3 4 6 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 10 1 7 2 5 3 4 6 20 30 15 25

Kruskal's Algorithm: An Example Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Repeat until the forest contains a single tree: Find the shortest edge F connecting two trees in the forest. Connect those two trees into a single tree using edge F. 10 1 7 2 5 3 4 6 20 30 15

Definitions (CLRS, pg 625) A cut (S, V-S) of an graph is a partition of its vertices, V. An edge (u,v) crosses the cut (S, V-S) if one of its endpoints is in S and the other in V-S. Let A be a subset of a minimum spanning tree over G. An edge (u,v) is safe for A if A {(u,v)} is still a subset of a minimum spanning tree. A cut respects a set of edges, A, if no edge in A crosses the cut. An edge is a light edge crossing a cut if its weight is the minimum weight of any edge crossing the cut.

Correctness of Prim and Kruskall (CLRS, pg 625) Let G = (V,E) be a connected, undirected, weighted graph. Let A be a subset of some minimum spanning tree, T, for G, let (S, V-S) be some cut of G that respects A, and let (u,v) be a light edge crossing (S, V-S). Then, edge (u,v) is safe for A. Proof: If (u,v) is part of T, done Else, in T, u and v must be connected through another path, p. One of the edges of p, must connect a vertex x from A and a vertex, y, from V-A. Adding edge(u,v) to T will create a cycle with the path p. (x,y) also crosses (A, V-A) and (u,v) is light => weight(u,v) <=

Note There may be several different MST in a graph, but they will all have the same weight (smallest possible).