Minimum Spanning Trees CSE 373 Data Structures

CSE 373 AU 05 - Minimum Spanning Trees Given (connected) graph G(V,E), a spanning tree T(V’,E’): Is a subgraph of G; that is, V’  V, E’  E. Spans the graph (V’ = V) Forms a tree (no cycle); So, E’ has |V| -1 edges 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Minimum Spanning Trees Edges are weighted: find minimum cost spanning tree Applications Find cheapest way to wire your house Find minimum cost to send messages on the Internet 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Strategy for Minimum Spanning Tree For any spanning tree T, inserting an edge enew not in T creates a cycle But Removing any edge eold from the cycle gives back a spanning tree If enew has a lower cost than eold we have progressed! 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Strategy Strategy for construction: Add an edge of minimum cost that does not create a cycle (greedy algorithm) Repeat |V| -1 times Correct since if we could replace an edge with one of lower cost, the algorithm would have picked it up 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Two Algorithms Prim: (build tree incrementally) Pick lower cost edge connected to known (incomplete) spanning tree that does not create a cycle and expand to include it in the tree Kruskal: (build forest that will finish as a tree) Pick lowest cost edge not yet in a tree that does not create a cycle. Then expand the set of included edges to include it. (It will be somewhere in the forest.) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A B C D F E 10 1 5 8 3 6 2 4 Starting from empty T, choose a vertex at random and initialize V = {A}, E’ ={} G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A Choose the vertex u not in V such that edge weight from u to a vertex in V is minimal (greedy!) V={A,C} E’= {(A,C) } 10 5 1 3 8 B C D 1 1 6 4 2 F E G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm Repeat until all vertices have been chosen Choose the vertex u not in V such that edge weight from v to a vertex in V is minimal (greedy!) V= {A,C,D} E’= {(A,C),(C,D)} V={A,C,D,E} E’={(A,C),(C,D),(D,E)} …. V={A,C,D,E,B,F,G} E’={(A,C),(C,D),(D,E), . (E,B),(B,F),(E,G)} A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Prim’s algorithm Repeat until all vertices have been chosen Final Cost: 1 + 3 + 4 + 1 + 1 + 6 = 16 A B C D F E 10 1 5 8 3 6 2 4 G 6 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Prim’s Algorithm Implementation Assume adjacency list representation Initialize connection cost of each node to “inf” and set “unvisited” Choose one node, say v and set cost[v] = 0 and prev[v] =0 While they are unmarked nodes Select the unvisited node u with minimum cost; mark it For each unvisited node w adjacent to u if weight(u,w) < cost(w) then cost(w) := weight (u,w) prev[w] = u How is this different from Dijkstra’s alg? 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Prim’s algorithm Analysis If the “Select the unmarked node u with minimum cost” is done with binary heap then O((|V|+|E|)log |V|) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Interlude: Some easy problems Topological sort Shortest path 2-coloring (bipartite?) Network flow (max flow, min cut) Minimum spanning tree Planarity Eulerian path (visit each edge) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Eulerian Path Visit all edges (bridges) exactly once Vertices w/ odd degrees must be endpoints A path can’t have more than two endpoints 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Some hard problems 2-SAT (not a graph problem per se) Subset sum (not a graph problem per se) Traveling Salesman Problem (TSP) Graph isomorphism Vertex coloring Vertex covering Independent set Hamiltonian path (visit each vertex) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Easy & Hard Problems complexity class P: decision problem computed in polynomial time complexity class NP: a solution is verifiable in polynomial time complexity class NP-complete: if C in NP-C and C in P, then P=NP Does P=NP??? 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Reducing hard problems Is it possible to have a fast solution to X? No, if I could use X to solve Y, and Y is proven to be slow. Suppose I were told that Hamiltonian path is a hard problem. We can prove that TSP is also a hard problem, because TSP w/ edges all weighted 1 solves Hamiltonian. 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Function/Optimization problems vs. Decision problems Find the smallest number N such that… Is there an N such that… Formulate function problem through a binary search of decision problems.. Is there an N such that.. N/2.. N/4.. 3N/8.. 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Primality and Factoring Polynomial: Is a number prime? Hard: What are the factors of an n-bit #? Hard: Does n-bit # have a factor w/ less than m bits? How hard? Current best alg is exponential.. Good, for encryption. But there’s no proof it’s NP-C… it could be “easier” than TSP. Quantum computer could factor in poly-time 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

What if you need to solve a hard problem quickly? Is n small enough that brute force is fine? Can you add assumptions/heuristics? Is an approximate solution good enough? e.g. for TSP, when triangle inequality holds, MST provides a factor-2 approx 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Kruskal’s Algorithm Select edges in order of increasing cost Accept an edge to expand tree or forest only if it does not cause a cycle Implementation using adjacency list, priority queues and disjoint sets 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Kruskal’s Algorithm Initialize a forest of trees, each tree being a single node Build a priority queue of edges with priority being lowest cost Repeat until |V| -1 edges have been accepted { Deletemin edge from priority queue If it forms a cycle then discard it else accept the edge – It will join 2 existing trees yielding a larger tree and reducing the forest by one tree } The accepted edges form the minimum spanning tree 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Detecting Cycles If the edge to be added (u,v) is such that vertices u and v belong to the same tree, then by adding (u,v) you would form a cycle Therefore to check, Find(u) and Find(v). If they are the same discard (u,v) If they are different Union(Find(u),Find(v)) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Properties of trees in K’s algorithm Vertices in different trees are disjoint True at initialization and Union won’t modify the fact for remaining trees Trees form equivalent classes under the relation “is connected to” u connected to u (reflexivity) u connected to v implies v connected to u (symmetry) u connected to v and v connected to w implies a path from u to w so u connected to w (transitivity) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

K’s Algorithm Data Structures Adjacency list for the graph To perform the initialization of the data structures below Disjoint Set ADT’s for the trees (recall Up tree implementation of Union-Find) Binary heap for edges 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Example A 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Initialization A Initially, Forest of 6 trees F= {{A},{B},{C},{D},{E},{F}} Edges in a heap (not shown) 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 1 A Select edge with lowest cost (B,E) Find(B) = B, Find(E) = E Union(B,E) F= {{A},{B,E},{C},{D},{F}} 1 edge accepted 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 2 A Select edge with lowest cost (B,F) Find(B) = B, Find(F) = F Union(B,F) F= {{A},{B,E,F},{C},{D}} 2 edges accepted 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 3 A Select edge with lowest cost (A,C) Find(A) = A, Find (C) = C Union(A,C) F= {{A,C},{B,E,F},{D}} 3 edges accepted 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 4 A Select edge with lowest cost (E,F) Find(E) = B, Find (F) = B Do nothing F= {{A,C},{B,E,F},{D}} 3 edges accepted 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 5 A Select edge with lowest cost (C,D) Find(C) = A, Find (D) = D Union(A,D) F= {{A,C,D},{B,E,F}} 4 edges accepted 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

CSE 373 AU 05 - Minimum Spanning Trees Step 6 Select edge with lowest cost (D,E) Find(D) = A, Find (E) = B Union(A,B) F= {{A,C,D,B,E,F}} 5 edges accepted : end Total cost = 10 Although there is a unique spanning tree in this example, this is not generally the case A 10 5 1 8 3 B C D 6 1 1 4 2 F E 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Kruskal’s Algorithm Analysis Initialize forest O(n) Initialize heap O(m), m = |E| Loop performed m times In the loop one deleteMin O(log m) Two Find, each O(log n) One Union (at most) O(1) So worst case O(m log m) = O(m log n) 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees

Time Complexity Summary Recall that m = |E| = O(V2) = O(n2 ) Prim’s runs in O((|V|+|E|) log |V|) Kruskal’s runs in: O(|V| log |E|) = O(|V| log |V|) In practice, Kruskal has a tendency to run faster since graphs might not be dense and not all edges need to be looked at in the deleteMin operations 11/29/2018 CSE 373 AU 05 - Minimum Spanning Trees