Minimum Spanning Tree in Graph - Week 13

2 Problem: Laying Telephone Wire Central office

3 Wiring: Naive Approach Central office Expensive!

4 Wiring: Better Approach Central office Minimize the total length of wire connecting ALL customers

5 Spanning trees Suppose you have a connected undirected graph: Connected: every node is reachable from every other node Undirected: edges do not have an associated direction...then a spanning tree of the graph is a connected subgraph which contains all the vertices and has no cycles. A connected, undirected graph Four of the spanning trees of the graph

All 16 of its Spanning TreesComplete Graph

7 Minimum-cost spanning trees Suppose you have a connected undirected graph with a weight (or cost) associated with each edge. The cost of a spanning tree would be the sum of the costs of its edges. A minimum-cost spanning tree is a spanning tree that has the lowest cost. AB ED FC A connected, undirected graph AB ED FC A minimum-cost spanning tree

8 Minimum Spanning Tree (MST) it is a tree (i.e., it is acyclic) it covers all the vertices V contains |V| - 1 edges the total cost associated with tree edges is the minimum among all possible spanning trees not necessarily unique. A minimum spanning tree is a subgraph of an undirected weighted graph G, such that Tree = connected graph without cycles

9 Applications of MST Any time you want to visit all vertices in a graph at minimum cost:  wire routing on printed circuit boards,  sewer pipe layout,  telephone lines to a set of houses,  road planning…

10 How Can We Generate a MST? a c e d b a c e d b

11 Finding minimum spanning trees There are two basic algorithms for finding minimum-cost spanning trees, and both are greedy algorithms. Kruskal’s algorithm: Start with no nodes or edges in the spanning tree, and repeatedly add the cheapest edge that does not create a cycle Kruskal’s algorithm Here, we consider the spanning tree to consist of edges only Prim’s algorithm: Start with any one node in the spanning tree, and repeatedly add the cheapest edge, and the node it leads to, for which the node is not already in the spanning tree. Prim’s algorithm Here, we consider the spanning tree to consist of both nodes and edges

12 Kruskal’s algorithm The steps are: 1. The forest is constructed - with each node in a separate tree. 2. The edges are placed in a priority queue. 3. Until we've added n-1 edges, (1). Extract the cheapest edge from the priority queue, (2). If it forms a cycle, reject it. Else add it to the forest. Adding it to the forest will join two trees together. Every step will have joined two trees in the forest together, so that at the end, there will only be one tree in T.

A BC D E F G H I J Complete Graph

A BC D E F G H I J AABD BB B CD JC C E F D DH JEG FFGI GGIJ HJJI

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Sort Edges (in reality they are placed in a priority queue - not sorted - but sorting them makes the algorithm easier to visualize)

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Cycle Don’t Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Cycle Don’t Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J A BC D E F G H I J Minimum Spanning TreeComplete Graph

Visualization of Kruskal's algorithm “repeatedly add the cheapest edge that does not create a cycle”

Time complexity of Kruskal's Algorithm Running Time = O(E log E) (E = # edges) Testing if an edge creates a cycle can be slow unless a complicated data structure called a “union-find” structure is used. This algorithm works best, of course, if the number of edges is kept to a minimum.

30 Prim’s algorithm The steps are: 1. Initialize a tree with a single node arbitrarily chose from graph. 2. Repeat until all nodes are in the tree: (1). Find the node from the graph with the smallest connecting edge to the tree, (2). Add it to the tree Every step will have joined one node, so that at the end we will have one new graph with all the nodes and it will be a minimum spanning tree of the original graph.

A BC D E F G H I J Complete Graph

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Complete GraphMinimum Spanning Tree A BC D E F G H I J

 Unlike Kruskal’s, Prim’s algorithm doesn’t need to see all of the graph at once. It can deal with it one piece at a time.  It also doesn’t need to worry if adding an edge will create a cycle since this algorithm deals primarily with the nodes, and not the edges. Summary of Prim's Algorithm

Time complexity of Prim's Algorithm Running Time = O(E + V log V) (E = #edges, V = #nodes)

45 Time Complexity Review Kruskal’s algorithm: O(e log e) Prim’s algorithm: O( e log v) Kruskal’s algorithm is preferable on sparse graphs, i.e., where e is very small compared to the total number of possible edges.  Prim’s algorithm is easy to implemented, but the number of vertices needs to be kept to a minimum in addition to the number of edges.

1. How does Google quickly find web pages that contain a search term? AnswerAnswer 2. How can a subsequence of DNA be quickly found within the genome? AnswerAnswer 3. How does your operating system track which memory (disk or RAM) is free? AnswerAnswer 4. How to generate a Maze game? Answer, Example implementationAnswerExample implementation 5. Etc… Slide 11 on the Week 1 – Introduction of the class “Example applications of data structures and algorithms” 46

47 Mazes Typically, Every location in a maze is reachable from the starting location There is only one path from start to finish If the cells are “vertices” and the open doors between cells are “edges,” this describes a spanning tree Since there is exactly one path between any pair of cells, any cells can be used as “start” and “finish” This describes a spanning tree

48 Mazes as spanning trees While not every maze is a spanning tree, most can be represented as such. The nodes are “places” within the maze. There is exactly one cycle- free path from any node to any other node.

49 Building a maze I This algorithm requires two sets of cells the set of cells already in the spanning tree, IN the set of cells adjacent to the cells in the spanning tree (but not in it themselves), called the FRONTIER Start with all walls present Pick any cell and put it into IN (red) Put all adjacent cells, that aren’t in IN, into FRONTIER (blue)

50 Building a maze II Repeatedly do the following: Remove any one cell C from FRONTIER and put it in IN Erase the wall between C and some one adjacent cell in IN  Add to FRONTIER all the cells adjacent to C that aren’t in IN (or in FRONTIER already) Continue until there are no more cells in FRONTIER When the maze is complete (or at any time), choose the start and finish cells

Generating a maze by Prim’s algorithm Prim's algorithm has many applications, such as in the generation of this maze, which applies Prim's algorithm to a randomly weighted grid graph.generationgrid graph Click on to see the Demo.Demo 51