November 13, Algorithms and Data Structures Lecture XII Simonas Šaltenis Aalborg University

November 13, This Lecture Weighted Graphs Minimum Spanning Trees Greedy Choice Theorem Kruskal’s Algorithm Prim’s Algorithm

November 13, Spanning Tree A spanning tree of G is a subgraph which is a tree contains all vertices of G How many edges are there in a spanning tree, if there are V vertices?

November 13, Minimum Spanning Trees Undirected, connected graph G = (V,E) Weight function W: E  R (assigning cost or length or other values to edges) Spanning tree: tree that connects all the vertices Optimization problem – Minimum spanning tree (MST): tree T that connects all the vertices and minimizes

November 13, Optimal Substructure ”Cut and paste” argument If G’ would have a cheaper ST T’, then we would get a cheaper ST of G: T’ + (u, v) MST(G) = T u v ”u+v” MST(G’) = T – (u,v)

November 13, Idea for an Algorithm We have to make V – 1 choices (edges of the MST) to arrive at the optimization goal After each choice we have a sub-problem one vertex smaller than the original Dynamic programming algorithm, at each stage, would consider all possible choices (edges) If only we could always guess the correct choice – an edge that definitely belongs to an MST

November 13, Greedy Choice Greedy choice property: locally optimal (greedy) choice yields a globally optimal solution Theorem Let G=(V, E), and let S  V and let (u,v) be min-weight edge in G connecting S to V – S : a light edge crossing a cut Then (u,v)  T – some MST of G

November 13, Greedy Choice (2) Proof Suppose (u,v) is light, but (u,v)  any MST look at path from u to v in some MST T Let (x, y) – the first edge on path from u to v in T that crosses from S to V – S. Swap (x, y) with (u,v) in T. this improves T – contradiction (T supposed to be an MST) u v x y S V-S

November 13, Generic MST Algorithm Generic-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3 Find an edge (u,v) that is safe for A 4 A  A  {(u,v)} 5 return A Generic-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3 Find an edge (u,v) that is safe for A 4 A  A  {(u,v)} 5 return A Safe edge – edge that does not destroy A’s property MoreSpecific-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S 4 A  A  {(u,v)} 5 return A MoreSpecific-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S 4 A  A  {(u,v)} 5 return A

November 13, Pseudocode assumptions Graph ADT with an operation V():VertexSet E(): EdgeSet w(u:Vertex, v:Vertex):int – returns a weight of (u,v) A looping construct “for each v  V ”, where V is of a type VertexSet, and v is of a type Vertex Vertex ADT with operations: adjacent():VertexSet key():int and setkey(k:int) parent():Vertex and setparent(p:Vertex)

November 13, Prim-Jarnik Algorithm Vertex based algorithm Grows one tree T, one vertex at a time A set A covering the portion of T already computed Label the vertices v outside of the set A with v.key() – the minimum weigth of an edge connecting v to a vertex in A, v.key() = , if no such edge exists

November 13, MST-Prim(G,r) 01 for each vertex u  G.V() 02 u.setkey(  03 u.setparent(NIL) 04 r.setkey(0) 05 Q.init(G.V()) // Q is a priority queue ADT 06 while not Q.isEmpty() 07 u  Q.extractMin() // making u part of T 08 for each v  u.adjacent() do 09 if v  Q and G.w(u,v) < v.key() then 10 v.setkey(G.w(u,v)) 11 Q.modifyKey(v) 12 v.setparent(u) Prim-Jarnik Algorithm (2) updating keys

November 13, Prim-Jarnik’s Example Let’s do an example: HBCIADGFE Let’s keep track: What is set A, which vertices are in Q, what cuts are we making, what are the key values

November 13, MST-Prim(G,r) 01 for each vertex u  G.V() 02 u.setkey(  03 u.setparent(NIL) 04 r.setkey(0) 05 Q.init(G.V()) // Q is a priority queue ADT 06 while not Q.isEmpty() 07 u  Q.extractMin() // making u part of T 08 for each v  u.adjacent() do 09 if v  Q and G.w(u,v) < v.key() then 10 v.setkey(G.w(u,v)) 11 Q.modifyKey(v) 12 v.setparent(u) Implementation issues

November 13, Priority Queues A priority queue is an ADT for maintaining a set S of elements, each with an associated value called key We need PQ to support the following operations init(S:VertexSet) extractMin():Vertex modifyKey(v:Vertex) To choose how to implement a PQ, we need to count how many times the operations are performed: init is performed just once and runs in O(n)

November 13, Prim-Jarnik’s Running Time Time = |V|T(extractMin) + O(E)T(modifyKey) Binary heap implementation: Time = O(V lgV + E lgV) = O(E lgV) QT(extractMin)T(modifyKey)Total arrayO(V)O(V)O(1)O(V 2 ) binary heapO(lg V) O(E lgV ) Fibonacci heap O(lg V)O(1) amortizedO(V lgV +E )

November 13, Greedy Algorithms One more algorithm design technique Greedy algorithms can be used to solve optimization problems, if: There is an optimal substructure We can prove that a greedy choice at each iteration leads to an optimal solution.

November 13, Kruskal's Algorithm Edge based algorithm Add the edges one at a time, in increasing weight order The algorithm maintains A – a forest of trees. An edge is accepted it if connects vertices of distinct trees (the cut respects A)

November 13, Disjoint-Set ADT We need a Disjoint-Set ADT that maintains a partition, i.e., a collection S of disjoint sets. Operations: makeSet(x:Vertex): SetID S  S  {{x}} union(x:Vertex, y:Vertex) – X  S.findSet(x) Y  S.findSet(y) S  S – {X, Y}  {X  Y} findSet(x:Vertex): SetID returns unique X  S, where x  X

November 13, Kruskal's Algorithm The algorithm keeps adding the cheapest edge that connects two trees of the forest MST-Kruskal(G) 01 A   02 S.init() // Init disjoint-set ADT 03 for each vertex v  G.V() 04 S.makeSet(v) 05 sort the edges of G.E() by non-decreasing G.w(u,v) 06 for each edge (u,v)  E, in sorted order do 07 if S.findSet(u)  S.findSet(v) then 08 A  A  {(u,v)} 09 S.union(u,v) 10 return A

November 13, Kruskal’s Example Let’s do an example: HBCIADGFE Let’s keep track: What is set A, what is a collection S, what cuts are we making

November 13, Disjoint Sets as Lists Each set – a list of elements identified by the first element, all elements in the list point to the first element Union – add a smaller list to a larger one FindSet: O(1), Union(u,v): O(min{|C(u)|, |C(v)|}) 123 ABC  4  123 ABC  4

November 13, Kruskal Running Time Initialization O(V) time Sorting the edges  (E lg E) =  (E lg V) (why?) O(E) calls to FindSet Union costs Let t(v) be the number of times v is moved to a new cluster Each time a vertex is moved to a new cluster the size of the cluster containing the vertex at least doubles: t(v)  log V Total time spent doing Union Total time: O(E lg V)

November 13, Next Lecture Shortest Paths in Weighted Graphs