1. Total running time is O(E lg E).
MST-Kruskal(G, w)
1. A   // initially A is empty
2. for each vertex v  V[G] // line 2-3 takes O(V) time
do Create-Set(v) // create set for each vertex
4. sort the edges of E by nondecreasing weight w
5. for each edge (u,v)  E, in order by nondecreasing weight
do if Find-Set(u)  Find-Set(v) // u&v on different trees
then A  A  {(u,v)}
Union(u,v)
9. return A
Total running time is O(E lg E).
UNC Chapel Hill
Lin/Foskey/Manocha

2. Analysis of Kruskal Lines 1-3 (initialization): O(V)
Line 4 (sorting): O(E lg E)
Lines 6-8 (set operations): O(E log E)
Total: O(E log E)
UNC Chapel Hill
Lin/Foskey/Manocha

3. Correctness of Kruskal
Idea: Show that every edge added is a safe edge for A
Assume (u, v) is next edge to be added to A.
Will not create a cycle
Let A’ denote the tree of the forest A that contains vertex u. Consider the cut (A’, V-A’).
This cut respects A (why?)
and (u, v) is the light edge across the cut (why?)
Thus, by the MST Lemma, (u,v) is safe.
UNC Chapel Hill
Lin/Foskey/Manocha

4. Intuition behind Prim’s Algorithm
Consider the set of vertices S currently part of the tree, and its complement (V-S). We have a cut of the graph and the current set of tree edges A is respected by this cut.
Which edge should we add next? Light edge!
UNC Chapel Hill
Lin/Foskey/Manocha

5. Basics of Prim ’s Algorithm
It works by adding leaves on at a time to the current tree.
Start with the root vertex r (it can be any vertex). At any time, the subset of edges A forms a single tree. S = vertices of A.
At each step, a light edge connecting a vertex in S to a vertex in V- S is added to the tree.
The tree grows until it spans all the vertices in V.
Implementation Issues:
How to update the cut efficiently?
How to determine the light edge quickly?
UNC Chapel Hill
Lin/Foskey/Manocha

6. Implementation: Priority Queue
Priority queue implemented using heap can support the following operations in O(lg n) time:
Insert (Q, u, key): Insert u with the key value key in Q
u = Extract_Min(Q): Extract the item with minimum key value in Q
Decrease_Key(Q, u, new_key): Decrease the value of u’s key value to new_key
All the vertices that are not in the S (the vertices of the edges in A) reside in a priority queue Q based on a key field. When the algorithm terminates, Q is empty. A = {(v, [v]): v  V - {r}}
UNC Chapel Hill
Lin/Foskey/Manocha

7. Example: Prim’s Algorithm
UNC Chapel Hill
Lin/Foskey/Manocha

8. MST-Prim(G, w, r)
1. Q  V[G]
2. for each vertex u  Q // initialization: O(V) time
do key[u]  
4. key[r]  0 // start at the root
5. [r]  NIL // set parent of r to be NIL
6. while Q   // until all vertices in MST
do u  Extract-Min(Q) // vertex with lightest edge
for each v  adj[u]
do if v  Q and w(u,v) < key[v]
then [v]  u
key[v]  w(u,v) // new lighter edge out of v
decrease_Key(Q, v, key[v])
UNC Chapel Hill
Lin/Foskey/Manocha

9. Analysis of Prim Extracting the vertex from the queue: O(lg n)
For each incident edge, decreasing the key of the neighboring vertex: O(lg n) where n = |V|
The other steps are constant time.
The overall running time is, where e = |E|
T(n) = uV(lg n + deg(u) lg n) = uV (1+ deg(u)) lg n
= lg n (n + 2e) = O((n + e) lg n)
Essentially same as Kruskal’s: O((n+e) lg n) time
UNC Chapel Hill
Lin/Foskey/Manocha

10. Correctness of Prim
Again, show that every edge added is a safe edge for A
Assume (u, v) is next edge to be added to A.
Consider the cut (A, V-A).
This cut respects A (why?)
and (u, v) is the light edge across the cut (why?)
Thus, by the MST Lemma, (u,v) is safe.
UNC Chapel Hill
Lin/Foskey/Manocha

11. Optimization Problems
In which a set of choices must be made in order to arrive at an optimal (min/max) solution, subject to some constraints. (There may be several solutions to achieve an optimal value.)
Two common techniques:
Dynamic Programming (global)
Greedy Algorithms (local)
UNC Chapel Hill
Lin/Foskey/Manocha

12. Dynamic Programming
Similar to divide-and-conquer, it breaks problems down into smaller problems that are solved recursively.
In contrast to D&C, DP is applicable when the sub-problems are not independent, i.e. when sub-problems share sub-subproblems. It solves every sub-subproblem just once and saves the results in a table to avoid duplicated computation.
UNC Chapel Hill
Lin/Foskey/Manocha

13. Elements of DP Algorithms
Substructure: decompose problem into smaller sub-problems. Express the solution of the original problem in terms of solutions for smaller problems.
Table-structure: Store the answers to the sub-problem in a table, because sub-problem solutions may be used many times.
Bottom-up computation: combine solutions on smaller sub-problems to solve larger sub-problems, and eventually arrive at a solution to the complete problem.
UNC Chapel Hill
Lin/Foskey/Manocha

14. Applicability to Optimization Problems
Optimal sub-structure (principle of optimality): for the global problem to be solved optimally, each sub-problem should be solved optimally. This is often violated due to sub-problem overlaps. Often by being “less optimal” on one problem, we may make a big savings on another sub-problem.
Small number of sub-problems: Many NP-hard problems can be formulated as DP problems, but these formulations are not efficient, because the number of sub-problems is exponentially large. Ideally, the number of sub-problems should be at most a polynomial number.
UNC Chapel Hill
Lin/Foskey/Manocha