1. CSCE350 Algorithms and Data Structure
Lecture 18
Jianjun Hu
Department of Computer Science and Engineering
University of South Carolina

2. Chapter 9: Greedy algorithms
Change-making problem
Coin-system in US: 25(quarter), 10 (dime), 5(nickel), 1(penny)
If you need to give a change of 48 cents using coins,
48 cents = 1 quarter + 2 dimes + 3 pennies
This is a greedy algorithm: reduce the amount in the fastest way
The greedy approach constructs a solution through a sequence of steps until a complete solution is reached, On each step, the choice made must be
Feasible: Satisfy the problem’s constraints
locally optimal: the best choice
Irrevocable: Once made, it cannot be changed later
Not all optimization problems can be approached in this manner!

3. Applications of the Greedy Strategy
Optimal solutions:
change making
Minimum Spanning Tree (MST)
Single-source shortest paths
simple scheduling problems
Huffman codes
Approximations:
Traveling Salesman Problem (TSP)
Knapsack problem
other combinatorial optimization problems

4. Minimum Spanning Tree (MST)
Motivation: Build a transportation system with the minimum length in a city  a tree structure (a connected acyclic graph)
Spanning tree of a connected graph G: a connected acyclic subgraph of G that includes all of G’s vertices.
Minimum Spanning Tree of a weighted, connected graph G: a spanning tree of G of minimum total weight.
Example: 3 4 2 1 6 7 3 4 2 1 6 7

5. Prim’s MST algorithm Start with tree consisting of one vertex
“grow” tree one vertex/edge at a time to produce MST
Construct a series of expanding subtrees T1, T2, …
at each stage construct Ti+1 from Ti: add minimum weight edge connecting a vertex in tree (Ti) to one not yet in tree
choose from “fringe” edges
(this is the “greedy” step!)
algorithm stops when all vertices are included

6. An Example:
Finding the MST of the following graph using Prim’s algorithm b d f a c 3 1 6 4 5 e 2 8

7. Step 1:
Start from empty tree T, pick one vertex, a(-,-) and add it to T
Remaining vertices: b(a,3), c(-,∞), d(-,∞), e(a,6), f(a,5) b d f a c 3 1 6 4 5 e 2 8

8. Step 2:
Add the minimum-weight fringe edge b(a,3) into T
Remaining vertices: c(b,1), d(-,∞), e(a,6), f(b,4) b d f a c 3 1 6 4 5 e 2 8

9. Step 3:
Add the minimum-weight fringe edge c(b,1) into T
Remaining vertices: d(c,6), e(a,6), f(b,4) b d f a c 3 1 6 4 5 e 2 8

10. Step 4:
Add the minimum-weight fringe edge f(b,4) into T
Remaining vertices: d(f,5), e(f,2) b d f a c 3 1 6 4 5 e 2 8

11. Step 5:
Add the minimum-weight fringe edge e(f,2) into T
Remaining vertices: d(f,5) b d f a c 3 1 6 4 5 e 2 8

12. Step 6:
Add the minimum-weight fringe edge d(f,5) into T
No remaining vertices and the algorithm is done! b d f a c 3 1 6 4 5 e 2 8

13. Does Prim’s Algorithm Really Produce MST?
Lemma: Let X be any subset of the vertices of G, and let edge e be the smallest-weight edge connecting X (tree Ti-1) to G-X (remaining vertices). Then e (minimum-weight fringe edge) is part of the MST
Proof: Using contradiction, suppose e=(u,v) is not part of MST. Then adding e to MST results in a cycle, which contains another edge e’=(u’,v’) between X and G-X. Replace e’ by e will result in a spanning tree with smaller total weight than MST. Contradiction!

14. Notes on Prim’s algorithm
Pseudocode can be found in Section 9.1
To locate the minimum-weight fringe edge, we can use the heap structure. But here we use the min-heap, where the root has a key smaller than both children
Delete the min – O(log |V|), it can to be performed |V|-1 times
Verify minimum weight from any remaining vertex to the tree – this may be performed |E| times. Each verification may result in a key priority change in the heap, which takes O(log |V|).
Therefore, the total complexity is
(|V|-1+|E|)O(log |V|)=O(|E| log |V|)

15. Another Greedy algorithm for MST: Kruskal
Start with empty forest of trees
“grow” MST one edge at a time
intermediate stages usually have forest of trees (not connected)
at each stage add minimum weight edge among those not yet used that does not create a cycle
edges are initially sorted by increasing weight
at each stage the edge may:
expand an existing tree
combine two existing trees into a single tree
create a new tree
need efficient way of detecting/avoiding cycles
algorithm stops when all vertices are included

16. An Example:
Finding the MST of the following graph using Kruskal’s algorithm b d f a c 3 1 6 4 5 e 2 8

17. Step 1: Sorted List of Edgesbc ef ab bf cf af df ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

18. Step 2: add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf afae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

19. Step 3: Add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf afae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

20. Step 4: add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf afae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

21. Step 5: add the next edge (without getting a cycle)bc ef ab bf cf af df ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

22. Step 6: Finish the MST b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf af dfae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

23. Does the Kruskal’s algorithm Produce MST
Each newly added edge e=(u,v) is the minimum-weight edge between S (all the vertices that are reachable from u in the current forest) and the remaining vertices.
According to the previous lemma, such an edge should be in MST.

24. Notes on Kruskal’s algorithm
Algorithm looks easier than Prim’s but is
harder to implement (checking for cycles!)
less efficient Θ(|E| log |E|)
Cycle checking: a cycle exists iff edge connects vertices in the same component.
Union-find algorithms – see section 9.2