1. Lecture 8 Greedy Approach
Minimum spanning tree
How to design greedy algorithms

2. Roadmap Minimum spanning tree How to design greedy algorithms
Change-making problem
Activity selection problem
Huffman code
Knapsack problem

3. MST Given. Undirected graph G with positive edge weights (connected).
Def. A spanning tree of G is a subgraph T that is connected and acyclic.
not connected 23 10 21 14 24 16 4 18 9 7 11 8 5 6 23 10 21 14 24 16 4 18 9 7 11 8 5 6
not acyclic

4. Minimum spanning tree a b h c i d e f g b c d a i e h g f
Problem: MinSpanning
Input: A connected undirected graph G = (V , E ) in which each edge has a weighted length
Output: A spanning tree of G that has minimum cost a b h c i d e f g 4 2 7 10 9 8 11 1 6 14 8 7 b c d 9 4 2 a i e 11 14 4 7 6 10 8 h g f 1 2

5. Cut property b c d a i e h g f 8 7 9 4 2 11 14 4 7 6 10 8 1 2
Definition: Cut
A cut {S,T} is a partition of the vertex set V into two subsets S and T.
Theorem (Cut property)Given any cut, the crossing edge of min weight is in some MST.

6. Correctness
Theorem
Kruskal algorithm and Prim algorithm correctly find a minimum cost spanning tree.

7. Kruskal algorithm b c d a i e h g f
(h,g), (c,i), (g,f), (a,b), (c,f), (c,d), (i,g), (i,h), (b,c), (a,h), (d,e), (e,f), (b,h) 8 7 b c d 9 4 2 a i e 11 14 4 7 6 10 8 h g f 1 2

8. Kruskal algorithm
Θ(mlogm)

9. Prim algorithm 8 7 b c d 4 9 2 a i e 11 14 4 7 6 10 8 h g f 1 2

10. Θ(n2)
Heap: Θ(mlogn)

11. Comparison General Dense Prim O(mlogn) O(m) Kruskal O(mlogm) Dijkstra

12. Comparison
Kruskal demo
Prim demo

13. Greed is good.
Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures, the essence of the evolutionary spirit. Greed, in all of its forms; greed for life, for money, for love, knowledge, has marked the upward surge of mankind and greed, you mark my words, will not only save Teldar Paper, but that other malfunctioning corporation called the U.S.A.
---- Michael Douglas, Wall Street
click Wall Street movie image to play clip from Wall Street (iconic film about 1980s excess - directed by Oliver Stone and starring Michael Douglas)
Wall Street 2 (directed by Oliver Stone and starring Michael Douglas) is the sequel revolving around the 2008 stock market crash.

14. Dijkstra algorithm, Prim algorithm, Kruskal algorithm
Greedy approach
A greedy algorithm always makes the choice that looks best at the moment.
locally optimal choices --> a globally optimal solution.
Greedy algorithms DO NOT always yield optimal solutions, but for many problems they do.
Dijkstra algorithm, Prim algorithm, Kruskal algorithm

15. Where are we? Minimum spanning tree How to design greedy algorithms
Change-making problem
Activity selection problem
Huffman code
Knapsack problem

16. Change making 1 2 5 1 2 7 10 16? Problem: ChangeMaking
Input: an integer m and a coin system a1, a2, ..., an
Output: the minimum number of coins needed
16?

17. Activity selection problem
Problem: ActivitySelection
Input: a set S = {a1, a2, ... , an} of n proposed activities, each ai has a starting time si and a finishing time fi with 0 ≤ si < fi <∞
Output: A maximum-size subset of compatible activities

18. Activity-selection Theorem Consider any nonempty subproblem Sk, and
let am be an activity in Sk with the earliest finish time.
Then am is included in some maximum-size subset of mutually compatible activities of Sk .

19. Huffman code
Robert Fano
Claude Shannon
David Huffman

20. Ambiguity
Morse code:
SOS ?
V7 ?
IAMIE ?
EEWNI ?

21. Prefix-free code
Prefix code: no codeword is a prefix of some other codeword.
Use a binary tree to represent a prefix-free code.

22. Huffman code a b c d e f Frequency Fixed-length Huffman code45 13 12 16 9 5
Fixed-length
000
001
010
011
100
101
Huffman code
111
1101
1100
Robert M. Fano,
100,000 characters, Fixed length code: 300,000 bits
Huffman code: 224,000 bits

23. Huffman code O(nlogn) a b c d e f 45 13 12 16 9 5 100 1 a:45 55 1 251
a:45 55 1 25 30 1 1
c:12
b:13 14
d:16 1
f:5
e:9

24. Huffman code

25. Correctness
Lemma
Let x and y be two characters in C having the lowest frequencies.
Then there exists an optimal prefix code for C in which x and y have the same length and differ only in the last bit.
Lemma
Let x and y be two characters in C having the lowest frequencies.
C’ = C-{x,y} +{z}, fz= fx + fy.
If T’ is an optimal prefix code for C’,
then T=T’+{(z,x),(z,y)} is an optimal prefix code for C.
Theorem
Procedure HUFFMAN produces an optimal prefix code.

26. Knapsack problem Problem: Knapsack
Input: A set of items U = {u1,...,un} with sizes s1,s2,...,sn and values v1,v2,...,vn and a knapsack capacity C
Output: The maximum value that can be put into the knapsack

27. What if the thief can take fractions of items?
Knapsack problem
What if the thief can take fractions of items?
0-1 knapsack

28. Conclusion Minimum spanning tree How to design greedy algorithms
Change-making problem
Activity selection problem
Huffman code
Knapsack problem