1. Binhai Zhu Computer Science Department, Montana State University
Greedy Methods
Binhai Zhu
Computer Science Department, Montana State University
Frequently, presenters must deliver material of a technical nature to an audience unfamiliar with the topic or vocabulary. The material may be complex or heavy with detail. To present technical material effectively, use the following guidelines from Dale Carnegie Training®.
Consider the amount of time available and prepare to organize your material. Narrow your topic. Divide your presentation into clear segments. Follow a logical progression. Maintain your focus throughout. Close the presentation with a summary, repetition of the key steps, or a logical conclusion.
Keep your audience in mind at all times. For example, be sure data is clear and information is relevant. Keep the level of detail and vocabulary appropriate for the audience. Use visuals to support key points or steps. Keep alert to the needs of your listeners, and you will have a more receptive audience.
11/23/2018

2. Idea You want to maximize a global function, which could be hard
(1) You always make the best local decision in the hope of getting the overall best solution.
(2) The idea is natural (and in some situation it is the best a human can do).
11/23/2018

3. Idea You want to maximize a global function, which could be hard
(1) You always make the best local decision in the hope of getting the overall best solution.
(2) The idea is natural (and in some situation it is the best a human can do).
(3) Of course, sometimes it might not work.
11/23/2018

4. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
Remember that in this case a penny and a dime is considered the same (possibly in weight).
11/23/2018

5. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
Remember that in this case a penny and a dime is considered the same (possibly in weight).
Let’s first look at the US system. 1¢
25¢ 5¢
10¢
11/23/2018

6. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $2.17? 1¢
25¢ 5¢
10¢
11/23/2018

7. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $2.17?
8 quarters = $2.00
1 dime = $0.10
1 nickel = $0.05
2 pennies = $0.02
So we use 12 coins! 1¢
25¢ 5¢
10¢
11/23/2018

8. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $2.17?
8 quarters = $2.00
1 dime = $0.10
1 nickel = $0.05
2 pennies = $0.02
Why this works? 1¢
25¢ 5¢
10¢
11/23/2018

9. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $2.17?
8 quarters = $2.00
1 dime = $0.10
1 nickel = $0.05
2 pennies = $0.02
Why this works?
If we make m x 50¢ of changes using
quarters and dimes, then the number of quarters used <
the number of dimes used
10¢ 1¢
25¢ 5¢
11/23/2018

10. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $0.15 under a new system with 3 coins? C3
11¢ C1 1¢ 5¢ C2
11/23/2018

11. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $0.15 under a new system with 3 coins?
Greedy:
1 C1 = 11¢
4 C3= 4¢
So we will have to use 5 coins. C3
11¢ C1 1¢ 5¢ C2
11/23/2018

12. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $0.15 under a new system with 3 coins?
Greedy:
1 C1 = 11¢
4 C3= 4¢
A better way:
3 C2, only 3 coins! C3
11¢ C1 1¢ 5¢ C2
11/23/2018

13. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $0.70 under a new system with 3 coins?
Greedy:
6 C1 = 66¢
4 C3= 4¢
A better way? C3
11¢ C1 1¢ 5¢ C2
11/23/2018

14. Example 1. Coin Change
Making changes for n cents using the minimum number of coins.
How to make change for $0.70 under a new system with 3 coins?
Greedy:
6 C1 = 66¢
4 C3= 4¢
A better way?
5 C1 = 55¢
3 C2= 15¢
While greedy methods do not always work (in fact, nothing always works), they are still useful in some situations. C3
11¢ C1 1¢ 5¢ C2
11/23/2018

15. Example 2. The knapsack problem
You have a knapsack which can only contain certain weight C of goods.
With this weight capacity constraint, you want to maximize the values of the goods you can put in the knapsack.
G1=candy, Total value=$1.0, Total weight=10 pounds
G2=chocolate, Total value=$2.0, Total weight=1 pounds
G3=ice cream, Total value=$2.5, Total weight=4 pounds
If C=4 pounds, what would you do?
11/23/2018

16. Example 2. The knapsack problem
G1=candy, Total value=$1.0, Total weight=10 pounds
G2=chocolate, Total value=$2.0, Total weight=1 pounds
G3=ice cream, Total value=$2.5, Total weight=4 pounds
If C=4 pounds, what would you do?
Greedy 1: by maximum value, 4 pounds of ice cream, profit=$2.5
Greedy 2: by maximum weight, 4 pounds of candy, profit=$0.4
Greedy 3: by maximum unit value, 1 pound of chocolate followed with 3 pounds of ice cream, profit=$3.875
11/23/2018

17. Example 2. The knapsack problem
G1=candy, Total value=$1.0, Total weight=10 pounds
G2=chocolate, Total value=$2.0, Total weight=1 pounds
G3=ice cream, Total value=$2.5, Total weight=4 pounds
In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi,
and you want to maximize the profit out of the goods you can put in the knapsack with capacity C.
How do we formulate this as a mathematical programming problem?
11/23/2018

18. Example 2. The knapsack problem
In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi,
and you want to maximize the profit out of the goods you can put in the knapsack with capacity C.
How do we formulate this as a mathematical programming problem?
Let fi be the fractional of Gi one would put in the knapsack.
11/23/2018

19. Example 2. The knapsack problem
In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi,
and you want to maximize the profit out of the goods you can put in the knapsack with capacity C.
How do we formulate this as a mathematical programming problem?
Let fi be the fractional of Gi one would put in the knapsack.
Maximize ∑i=1..nfivi
Subject to ∑i=1..nfiwi≤C,
0≤fi≤1,i=1..n
Fractional Knapsack Problem
11/23/2018

20. Example 2. The knapsack problem
In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi,
and you want to maximize the profit out of the goods you can put in the knapsack with capacity C.
How do we formulate this as a mathematical programming problem?
Let fi be the fractional of Gi one would put in the knapsack.
Maximize ∑i=1..nfivi Maximize ∑i=1..nfivi
Subject to ∑i=1..nfiwi≤C, Subject to ∑i=1..nfiwi≤C,
0≤fi≤1,i=1..n fi ε {0,1}, i =1..n
Fractional Knapsack Problem Integer Knapsack Problem
11/23/2018

21. Example 2. The knapsack problem
The Fractional Knapsack Problem can be solved optimally using Greedy 3.
The Integer Knapsack Problem is NP-complete, so unlikely to be solvable in polynomial time unless P=NP.
11/23/2018

22. Example 3. Activity-Selection
Given S={1,2,…,n} activities, each activity i has a start time si and a finish time fi, si ≤ fi . Once i is selected, it must be served in interval
[si,fi).
We can’t serve two activities if they are incompatible (i.e., their service intervals intersect).
Question: How to select a maximum-size set of mutually compatible activities?
11/23/2018

23. Example 3. Activity-Selection
Given S={1,2,…,n} activities, each activity i has a start time si and a finish time fi, si ≤ fi . Once i is selected, it must be served in interval
[si,fi).
We can’t serve two activities if they are incompatible (i.e., their service intervals intersect).
Question: How to select a maximum-size set of mutually compatible activities?
11/23/2018

24. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn f9 f3 f6 f2 f5
f10 f1 f7 f8 f4
11/23/2018

25. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn
Greedy-Activity-Selector(s,f)
1. n ← length[s]
2. A ← {1}
3. j ← 1
4. for i = 2 to n
If si ≥ fj
then A ← A U {i}
j ← i
8. Return A f9 f3 f6 f2 f5
f10 f1 f7
11/23/2018 f8 f4

26. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn
Greedy-Activity-Selector(s,f)
1. n ← length[s]
2. A ← {1}
3. j ← 1
4. for i = 2 to n
If si ≥ fj
then A ← A U {i}
j ← i
8. Return A f9 f3 f6 f2 f5
f10 f1 f7
11/23/2018 f8 f4

27. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn
Greedy-Activity-Selector(s,f)
1. n ← length[s]
2. A ← {1}
3. j ← 1
4. for i = 2 to n
If si ≥ fj
then A ← A U {i}
j ← i
8. Return A f9 f3 f6 f2 f5
f10 f1 f7
11/23/2018 f8 f4

28. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn
Greedy-Activity-Selector(s,f)
1. n ← length[s]
2. A ← {1}
3. j ← 1
4. for i = 2 to n
If si ≥ fj
then A ← A U {i}
j ← i
8. Return A f9 f3 f6 f2 f5
f10 f1 f7
11/23/2018 f8 f4

29. Example 3. Activity-Selection
Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn
Greedy-Activity-Selector(s,f)
1. n ← length[s]
2. A ← {1}
3. j ← 1
4. for i = 2 to n
If si ≥ fj
then A ← A U {i}
j ← i
8. Return A f9 f3 f6 f2 f5
f10 f1 f7
11/23/2018 f8 f4

30. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary).
a b c d e f
Frequency
Fixed-length
codeword
11/23/2018

31. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary).
a b c d e f
Frequency
Fixed-length
codeword
Variable-length
11/23/2018

32. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary).
a b c d e f
Frequency
Fixed-length
codeword
Cost: 100,000 x 3 = 300,000 bits
Variable-length
11/23/2018

33. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary).
a b c d e f
Frequency
Fixed-length
codeword
Cost: 100,000 x 3 = 300,000 bits
Variable-length
Cost: 45000x x x x3+9000x4+5000x4
= 224,000 bits
11/23/2018

34. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary).
So we want to design an optimal variable-length codes.
11/23/2018

35. Example 4. Huffman codes
Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember at the bottom-most level, everything is binary).
So we want to design an optimal variable-length codes.
Prefix-free codes: no codeword is a prefix of some other codeword.
Easy to encode and decode, no ambiguity.
Example. c◦d◦f◦a=100◦111◦1100◦0=
11/23/2018

36. Example 4. Huffman codes
Prefix-free codes: no codeword is a prefix of some other codeword.
Easy to encode and decode, no ambiguity.
Example. c◦d◦f◦a=100◦111◦1100◦0=
We usually use a binary tree to represent the prefix-free codes, its leaves are the given characters. The binary codeword for a character is the path from the root to it, where 0 means go to left child and 1 means go to right child.
11/23/2018

37. Example 4. Huffman codes 100 100 1 1 a 45 55 86 14 1 1 58 30 28 14 251 1 a 45 55 86 14 1 1 58 30 28 14 25 1 1 1 1 1 45 14 13 12 16 9 5 c 12 b 13 16 d a b c d e f 1 f 5 9 e
11/23/2018
Fixed-length codeword
Variable-length codeword

38. Example 4. Huffman codes Huffman(C) n ← |C|
Q ← C //Q is a priority queue, keyed on frequency f
for i=1 to n-1
z ← Allocate-node()
left[z]←x←Extract-Min(Q)
right[z]←y←Extract-Min(Q)
f[z] ← f[x]+f[y]
Insert(Q,z)
Return Extract-Min(Q) //now we have the binary tree
11/23/2018

39. Example 4. Huffman codes Huffman(C) n ← |C|
Q ← C //Q is a priority queue, keyed on frequency f
for i=1 to n-1
z ← Allocate-node()
left[z]←x←Extract-Min(Q)
right[z]←y←Extract-Min(Q)
f[z] ← f[x]+f[y]
Insert(Q,z)
Return Extract-Min(Q) //now we have the binary tree
What is the running time?
11/23/2018

40. Example 4. Huffman codes Huffman(C) n ← |C|
Q ← C //Q is a priority queue, keyed on frequency f
for i=1 to n-1
z ← Allocate-node()
left[z]←x←Extract-Min(Q)
right[z]←y←Extract-Min(Q)
f[z] ← f[x]+f[y]
Insert(Q,z)
Return Extract-Min(Q) //now we have the binary tree
What is the running time? O(n log n)!
11/23/2018