Binhai Zhu Computer Science Department, Montana State University Greedy Methods Binhai Zhu Computer Science Department, Montana State University Frequently, presenters must deliver material of a technical nature to an audience unfamiliar with the topic or vocabulary. The material may be complex or heavy with detail. To present technical material effectively, use the following guidelines from Dale Carnegie Training®.   Consider the amount of time available and prepare to organize your material. Narrow your topic. Divide your presentation into clear segments. Follow a logical progression. Maintain your focus throughout. Close the presentation with a summary, repetition of the key steps, or a logical conclusion. Keep your audience in mind at all times. For example, be sure data is clear and information is relevant. Keep the level of detail and vocabulary appropriate for the audience. Use visuals to support key points or steps. Keep alert to the needs of your listeners, and you will have a more receptive audience. 11/23/2018

Idea You want to maximize a global function, which could be hard (1) You always make the best local decision in the hope of getting the overall best solution. (2) The idea is natural (and in some situation it is the best a human can do). 11/23/2018

Idea You want to maximize a global function, which could be hard (1) You always make the best local decision in the hope of getting the overall best solution. (2) The idea is natural (and in some situation it is the best a human can do). (3) Of course, sometimes it might not work. 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. Remember that in this case a penny and a dime is considered the same (possibly in weight). 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. Remember that in this case a penny and a dime is considered the same (possibly in weight). Let’s first look at the US system. 1¢ 25¢ 5¢ 10¢ 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $2.17? 1¢ 25¢ 5¢ 10¢ 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $2.17? 8 quarters = $2.00 1 dime = $0.10 1 nickel = $0.05 2 pennies = $0.02 So we use 12 coins! 1¢ 25¢ 5¢ 10¢ 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $2.17? 8 quarters = $2.00 1 dime = $0.10 1 nickel = $0.05 2 pennies = $0.02 Why this works? 1¢ 25¢ 5¢ 10¢ 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $2.17? 8 quarters = $2.00 1 dime = $0.10 1 nickel = $0.05 2 pennies = $0.02 Why this works? If we make m x 50¢ of changes using quarters and dimes, then the number of quarters used < the number of dimes used 10¢ 1¢ 25¢ 5¢ 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $0.15 under a new system with 3 coins? C3 11¢ C1 1¢ 5¢ C2 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $0.15 under a new system with 3 coins? Greedy: 1 C1 = 11¢ 4 C3= 4¢ So we will have to use 5 coins. C3 11¢ C1 1¢ 5¢ C2 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $0.15 under a new system with 3 coins? Greedy: 1 C1 = 11¢ 4 C3= 4¢ A better way: 3 C2, only 3 coins! C3 11¢ C1 1¢ 5¢ C2 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $0.70 under a new system with 3 coins? Greedy: 6 C1 = 66¢ 4 C3= 4¢ A better way? C3 11¢ C1 1¢ 5¢ C2 11/23/2018

Example 1. Coin Change Making changes for n cents using the minimum number of coins. How to make change for $0.70 under a new system with 3 coins? Greedy: 6 C1 = 66¢ 4 C3= 4¢ A better way? 5 C1 = 55¢ 3 C2= 15¢ While greedy methods do not always work (in fact, nothing always works), they are still useful in some situations. C3 11¢ C1 1¢ 5¢ C2 11/23/2018

Example 2. The knapsack problem You have a knapsack which can only contain certain weight C of goods. With this weight capacity constraint, you want to maximize the values of the goods you can put in the knapsack. G1=candy, Total value=$1.0, Total weight=10 pounds G2=chocolate, Total value=$2.0, Total weight=1 pounds G3=ice cream, Total value=$2.5, Total weight=4 pounds If C=4 pounds, what would you do? 11/23/2018

Example 2. The knapsack problem G1=candy, Total value=$1.0, Total weight=10 pounds G2=chocolate, Total value=$2.0, Total weight=1 pounds G3=ice cream, Total value=$2.5, Total weight=4 pounds If C=4 pounds, what would you do? Greedy 1: by maximum value, 4 pounds of ice cream, profit=$2.5 Greedy 2: by maximum weight, 4 pounds of candy, profit=$0.4 Greedy 3: by maximum unit value, 1 pound of chocolate followed with 3 pounds of ice cream, profit=$3.875 11/23/2018

Example 2. The knapsack problem G1=candy, Total value=$1.0, Total weight=10 pounds G2=chocolate, Total value=$2.0, Total weight=1 pounds G3=ice cream, Total value=$2.5, Total weight=4 pounds In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi, and you want to maximize the profit out of the goods you can put in the knapsack with capacity C. How do we formulate this as a mathematical programming problem? 11/23/2018

Example 2. The knapsack problem In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi, and you want to maximize the profit out of the goods you can put in the knapsack with capacity C. How do we formulate this as a mathematical programming problem? Let fi be the fractional of Gi one would put in the knapsack. 11/23/2018

Example 2. The knapsack problem In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi, and you want to maximize the profit out of the goods you can put in the knapsack with capacity C. How do we formulate this as a mathematical programming problem? Let fi be the fractional of Gi one would put in the knapsack. Maximize ∑i=1..nfivi Subject to ∑i=1..nfiwi≤C, 0≤fi≤1,i=1..n Fractional Knapsack Problem 11/23/2018

Example 2. The knapsack problem In general, you have G1,G2,…,Gn, each Gi with weight wi and value vi, and you want to maximize the profit out of the goods you can put in the knapsack with capacity C. How do we formulate this as a mathematical programming problem? Let fi be the fractional of Gi one would put in the knapsack. Maximize ∑i=1..nfivi Maximize ∑i=1..nfivi Subject to ∑i=1..nfiwi≤C, Subject to ∑i=1..nfiwi≤C, 0≤fi≤1,i=1..n fi ε {0,1}, i =1..n Fractional Knapsack Problem Integer Knapsack Problem 11/23/2018

Example 2. The knapsack problem The Fractional Knapsack Problem can be solved optimally using Greedy 3. The Integer Knapsack Problem is NP-complete, so unlikely to be solvable in polynomial time unless P=NP. 11/23/2018

Example 3. Activity-Selection Given S={1,2,…,n} activities, each activity i has a start time si and a finish time fi, si ≤ fi . Once i is selected, it must be served in interval [si,fi). We can’t serve two activities if they are incompatible (i.e., their service intervals intersect). Question: How to select a maximum-size set of mutually compatible activities? 11/23/2018

Example 3. Activity-Selection Given S={1,2,…,n} activities, each activity i has a start time si and a finish time fi, si ≤ fi . Once i is selected, it must be served in interval [si,fi). We can’t serve two activities if they are incompatible (i.e., their service intervals intersect). Question: How to select a maximum-size set of mutually compatible activities? 11/23/2018

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn f9 f3 f6 f2 f5 f10 f1 f7 f8 f4 11/23/2018

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn Greedy-Activity-Selector(s,f) 1. n ← length[s] 2. A ← {1} 3. j ← 1 4. for i = 2 to n If si ≥ fj then A ← A U {i} j ← i 8. Return A f9 f3 f6 f2 f5 f10 f1 f7 11/23/2018 f8 f4

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn Greedy-Activity-Selector(s,f) 1. n ← length[s] 2. A ← {1} 3. j ← 1 4. for i = 2 to n If si ≥ fj then A ← A U {i} j ← i 8. Return A f9 f3 f6 f2 f5 f10 f1 f7 11/23/2018 f8 f4

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn Greedy-Activity-Selector(s,f) 1. n ← length[s] 2. A ← {1} 3. j ← 1 4. for i = 2 to n If si ≥ fj then A ← A U {i} j ← i 8. Return A f9 f3 f6 f2 f5 f10 f1 f7 11/23/2018 f8 f4

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn Greedy-Activity-Selector(s,f) 1. n ← length[s] 2. A ← {1} 3. j ← 1 4. for i = 2 to n If si ≥ fj then A ← A U {i} j ← i 8. Return A f9 f3 f6 f2 f5 f10 f1 f7 11/23/2018 f8 f4

Example 3. Activity-Selection Idea: Order the activities by increasing finishing time, i.e., f1≤f2≤…≤fn Greedy-Activity-Selector(s,f) 1. n ← length[s] 2. A ← {1} 3. j ← 1 4. for i = 2 to n If si ≥ fj then A ← A U {i} j ← i 8. Return A f9 f3 f6 f2 f5 f10 f1 f7 11/23/2018 f8 f4

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary). a b c d e f Frequency 45000 13000 12000 16000 9000 5000 Fixed-length 000 001 010 011 100 101 codeword 11/23/2018

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary). a b c d e f Frequency 45000 13000 12000 16000 9000 5000 Fixed-length 000 001 010 011 100 101 codeword Variable-length 0 101 100 111 1101 1100 11/23/2018

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary). a b c d e f Frequency 45000 13000 12000 16000 9000 5000 Fixed-length 000 001 010 011 100 101 codeword Cost: 100,000 x 3 = 300,000 bits Variable-length 0 101 100 111 1101 1100 11/23/2018

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary). a b c d e f Frequency 45000 13000 12000 16000 9000 5000 Fixed-length 000 001 010 011 100 101 codeword Cost: 100,000 x 3 = 300,000 bits Variable-length 0 101 100 111 1101 1100 Cost: 45000x1+13000x3+12000x3+16000x3+9000x4+5000x4 = 224,000 bits 11/23/2018

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember that at the bottom-most level, everything is binary). So we want to design an optimal variable-length codes. 11/23/2018

Example 4. Huffman codes Motivation: You have a 100,000-character data file F, with only 6 characters {a,b,c,d,e,f}. You want to have a way to encode them to save space (remember at the bottom-most level, everything is binary). So we want to design an optimal variable-length codes. Prefix-free codes: no codeword is a prefix of some other codeword. Easy to encode and decode, no ambiguity. Example. c◦d◦f◦a=100◦111◦1100◦0=10011111000 11/23/2018

Example 4. Huffman codes Prefix-free codes: no codeword is a prefix of some other codeword. Easy to encode and decode, no ambiguity. Example. c◦d◦f◦a=100◦111◦1100◦0=10011111000 We usually use a binary tree to represent the prefix-free codes, its leaves are the given characters. The binary codeword for a character is the path from the root to it, where 0 means go to left child and 1 means go to right child. 11/23/2018

Example 4. Huffman codes 100 100 1 1 a 45 55 86 14 1 1 58 30 28 14 25 1 1 a 45 55 86 14 1 1 58 30 28 14 25 1 1 1 1 1 45 14 13 12 16 9 5 c 12 b 13 16 d a b c d e f 1 f 5 9 e 11/23/2018 Fixed-length codeword Variable-length codeword

Example 4. Huffman codes Huffman(C) n ← |C| Q ← C //Q is a priority queue, keyed on frequency f for i=1 to n-1 z ← Allocate-node() left[z]←x←Extract-Min(Q) right[z]←y←Extract-Min(Q) f[z] ← f[x]+f[y] Insert(Q,z) Return Extract-Min(Q) //now we have the binary tree 11/23/2018

Example 4. Huffman codes Huffman(C) n ← |C| Q ← C //Q is a priority queue, keyed on frequency f for i=1 to n-1 z ← Allocate-node() left[z]←x←Extract-Min(Q) right[z]←y←Extract-Min(Q) f[z] ← f[x]+f[y] Insert(Q,z) Return Extract-Min(Q) //now we have the binary tree What is the running time? 11/23/2018

Example 4. Huffman codes Huffman(C) n ← |C| Q ← C //Q is a priority queue, keyed on frequency f for i=1 to n-1 z ← Allocate-node() left[z]←x←Extract-Min(Q) right[z]←y←Extract-Min(Q) f[z] ← f[x]+f[y] Insert(Q,z) Return Extract-Min(Q) //now we have the binary tree What is the running time? O(n log n)! 11/23/2018