1. CSCI 58000, Algorithm Design, Analysis & Implementation Lecture 12 Greedy Algorithms (Chapter 16)

2. Greedy Algorithm (Chapter 16) For many optimization problem, a dynamic problem is just overkill; when? If we can make the optimal choice before finding optimal values of the sub- problems. – We just make the choice that looks best at the moment; that justify the name, “greedy” For many problems, greedy solution does not yield optimal solution, for many it does – Activity-Selection problem (greedy works) – Minimum Spanning Tree (greedy works) – Vertex Cover (greedy does not work) In a dynamic programming solution, we solve the sub-problem first and then use that to find optimal choices for solving larger problems – What if that optimal choice can be achieved without knowing the optimal value of the sub- problems – In other words, the choice that you make greedily is the same as the optimal choice

3. Activity-Selection problem I1234567891011 S(i)130535688212 F(i)4567991011121416 {2, 4, 9, 11} can be one answer

4. One Solution approach using DP

5. A better DP Solution (we have shown it’s weighted version in earlier class)

6. Solution Using Greedy approach

8. How do we know whether it works

9. Elements of greedy strategy Two important criteria need to be satisfied – Greedy choice property (locally optimal choice will ultimately provide a global optimal solution) – Optimal sub-structure property Greedy algorithm design steps – Cast the optimization problem as one in which we can make a choice using a greedy criteria and are left with one sub-problem to solve – Prove that there is always an optimal solution to the original problem that makes the greedy choice (greedy choice is safe)

10. Greedy vs Dynamic Dynamic – Optimal Substructure property – Smaller sub-problems need to be solved first, as their optimal value affects the choice we make when we solve the larger problems – Sub-problems are overlapping, so memoization is important Greedy – Optimal substructure property – Smaller sub-problem don’t need to be solved first, as greedy choice let us solve the problem top-down – After greedy choice, only one sub-problem exist, so sub- problems are not overlapping

11. Huffman Codes Huffman codes are used to compress data by encoding each character to a binary string – The compression is non-lossy. Such codes are prefix-free code (also known as prefix codes), i.e., no codeword is a prefix of some other codeword. – Prefix codes are optimal (no proof) – No delimiter is required between two codewords in the compressed file The idea of compression is to use variable-length codes instead of fixed length code – The most frequent character should have the shortest-length code, and the rarest character should have the longest-length code – Huffman proposed a greedy algorithm that find coding that is optimal among all prefix-free codes. – Optimality is defined over the expected length of each codeword

12. Example 30,000 bit for a 10,000 character file) 22,400 bit for a 10,000 character file)

13. Constructing Huffman code

14. Pseudo-code

15. Example

16. Correctness Proof

17. Correctness Proof (cont.)

19. Knapsack problem

20. Matroids and Greedy Methods

21. Example: Graphical Metroid

22. Matroid (cont.)

23. Greedy algorithm on weighted matroid

24. Scheduling unit-time tasks with penalty

25. Canonical Scheduling

26. Lemma 16.12

27. Lemma 16.12 (cont.)

28. Task with Deadlines shows Matroid property

29. Greedy Algorithm for Unit-length task with penalty