CSCI 58000, Algorithm Design, Analysis & Implementation Lecture 12 Greedy Algorithms (Chapter 16)

Greedy Algorithm (Chapter 16) For many optimization problem, a dynamic problem is just overkill; when? If we can make the optimal choice before finding optimal values of the sub- problems. – We just make the choice that looks best at the moment; that justify the name, “greedy” For many problems, greedy solution does not yield optimal solution, for many it does – Activity-Selection problem (greedy works) – Minimum Spanning Tree (greedy works) – Vertex Cover (greedy does not work) In a dynamic programming solution, we solve the sub-problem first and then use that to find optimal choices for solving larger problems – What if that optimal choice can be achieved without knowing the optimal value of the sub- problems – In other words, the choice that you make greedily is the same as the optimal choice

Activity-Selection problem I S(i) F(i) {2, 4, 9, 11} can be one answer

One Solution approach using DP

A better DP Solution (we have shown it’s weighted version in earlier class)

Solution Using Greedy approach

How do we know whether it works

Elements of greedy strategy Two important criteria need to be satisfied – Greedy choice property (locally optimal choice will ultimately provide a global optimal solution) – Optimal sub-structure property Greedy algorithm design steps – Cast the optimization problem as one in which we can make a choice using a greedy criteria and are left with one sub-problem to solve – Prove that there is always an optimal solution to the original problem that makes the greedy choice (greedy choice is safe)

Greedy vs Dynamic Dynamic – Optimal Substructure property – Smaller sub-problems need to be solved first, as their optimal value affects the choice we make when we solve the larger problems – Sub-problems are overlapping, so memoization is important Greedy – Optimal substructure property – Smaller sub-problem don’t need to be solved first, as greedy choice let us solve the problem top-down – After greedy choice, only one sub-problem exist, so sub- problems are not overlapping

Huffman Codes Huffman codes are used to compress data by encoding each character to a binary string – The compression is non-lossy. Such codes are prefix-free code (also known as prefix codes), i.e., no codeword is a prefix of some other codeword. – Prefix codes are optimal (no proof) – No delimiter is required between two codewords in the compressed file The idea of compression is to use variable-length codes instead of fixed length code – The most frequent character should have the shortest-length code, and the rarest character should have the longest-length code – Huffman proposed a greedy algorithm that find coding that is optimal among all prefix-free codes. – Optimality is defined over the expected length of each codeword

Example 30,000 bit for a 10,000 character file) 22,400 bit for a 10,000 character file)

Constructing Huffman code

Pseudo-code

Example

Correctness Proof

Correctness Proof (cont.)

Knapsack problem

Matroids and Greedy Methods

Example: Graphical Metroid

Matroid (cont.)

Greedy algorithm on weighted matroid

Scheduling unit-time tasks with penalty

Canonical Scheduling

Lemma 16.12

Lemma (cont.)

Task with Deadlines shows Matroid property

Greedy Algorithm for Unit-length task with penalty