Introduction to Algorithms LECTURE 15 Greedy Algorithms II • Activity-Selection Problem • Knapsack Problem • Huffman Codes • Task Scheduling

Greedy Algorithm: Vertex Cover A vertex cover of an undirected graph G=(V, E) is a subset V'  V such that if (u, v)  E, then u  V' or v  V', or both. The set of vertices covers all the edges. The size of a vertex cover is the number of vertices in the cover. The vertex-cover problem is to find a vertex cover of minimum size in a graph. Greedy heuristic: cover as many edges as possible (vertex with the maximum degree) at each stage and then delete the covered edges. The greedy heuristic cannot always find an optimal solution! The vertex-cover problem is NP-complete.

A Greedy Algorithm A greedy algorithm always makes the choice that looks best at the moment. An Activity-Selection Problem: Given a set S = {1, 2, …, n} of n proposed activities, with a start time si and a finish time fi for each activity i, select a maximum-size set of mutually compatible activities. If selected, activity i takes place during the half-open time interval [si, fi). Activities i and j are compatible if [si, fi) and [sj, fj) do not overlap (i.e., si  fj or sj  fi).

Activity Selection

The Activity-Selection Algorithm Greedy-Activity-Selector(s,f) /* Assume f1  f2  …  fn. */ 1. n  length[s]; 2. A  {1}; 3. j  1; 4. for I  2 to n 5. if si  fj 6. A  A  {i}; 7. j  i; 8. return A. Time complexity excluding sorting: O(n) Theorem: Algorithm Greedy-Activity-Selector produces solutions of maximum size for the activity-selection problem. (Greedy-choice property) Suppose A  S is an optimal solution. Show that if the first activity in A activity k  1, then B = A - {k}  {1} is an optimal solution. (Optimal substructure) Show that if A is an optimal solution to S, then A' = A - {1} is an optimal solution to S' = {i  S: si  f1}. Prove by induction on the number of choices made.

Elements of the Greedy Strategy When to apply greedy algorithms? Greedy-choice property: A global optimal solution can be arrived at by making a locally optimal (greedy) choice. Dynamic programming needs to check the solutions to subproblems. Optimal substructure: An optimal solution to the problem contains within its optimal solutions to subproblems. E.g., if A is an optimal solution to S, then A' = A - {1} is an optimal solution to S' = {i  S: si  f1}. Greedy algorithms (heuristics) do not always produce optimal solutions. Greedy algorithms v.s. dynamic programming (DP) Common: optimal substructure Difference: greedy-choice property DP can be used if greedy solutions are not optimal.

Knapsack Problem Knapsack Problem: Given n items, with ith item worth vi dollars and weighing wi pounds, a thief wants to take as valuable a load as possible, but can carry at most W pounds in his knapsack. The 0-1 knapsack problem: Each item is either taken or not taken (0-1 decision). The fractional knapsack problem: Allow to take fraction of items. Exp: = (60, 100, 120), = (10, 20, 30), W = 50 Greedy solution by taking items in order of greatest value per pound is optimal for the fractional version, but not for the 0-1 version. The 0-1 knapsack problem is NP-complete, but can be solved in O(nW) time by DP. (A polynomial-time DP??)

Coding Used for data compression, instruction-set encoding, etc. Binary character code: character is represented by a unique binary string Fixed-length code (block code): a: 000, b: 001, …, f: 101  ace  000 010 100. Variable-length code: frequent characters  short codeword; infrequent characters  long codeword

Binary Tree v.s. Prefix Code Prefix code: No code is a prefix of some other code.

Optimal Prefix Code Design Coding Cost of T: B(T)= c  C f(c)dT(c) c : each character in the alphabet C f(c): frequency of c dT(c): depth of c's leaf (length of the codeword of c) Code design: Given f(c1), f(c2), …, f(cn), construct a binary tree with n leaves such that B(T) is minimized. Idea: more frequently used characters use shorter depth.

Huffman's Procedure Pair two nodes with the least costs at each step.

Huffman's Algorithm Time complexity: O(n lgn). Huffman(C) 1. n  |C|; 2. Q  C; 3. for I  1 to n-1 4. z  Allocate-Node(); 5. x  left[z]  Extract-Min(Q); 6. y  right[z]  Extract-Min(Q); 7. f[z]  f[x]+f[y]; 8. Insert(Q, z); 9. return Extract-Min(Q) Time complexity: O(n lgn). Extract-Min(Q) needs O(lg n) by a heap operation. Requires initially O(n lgn) time to build a binary heap.

Huffman Algorithm: Greedy Choice Greedy choice: Two characters x and y with the lowest frequencies must have the same length and differ only in the last bit.

Huffman's Algorithm: Optimal Substructure Optimal substructure: Let T be a full binary tree for an optimal prefix code over C. Let z be the parent of two leaf characters x and y. If f [z]=f [x]+f [y], tree T' = T - {x, y} represents an optimal prefix code for C'= C - {x, y}  {z}.

Task Scheduling The task scheduling problem: Schedule unit-time tasks with deadlines and penalties s.t. the total penalty for missed deadlines is minimized. S = {1, 2, …, n} of n unit-time tasks. Deadlines d1, d2, …, dn for tasks, 1  di  n. Penalties w1, w2, …, wn : wi is incurred if task i misses deadline. Set A of tasks is independent if  a schedule with no late tasks. Nt(A): number of tasks in A with deadlines t or earlier, t = 1, 2, …, n. Three equivalent statements for any set of tasks A A is independent. Nt(A)  t, t = 1, 2, …, n. If the task in A are scheduled in order of nondecreasing deadlines, then no task is late.

Greedy Algorithm: Task Scheduling The optimal greedy scheduling algorithm: Sort penalties in non-increasing order. Find tasks of independent sets: no late task in the sets. Schedule tasks in a maximum independent set in order of nondecreasing deadlines. Schedule other tasks (missing deadlines) at the end arbitrarily.