1. Greed "Greed is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit." Gordon Gecko (Michael Douglas) Some of these lecture slides are adapted from Kleinberg and Tardos.

2. 2 Greedy Algorithms Some possibly familiar examples: n Gale-Shapley stable matching algorithm. n Dijkstra's shortest path algorithm. n Prim and Kruskal MST algorithms. n Huffman codes. n...

3. 3 Selecting Breakpoints Minimizing breakpoints. n Truck driver going from Princeton to Palo Alto along predetermined route. n Refueling stations at certain points along the way. n Truck fuel capacity = C. Greedy algorithm. n Go as far as you can before refueling. Princeton Palo Alto 1 C C 2 C 3 C 4 C 5 C 6 C 7

4. 4 Sort breakpoints by increasing value: 0 = b 0 < b 1 < b 2 <... < b n. S  {0} x = 0 while (x  b n ) let p be largest integer such that b p  x + C if (b p = x) return "no solution" x  b p S  S  {p} return S Greedy Breakpoint Selection Algorithm Selecting Breakpoints: Greedy Algorithm S = breakpoints selected.

5. 5 Selecting Breakpoints Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let 0 = g 0 < g 1 <... < g p = L denote set of breakpoints chosen by greedy and assume it is not optimal. n Let 0 = f 0 < f 1 <... < f q = L denote set of breakpoints in optimal solution with f 0 = g 0, f 1 = g 1,..., f r = g r for largest possible value of r. n Note: q < p. 1234567 123456798 r = 4 Greedy: OPT: g0g0 g1g1 g2g2 f0f0 f1f1 f2f2 gpgp fqfq grgr frfr

6. 6 Selecting Breakpoints Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let 0 = g 0 < g 1 <... < g p = L denote set of breakpoints chosen by greedy and assume it is not optimal. n Let 0 = f 0 < f 1 <... < f q = L denote set of breakpoints in optimal solution with f 0 = g 0, f 1 = g 1,..., f r = g r for largest possible value of r. n Note: q < p. 1234567 123456798 Greedy: OPT: 5 5 5 r = 5 g0g0 g1g1 g2g2 f0f0 f1f1 f2f2 gpgp fqfq r = 4

7. 7 Selecting Breakpoints Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let 0 = g 0 < g 1 <... < g p = L denote set of breakpoints chosen by greedy and assume it is not optimal. n Let 0 = f 0 < f 1 <... < f q = L denote set of breakpoints in optimal solution with f 0 = g 0, f 1 = g 1,..., f r = g r for largest possible value of r. n Note: q < p. n Thus, f 0 = g 0, f 1 = g 1,..., f q = g q 1234 123467 Greedy: OPT: 5 5 r = q = 5 g0g0 g1g1 g2g2 f0f0 f1f1 f2f2 gqgq fqfq gpgp

8. 8 Activity Selection Activity selection problem (CLR 17.1). n Job requests 1, 2, …, n. n Job j starts at s j and finishes at f j. n Two jobs compatible if they don't overlap. n Goal: find maximum subset of mutually compatible jobs. Time 0 A C F B D G E 1234567891011 H

9. 9 Activity Selection: Greedy Algorithm Sort jobs by increasing finish times so that f 1  f 2 ...  f n. S =  for j = 1 to n if (job j compatible with A) S  S  {j} return S Greedy Activity Selection Algorithm S = jobs selected.

10. 10 Activity Selection Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let g 1, g 2,... g p denote set of jobs selected by greedy and assume it is not optimal. n Let f 1, f 2,... f q denote set of jobs selected by optimal solution with f 1 = g 1, f 2 = g 2,..., f r = g r for largest possible value of r. n Note: r < q. 158 158913 152117 r = 3 p = 6 q = 7 Greedy: OPT: f 1 = g 1 21 f 2 = g 2 f 3 = g 3 11

11. 9 Activity Selection Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let g 1, g 2,... g p denote set of jobs selected by greedy and assume it is not optimal. n Let f 1, f 2,... f q denote set of jobs selected by optimal solution with f 1 = g 1, f 2 = g 2,..., f r = g r for largest possible value of r. n Note: r < q. 15811 158913 152117 r = 3 p = 6 q = 7 Greedy: OPT: f 1 = g 1 21 f 2 = g 2 f 3 = g 3 Replace 11 with 9

12. 12 9 Activity Selection Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let g 1, g 2,... g p denote set of jobs selected by greedy and assume it is not optimal. n Let f 1, f 2,... f q denote set of jobs selected by optimal solution with f 1 = g 1, f 2 = g 2,..., f r = g r for largest possible value of r. n Note: r < q. 158 158913 152117 r = 3 p = 6 q = 7 Greedy: OPT: f 1 = g 1 21 f 2 = g 2 f 3 = g 3 Replace 11 with 9

13. 13 9 Activity Selection Theorem: greedy algorithm is optimal. Proof (by contradiction): n Let g 1, g 2,... g p denote set of jobs selected by greedy and assume it is not optimal. n Let f 1, f 2,... f q denote set of jobs selected by optimal solution with f 1 = g 1, f 2 = g 2,..., f r = g r for largest possible value of r. n Note: r < q. 158 158913 152117 r = 3 p = 6 q = 7 Greedy: OPT: f 1 = g 1 21 f 2 = g 2 f 3 = g 3 r = 4 9 9

14. 14 Making Change Given currency denominations: 1, 5, 10, 25, 100, devise a method to pay amount to customer using fewest number of coins. n Ex. 34¢. Greedy algorithm. n At each iteration, add coin of the largest value that does not take us past the amount to be paid. n Ex. $2.89.

15. 15 Coin-Changing: Greedy Algorithm Sort coins denominations by increasing value: c 1 < c 2 <... < c n. S   while (x  0) let p be largest integer such that c p  x if (p = 0) return "no solution found" x  x - c p S  S  {p} return S Greedy Coin-Changing Algorithm S = coins selected.

16. 16 Is Greedy Optimal for Coin-Changing Problem? Yes, for U.S. coinage: {c 1, c 2, c 3, c 4, c 5 } = {1, 5, 10, 25, 100}. Ad hoc proof. n Consider optimal way to change amount c k  x < c k+1. n Greedy takes coin k. n Suppose optimal solution does not take coin k. – it must take enough coins of type c 1, c 2,..., c k-1 to add up to x. 1 ckck 10 25 100 4 Max # taken by optimal solution 2 3 51 no limit k 1 3 4 5 2 4 Max value of coins 1, 2,..., k in any OPT 20 + 4 = 24 75 + 24 = 99 4 + 5 = 9 no limit 2 dimes  no nickels

17. 17 Does Greedy Always Work? US postal denominations: 1, 10, 21, 34, 70, 100, 350, 1225, 1500. n Ex. 140¢. n Greedy: 100, 34, 1, 1, 1, 1, 1, 1. n Optimal: 70, 70.

18. 18 Characteristics of Greedy Algorithms Greedy choice property. n Globally optimal solution can be arrived at by making locally optimal (greedy) choice. n At each step, choose most "promising" candidate, without worrying whether it will prove to be a sound decision in long run. Optimal substructure property. n Optimal solution to the problem contains optimal solutions to sub- problems. – if best way to change 34¢ is {25, 5, 1, 1, 1, 1} then best way to change 29¢ is {25, 1, 1, 1, 1}. Objective function does not explicitly appear in greedy algorithm! Hard, if not impossible, to precisely define "greedy algorithm." n See matroids (CLR 17.4), greedoids for very general frameworks.

19. 19 Minimizing Lateness Minimizing lateness problem. n Single resource can process one job at a time. n n jobs to be processed. – job j requires p j units of processing time. – job j has due date d j. n If we assign job j to start at time s j, it finishes at time f j = s j + p j. n Lateness: j = max { 0, f j - d j }. n Goal: schedule all jobs to minimize maximum lateness L = max j. 01234567891011 12131415 d = 11 d = 8 d = 15 d = 6d = 9 d = 11d = 8d = 2d = 6d = 9 Lateness = 3

20. 20 Minimizing Lateness: Greedy Algorithm Sort jobs by increasing deadline so that d 1  d 2  …  d n. t = 0 for j = 1 to n Assign job j to interval [t, t + p j ] s j  t, f j  t + p j t  t + p j output intervals [s j, f j ] Greedy Activity Selection Algorithm max lateness = 2 01234567891011 12131415 d 5 = 11d 2 = 8d 6 = 15d 1 = 6d 4 = 9 d 3 = 9

21. 21 Minimizing Lateness: No Idle Time Fact 1: there exists an optimal schedule with no idle time. Fact 2: the greedy schedule has no idle time. 0123456 d = 4d = 6 7891011 d = 12 0123456 d = 4d = 6 7891011 d = 12

22. 22 Minimizing Lateness: Inversions An inversion in schedule S is a pair of jobs i and j such that: n i < j n j scheduled before i Fact 3: greedy schedule  no inversions. Fact 4: if a schedule (with no idle time) has an inversion, it has one whose with a pair of inverted jobs scheduled consecutively. d 2 = 4d 5 = 8 inversion

23. 23 Minimizing Lateness: Inversions An inversion in schedule S is a pair of jobs i and j such that: n i < j n j scheduled before i Fact 3: greedy schedule  no inversions. Fact 4: if a schedule (with no idle time) has an inversion, it has one whose with a pair of inverted jobs scheduled consecutively. Fact 5: swapping two adjacent, inverted jobs: n Reduces the number of inversions by one. n Does not increase the maximum lateness. Theorem: greedy schedule is optimal. d 2 = 4d 5 = 8 d 2 = 4d 5 = 8 inversion

24. 24 Minimizing Lateness: Proof of Fact 5 An inversion in schedule S is a pair of jobs i and j such that: n i < j n j scheduled before i Swapping two adjacent, inverted jobs does not increase max lateness. n ' k = k for all k  i, j n ' i  l i n If job j is late: ij ij fifi f' j