1. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems

2. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals

3. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

4. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

5. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

6. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

7. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

8. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals (Greedy) algo:

9. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals total time selected

10. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection 2 a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals Greedy algo ? total time selected

11. Greedy Algorithms Input: Output: Objective: - make decisions “greedily”, previous decisions are never reconsidered Optimization problems – Activity selection 2 a set of time-intervals a subset of non-overlapping intervals maximize # of selected intervals An algo ? total time selected

12. BackTracking - “brute force”: try all possibilities - running time?

13. BackTracking - “brute force”: try all possibilities - running time? O(n!) Anything better for Activity selection 2?

14. BackTracking - “brute force”: try all possibilities - running time? O(n!) Anything better for Activity selection 2?

15. BackTracking - “brute force”: try all possibilities - running time? O(n!) Anything better for Activity selection 2? - dynamic programming

16. Dynamic Programming - idea: remember values for smaller instances, use it for computing the value for a larger instance

17. Dynamic Programming – Activity selection 2 - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length?

18. - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length? Which value are we ultimately looking for? Dynamic Programming – Activity selection 2

19. - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length? Which value are we ultimately looking for? S[n] How to compute S[k+1], having computed S[1], …, S[k] ? Dynamic Programming – Activity selection 2

20. - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? Which value are we ultimately looking for? S[n] How to compute S[k+1], having computed S[1], …, S[k] ? S[k,j] Dynamic Programming – Activity selection 2

21. - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? Which value are we ultimately looking for? S[n] How to compute S[k+1], having computed S[1], …, S[k] ? S[k,j] Dynamic Programming – Activity selection 2

22. Which value are we ultimately looking for? max j S[n,j] - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? How to compute S[k+1], having computed S[1], …, S[k] ? S[k,j] Dynamic Programming – Activity selection 2

23. Which value are we ultimately looking for? max j S[n,j] - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? How to compute S[k+1], having computed S[1], …, S[k] ? S[k,j] ? Dynamic Programming – Activity selection 2

24. Which value are we ultimately looking for? max j S[n,j] - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? How to compute S[k,j], having computed S[k’,j’] for every (k’,j’) such that k’<k ? S[k,j] Dynamic Programming – Activity selection 2

25. Which value are we ultimately looking for? max j S[n,j] - idea: remember values for smaller instances, use it for computing the value for a larger instance Let S[k] =considering only the first k intervals, what is the optimal length ending with interval j ? How to compute S[k,j], having computed S[k’,j’] for every (k’,j’) such that k’<k ? S[k,j] S[k,j] = max j’: f[j’] · b[j] [ S[k-1,j’] + (f[j]-b[j]) ] Dynamic Programming – Activity selection 2

26. S[k,j] = max j’: f[j’] · b[j] [ S[k-1,j’] + (f[j]-b[j]) ] Dynamic Programming – Activity selection 2 Can output the optimal time, how to find the optimal selection?

27. Approaches to Problem Solving greedy algorithms dynamic programming backtracking divide-and-conquer

28. Approaches to Problem Solving greedy algorithms dynamic programming backtracking divide-and-conquer our focus this week

29. Problem: Huffman Coding binary character code = assignment of binary strings to characters e.g. ASCII code A = 01000001 B = 01000010 C = 01000011 … fixed-length code How to decode: ? 01000001010000100100001101000001

30. Problem: Huffman Coding binary character code = assignment of binary strings to characters e.g. code A = 0 B = 10 C = 11 … variable-length code How to decode: ? 0101001111

31. Problem: Huffman Coding binary character code = assignment of binary strings to characters e.g. code A = 0 B = 10 C = 11 … variable-length code How to decode: ? 0101001111 DEF: A code is prefix-free if no codeword is a prefix of another codeword.

32. Problem: Huffman Coding binary character code = assignment of binary strings to characters e.g. another code A = 1 B = 10 C = 11 … variable-length code How to decode: ? 10101111 DEF: A code is prefix-free if no codeword is a prefix of another codeword.

33. Problem: Huffman Coding binary character code = assignment of binary strings to characters e.g. another code A = 1 B = 10 C = 11 … variable-length code How to decode: ? 10101111 DEF: A code is prefix-free if no codeword is a prefix of another codeword. not prefix-free

34. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character E11.1607% A8.4966% R7.5809% I7.5448% O7.1635% T6.9509% N6.6544% S5.7351% L5.4893% C4.5388% U3.6308% D3.3844% P3.1671% M3.0129% H3.0034% G2.4705% B2.0720% F1.8121% Y1.7779% W1.2899% K1.1016% V1.0074% X0.2902% Z0.2722% J0.1965% Q0.1962%

35. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character A60% B20% C10% D10%

36. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character A60% B20% C10% D10% 00 01 10 11

37. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character A60% B20% C10% D10% 00 01 10 11 Optimal ?

38. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character A60% B20% C10% D10% 00 01 10 11 Optimal ? 2-bits per character Can do better ?

39. Problem: Huffman Coding - optimal prefix-free code Input: Output: Objective: an alphabet with frequencies prefix code expected number of bits per character A60% B20% C10% D10% 00 01 10 11 Optimal ? 2-bits per character Can do better ? YES, 1.6-bits per character

40. Problem: Huffman Coding - optimal prefix-free code Huffman ( [a 1,f 1 ],[a 2,f 2 ],…,[a n,f n ]) if n=1 then code[a 1 ]  “” else let f i,f j be the 2 smallest f’s Huffman ( [a i,f i +f j ],[a 1,f 1 ],…,[a n,f n ] ) code[a j ]  code[a i ] + “0” code[a i ]  code[a i ] + “1”

41. Problem: Huffman Coding Let x,y be the symbols with frequencies f x < f y. Then in an optimal prefix code length(C x )  length(C y ). Lemma 1:

42. Problem: Huffman Coding Let x,y be the symbols with frequencies f x < f y. Then in an optimal prefix code length(C x )  length(C y ). Lemma 1: Let C = w0 be a longest codeword in an optimal code. Then w1 is also a codeword. Lemma 2:

43. Problem: Huffman Coding Let x,y be the symbols with frequencies f x < f y. Then in an optimal prefix code length(C x )  length(C y ). Lemma 1: Let C = w0 be a longest codeword in an optimal code. Then w1 is also a codeword. Lemma 2: Let x,y be the symbols with the smallest frequencies. Then there exists an optimal prefix code such that the codewords for x and y differ only in the last bit. Lemma 3:

44. Problem: Huffman Coding Let x,y be the symbols with frequencies f x < f y. Then in an optimal prefix code length(C x )  length(C y ). Lemma 1: Let C = w0 be a longest codeword in an optimal code. Then w1 is also a codeword. Lemma 2: Let x,y be the symbols with the smallest frequencies. Then there exists an optimal prefix code such that the codewords for x and y differ only in the last bit. Lemma 3: The prefix code output by the Huffman algorithm is optimal. Theorem: