1. Dynamic Programming (DP) By Denon

2. Outline Introduction Fibonacci Numbers (Review) Longest Common Subsequence (LCS) More formal view on DP Subset Sum 0-1 Knapsack Problem In-class Exercises Resources

3. What is DP? Dynamic Programming Is NOT a specified algorithm Is a Design technique (like Divide-and- Conquer)

4. What is DP? To give you the feeling of DP, we will solve some problems first. Then, we will extract the common methods/approaches used to solve the problems. And come back to the exact definition of DP later.

5. Fibonacci Numbers Fibonacci Numbers f 0 =0, f 1 =1, f n = f n-1 + f n-2 How to find the n-th Fibonacci Numbers?

6. Fibonacci Numbers Fibonacci Numbers f 0 =0, f 1 =1, f n = f n-1 + f n-2 Sloooooooooooooooow way: int fib(int n){ if (n == 0){ return 0; } if( n == 1){ return 1; } return fib(n-1) + fib(n-2); }

7. Fibonacci Numbers Very Sloooooooooooooooow Note how many time F2 is called, imagine you need to find F100

8. Fibonacci Numbers Fibonacci Numbers f 0 =0, f 1 =1, f n = f n-1 + f n-2 Improve by memorize the intermediate results (Memorization) Sloooow Way: int table[100000]; memset(table, 0xff, sizeof(table)); //negative number indicates undefined table[0] = 0; table[1] = 1; int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; }

9. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 01?

10. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 01??

11. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 01???

12. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 01????????

13. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 011???????

14. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 0112??????

15. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 01123?????

16. Fibonacci Numbers How is the table filled? int table[100000]; … int fib(int n){ if(table[n] < 0){ table[n] = fib(n-1) + fib(n-2); } return table[n]; } f0f1f2f3f4f5f6f7f8f9 0112358132134

17. Fibonacci Numbers Fibonacci Numbers f 0 =0, f 1 =1, f n = f n-1 + f n-2 Further improve by observing how the table is filled from f0, f1, f2 … to … fn Slow way int table[100000]; table[0] = 0; table[1] = 1; for (int i = 2; i < 100000; i++){ table[i] = table[i-1] + table[i-2]; } int fib(int n){ return table[n]; }

18. Fibonacci Numbers What is the difference between filling it recursively (top-down) or iteratively (bottom- up)?

19. Fibonacci Numbers What is the difference filling it recursively (top-down) or iteratively (bottom-up)? Bottom-upTop-down Order of evaluation ImportantNot important Function callsNopeYup Space requirement LowerHigher (Why??) RedundantPossibleNope

20. Fibonacci Numbers Why the recursive one consumes more space then iterative one?

21. Fibonacci Numbers Note that when f2, f3, …, f8 is being called, f9 is not yet returned. Thus, it requires memory space to store the local variable, returning address of the f2, f3, …, f8 (in a place called “Call Stack”) f0f1f2f3f4f5f6f7f8f9 01????????

22. Fibonacci Numbers In some cases, the stack space may be insufficient to fill the table recursively (i.e. Explode Stack ‘ 爆 Stack’)

23. Fibonacci Numbers In some cases, the stack space may be insufficient to fill the table recursively (i.e. Explode Stack ‘ 爆 Stack’) But in the contest problems, filling the table recursively is USUALLY okay.

24. Fibonacci Numbers What have we done to find the Fib. No.? 1.Solve it by recursion 2.Find out that it is tooooooooo slow 3.Do some memorizations 4.Figure out the table 5.Find out the order of filling the table 6.Find the solution by filling the table instead of doing recursion

25. Longest Common Subsequence (LCS) What is longest common subsequence? Given two strings S1 and S2, find their longest common subsequence (NOT substring) Example – S1: aabcaabc – S2: caadbgaccpoo a b c a is a sub-sequence of S1: aabcaabc a c b c is a sub-sequence of S1: aabcaabc

26. Longest Common Subsequence (LCS) Example – S1: aabcaabc – S2: caadbgaccpoo Note that, a a a is a sub-sequence of S1: aabcaabc a a a is also a sub-sequence of S2: caadbgaccpoo So, aaa is a common subsequence of S1, S2

27. Longest Common Subsequence (LCS) So a longest common subsequence is one of the longest common subsequences among all the common subsequences of S1, S2 In our example, – S1: aabcaabc – S2: caadbgaccpoo aabac is one of the lcs of S1, S2

28. Longest Common Subsequence (LCS) So, how to find the length of the longest common subsequence of S1 and S2? Of course, you can generate all subsequences of S1, S2. Then, compare them one by one. But how long would it takes? (i.e. How many different subsequence exists for a string of length n)

29. Longest Common Subsequence (LCS) In order to use DP, we first need to able to solve it by recursion. (Divide-and-Conqeur) Here we focus on the last character.

30. Longest Common Subsequence (LCS) Here we focus on the last character. Case I: Same Character Case II: Different Characters

31. Longest Common Subsequence (LCS) Case II: Different Characters Let the length of LCS of S1 (of length n1) and S2 (of length n2) be llcs(n1, n2) Claim: llcs(n1, n2) = max( llcs(n1-1, n2), llcs(n1,n2 -1) )

32. Longest Common Subsequence (LCS) Case 2a) A is in the LCS Case 2b) B is in the LCS Case 2c) A, B are both not in the LCS llcs(n1, n2) = max( llcs(n1-1, n2), llcs(n1,n2 -1) )

33. Longest Common Subsequence (LCS) Case I: Same Character Let the length of LCS of S1 (of length n1) and S2 (of length n2) be llcs(n1, n2) Claim: llcs(n1, n2) = llcs(n1-1, n2-1) + 1

34. Longest Common Subsequence (LCS) Case 1a) Both A is in the LCS Case 1b) One of the A is in the LCS Case 1c) Both A is not in the LCS llcs(n1, n2) = llcs(n1-1, n2-1) + 1

35. Longest Common Subsequence (LCS) Case 1a) Both A is in the LCS  Make sense Case 1b) One of the A is in the LCS  The same Case 1c) Both A is not in the LCS  Contradiction llcs(n1, n2) = llcs(n1-1, n2-1) + 1

36. Longest Common Subsequence (LCS) string s1, s2; int llcs(n1, n2){ //base case if(n1 == 0 || n2 == 0) return 0; if(s1[n1-1] == s2[n2-1]) { return llcs(n1-1, n2-1) + 1; }else{ return max(llcs(n1-1, n2), llcs(n1, n2-1)); }

37. Longest Common Subsequence (LCS) Of course, this is too sloooooooooooow. Improve by memorization. string s1, s2; int table[N][N]; memset(table, 0xff, sizeof(table)); int llcs(n1, n2){ //base case if(n1 == 0 || n2 == 0) return 0; if(table[n1][n2] < 0){ if(s1[n1-1] == s2[n2-1]) { table[n1][n2] = llcs(n1-1, n2-1) + 1; }else{ table[n1][n2] = max(llcs(n1-1, n2), llcs(n1, n2-1)); } return table[n1][n2]; }

38. Longest Common Subsequence (LCS) How the table is filled? if(s1[n1-1] == s2[n2-1]) { return llcs(n1-1, n2-1) + 1; }else{ return max(llcs(n1-1, n2), llcs(n1, n2-1)); } 0123 00000 10 20 30 40

39. Longest Common Subsequence (LCS) If s1[2] == s2[3] if(s1[n1-1] == s2[n2-1]) { return llcs(n1-1, n2-1) + 1; }else{ return max(llcs(n1-1, n2), llcs(n1, n2-1)); } s2 \ s10123 00000 10 20k 30K + 1 40

40. Longest Common Subsequence (LCS) If s1[2] != s2[3] if(s1[n1-1] == s2[n2-1]) { return llcs(n1-1, n2-1) + 1; }else{ return max(llcs(n1-1, n2), llcs(n1, n2-1)); } s2 \ s10123 00000 10 20h 30kmax(h, k) 40

41. Longest Common Subsequence (LCS) If s1[2] == s2[3] s2 \ s10123 00000 10 20k 30K + 1 40 If s1[2] != s2[3] s2 \ s10123 00000 10 20h 30kmax(h, k) 40 //s1 of length n1, s2 of length n2 for(int c1 = 0; c1 < n1; c1 ++){ for(int c2 = 0; c2 < n2; c2 ++){ if(c1 == 0 || c2 == ){ table[c1][c2] = 0; }else if(s1[c1-1] == s2[c2-1]){ table[c1][c2] = table[c1-1][c2-1] + 1; }else{ table[c1][c2] = max(table[c1-1][c2], table[c1][c2-1]); }

42. Longest Common Subsequence (LCS) What have we done to find the LCS? 1.Solve it by recursion 2.Find out that it is tooooooooo slow 3.Do some memorizations 4.Figure out the table 5.Find out the order of filling the table 6.Find the solution by filling the table instead of doing recursion

43. What is DP? (2) It is all about filling the “right” table. In fact, the word “programming” here is an old word that means any tabular method for accomplishing something – Dynamic Programming, Linear Programming …

44. More formal view on DP Recall that, when we find fib(10), we make use the result of fib(9) and fib(8). And when we find the lcs(12, 11), we make use the result of lcs(11, 10), lcs(12, 10) and lcs(11, 11).

45. More formal view on DP To be more specific, S1 = “a b a f s” S2 = “a a b s t” The LCS of S1, S2 contains the LCS of the sub- string of S1, “a b a” and the sub-string S2, “a a b”.

46. More formal view on DP In other words, an optimal solution to a problem instance contains (one of) the optimal solutions to its sub-problems. This is called Optimal substructure

47. More formal view on DP Also recall that, without memorization, when we find fib(10), we evaluate the value of fib(1), fib(2), fib(3) … fib(9) over and over again. And when we find the lcs(12, 11), we evaluate the value of lcs(11, 10), lcs(12, 10), lcs(11, 11)… over and over again.

48. More formal view on DP In other words, The recursive solution contains a “small” number of distinct sub-problems repeated many times. This is called Overlapping subproblems.

49. When to use DP? So, when your problem exhibits these 2 properties, i.e. Optimal substructure Overlapping subproblems You may consider solving it using DP.

50. Break Let’s have 10 mins break Question: What if I want to print the LCS instead of just its length?

51. Subset Sum Given a set S of n numbers ai sum up to M, and any K ≤ M, what are the possible sums of its subset? i.e. S = { 1, 3, 1, 4} Some possible Subset Sums are 0, (1), (3), (4), (1 + 1 = 2), (1 + 4 = 5), (3 + 4 = 7), (1 + 3 + 4 = 8), (1 + 3 + 4 + 1 = 9).

52. Subset Sum First, how to solve it recursively?

53. Subset Sum First, how to solve it recursively? Hints: Let S = { 1, 3, 1, 4} If I know that {1, 3, 1} can form {0, 1, 3, 4, 5}, i.e. bool[ length = 3 ][] = {T, T, F, T, T, T, F,…} What can {1, 3, 1, 4} form? i.e. bool[ length = 4][] = {????}

54. Subset Sum First, how to solve it recursively? Let S = { 1, 3, 1, 4} 012345678 {}TFFFFFFFF {1} {1,3} {1,3,1 } {1,3,1,4}

55. Subset Sum First, how to solve it recursively? Let S = { 1, 3, 1, 4} 012345678 {}TFFFFFFFF {1}TTFFFFFFF {1,3} {1,3,1 } {1,3,1,4}

56. Subset Sum First, how to solve it recursively? Let S = { 1, 3, 1, 4} 012345678 {}TFFFFFFFF {1}TTFFFFFFF {1,3}TTFTTFFFF {1,3,1 } {1,3,1,4}

57. Subset Sum First, how to solve it recursively? Let S = { 1, 3, 1, 4} 012345678 {}TFFFFFFFF {1}TTFFFFFFF {1,3}TTFTTFFFF {1,3,1 } TTTTTTFFF {1,3,1,4}

58. Subset Sum First, how to solve it recursively? Let S = { 1, 3, 1, 4} 012345678 {}TFFFFFFFF {1}TTFFFFFFF {1,3}TTFTTFFFF {1,3,1 } TTTTTTFFF {1,3,1,4} TTTTTTTTT

59. Subset Sum bool table[M+10]; Memset(table, 0, sizeof(table); m[0]=true; for(i=0; i<n; i++) for(j=M; j>=s[i]; j--)// Must be decreasing here m[j] |= m[j-a[i]];

60. 0-1 Knapsack There are n items in a shop. The i-th item has weight w i and value v i. A thief has a knapsack which can carry at most a weight of W. What should she steal to maximize the total value of the stolen items? Assumption: all numbers in this problem are positive integers

61. 0-1 Knapsack There are n items in a shop. The i-th item has weight w i and value v i. A thief has a knapsack which can carry at most a weight of TW. What should she steal to maximize the total value of the stolen items? W = { 10, 5, 2, 7} V = { 4, 2, 1 8} TW = 17

62. 0-1 Knapsack Hints: – Take it or leave it.

63. 0-1 Knapsack 0 if i = 0 or w = 0 c[i,w] = c[i-1, w] if w i ≥ 0 max [v i + c[i-1, w-w i ], c[i-1, w]} if i>0 and w ≥ w i

64. Practices In-class Exercises – 900 Brick Wall Patterns – 10066 The Twin Towers – 562 Dividing Coins – 357 Let Me Count The Ways (Hints: Top-Down filling is good enough)

65. Resources http://i.cs.hku.hk/~provinci/utilities.html http://uvatoolkit.com

66. Suggested Problems (from Louis Siu) Easy: 116, 147, 231, 847, 926, 988, 10036, 10081, 10192, 10198, 10304, 10337, 10359, 10400, 10404, 10453, 10532, 10617, 10651, 10702, 10759, 10891, 10912, 10970, 11151 Medium: 562, 607, 711, 714, 882, 10003, 10130, 10944, 10890, 10981, 11002, 11003, 11137 Hard: 757, 10020, 10564, 10663, 10859, 10911, 10934, 10940, 11045, 11081