1. Upcoming Contests TopCoder Marathon Match 49 (currently running – ends Feb. 18 th )

2. TopCoder Open 2009 Algorithms qualifying rounds: –Tuesday, February 24 th @ 7:00 AM –Saturday, February 28 th @ 12:00 PM –Wednesday, March 4 th @ 9:00 PM

3. TopCoder Open 2009 Marathon Matches: –Wednesday, February 25 th @ 12:00 PM –Wednesday, March 11 th @ 12:00 PM –Wednesday, March 25 th @ 12:00 PM

4. TopCoder Open 2009 Other contests as well –Visit http://www.topcoder.com/tco09 for more informationhttp://www.topcoder.com/tco09

5. TopCoder Events Calendar (Feb.)

6. Events Calendar (March)

7. Dynamic Programming Programming Puzzles and Competitions CIS 4900 / 5920 Spring 2009

8. Lecture Outline Order Notation Solution to Text Segmentation Dynamic Programming Text Segmentation w/ Memoization Practice

9. Algorithmic Complexity and Order Notation We would like to be able to describe how long a program takes to run

10. Algorithmic Complexity and Order Notation We would like to be able to describe how long a program takes to run We should express the runtime in terms of the input size

11. Algorithmic Complexity and Order Notation Absolute time measurements are not helpful, as computers get faster all the time Also, runtime is clearly dependent on the input (and input size)

12. Algorithmic Complexity and Order Notation We define O(*) as follows

13. Text Segmentation Some languages are written without spaces between the words

14. Text Segmentation Difficult for search engines to decipher the meaning of a search

15. Text Segmentation Difficult for search engines to decipher the meaning of a search Difficult to return relevant results

16. Text Segmentation Given a string without spaces, what is the best segmentation of the string?

17. Text Segmentation Examples: “upordown”  “up or down”

18. Text Segmentation Examples: “upordown”  “up or down” “upsidedown”  “upside down”

19. Text Segmentation Examples: “upordown”  “up or down” “upsidedown”  “upside down” “haveaniceday”  “have a nice day”

20. Text Segmentation Ambiguities are possible, e.g.: “theyouthevent”  ?

21. Text Segmentation At least three ways to segment “theyouthevent” into valid words: 1) “they out he vent” 2) “the you the vent” 3) “the youth event”

22. Text Segmentation At least three ways to segment “theyouthevent” into valid words: 1) “they out he vent” 2) “the you the vent” 3) “the youth event” (most likely)

23. Text Segmentation: Some ambiguities whorepresents.com therapistfinder.com speedofart.com expertsexchange.com penisland.com These examples are borrowed from Peter Norvig.

24. Text Segmentation How can this be done?

25. Text Segmentation: Solution Recursion: P max (y) = max i P 0 (y[0:i]) x P max (y[i:n]); P max (“”) = 1; where n = length(y)

26. Text Segmentation: Solution Recursion: P max (y) = max i P 0 (y[0:i]) x P max (y[i:n]); P max (“”) = 1; where n = length(y) “base case”

27. Text Segmentation: Solution P max (“theyouthevent”) = max( P 0 (“t”) x P max (“heyouthevent”), P 0 (“th”) x P max (“eyouthevent”), … P 0 (“theyouthevent”) x P max (“”) );

28. Text Segmentation: Solution double get_Pmax(const string &s) { if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } return max; }

29. Text Segmentation: Solution P 0 (“”) x P max (“”) double get_Pmax(const string &s) { if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } return max; }

30. Text Segmentation: Solution max(…) P 0 (“”) x P max (“”) double get_Pmax(const string &s) { if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } return max; }

31. Text Segmentation: Solution max(…) P 0 (“”) x P max (“”) double get_Pmax(const string &s) { if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } return max; } base case

32. Text Segmentation: Solution max(…) P 0 (“”) x P max (“”) double get_Pmax(const string &s) { if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } return max; } base case P max (y) = max i P 0 (y[0:i]) x P max (y[i:n])

33. Text Segmentation: Solution Recursion: P max (y) = max i P 0 (y[0:i]) x P max (y[i:n]); P max (“”) = 1; where n = length(y) base case

34. Text Segmentation: Solution Note that this algorithm is (extremely) inefficient

35. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”)

36. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”) Unnecessary Call

37. Fibonacci Numbers F 0 = 0 F 1 = 1 F n = F n-1 + F n-2 for n > 1

38. Fibonacci Numbers F 0 = 0 F 1 = 1 F n = F n-1 + F n-2 for n > 1 0, 1, 1, 2, 3, 5, 8, 13, …

39. Fibonacci Numbers How would we write code for this?

40. Fibonacci Numbers How would we write code for this? F 0 = 0 F 1 = 1 F n = F n-1 + F n-2 for n > 1

41. Fibonacci Numbers int fibonacci(int n) { if(n == 0 || n == 1) return n; else return fibonacci(n-1) + fibonacci(n-2); }

42. Fibonacci Numbers: Computing F(5) F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1)

43. Dynamic Programming Same algorithm as before, but compute each value only once

44. Dynamic Programming Same algorithm as before, but compute each value only once This significantly reduces the runtime

45. Dynamic Programming Can be done in one of two ways –iteration –recursion w/ memoization

46. Dynamic Programming Can be done in one of two ways –iteration (bottom-up) –recursion w/ memoization (top-down)

47. Dynamic Programming Can be done in one of two ways –iteration (bottom-up) –recursion w/ memoization (top-down) We will focus on the second of these (for now)

48. Dynamic Programming: Recursion w/ Memoization “Memoize”* previously computed function evaluations *the word “memoize” stems from the word “memo”; it is not a misspelling of the word “memorize”

49. Dynamic Programming: Recursion w/ Memoization “Memoize”* previously computed function evaluations There is no reason to run the same computation twice *the word “memoize” stems from the word “memo”; it is not a misspelling of the word “memorize”

50. Fibonacci Numbers: F(5) revisited F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1)

51. Fibonacci Numbers: F(5) revisited F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1) = re-computations

52. Fibonacci Numbers: F(5) revisited F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1)

53. Dynamic Programming: Recursion w/ Memoization Implementation –keep a map from argument values to return values: lookup[arguments] = return value;

54. Dynamic Programming: Recursion w/ Memoization Why does a lookup table help?

55. Dynamic Programming: Recursion w/ Memoization Why does a lookup table help? Avoids re-computations by saving previously computed values

56. Fibonacci Numbers w/ Dynamic Programming int fibonacci(int n) { static map memo; if(memo.find(n) != memo.end()) return memo[n]; if(n == 0 || n == 1) memo[n] = n; else memo[n] = fibonacci(n-1) + fibonacci(n-2); return memo[n]; }

57. Fibonacci Numbers w/ Dynamic Programming int fibonacci(int n) { static map memo; if(memo.find(n) != memo.end()) return memo[n]; if(n == 0 || n == 1) memo[n] = n; else memo[n] = fibonacci(n-1) + fibonacci(n-2); return memo[n]; } Memoization

58. Fibonacci Numbers: Computing F(5) F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1)

59. Fibonacci Numbers: Computing F(5) F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1) Table Lookups

60. Fibonacci Numbers: Computing F(5) F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) F(2) F(0)F(1) F(0)F(1) Table Lookups

61. Fibonacci Numbers: F(5) w/ memoization F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1) Table Lookups

62. Fibonacci Numbers: F(5) w/ memoization F(5) F(3)F(4) F(2)F(3) F(1)F(2) F(0)F(1)

63. Text Segmentation w/ Dynamic Programming double get_Pmax(const string &s) { static map probabilities; if(probabilities.find(s) != probabilities.end()) return probabilities[s]; if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } probabilities[s] = max; return max; }

64. Text Segmentation w/ Dynamic Programming double get_Pmax(const string &s) { static map probabilities; if(probabilities.find(s) != probabilities.end()) return probabilities[s]; if(s.length() == 0) return get_P0(s); double max = 0.0; for(int i = 1; i <= s.length(); ++i) { double temp = get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i)); if(temp > max) max = temp; } probabilities[s] = max; return max; } Memoization

65. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”)

66. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”) Table Lookup

67. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”) Table Lookup

68. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”) Table Lookup

69. Text Segmentation: Computing P max (“ent”) P max (“ent”) P 0 (“e”) x P max (“nt”)P 0 (“en”) x P max (“t”) P 0 (“n”) x P max (“t”) P 0 (“ent”) x P max (“”) P 0 (“t”) x P max (“”) P 0 (“nt”) x P max (“”)

70. Dynamic Programming through Memoization Any questions?

71. Options TopCoder… –Marathon Match 49 –SRM 418 – Division II Solve text segmentation using dynamic programming