1. Outline 1. General Design and Problem Solving Strategies 2. More about Dynamic Programming – Example: Edit Distance 3. Backtracking (if there is time) – Another Strategy for the Knapsack Problem:

2. Design Strategies Dynamic Programming Design Strategy – Solve an “easy” sub-problem – Store the solution – Use stored solution to solve a more difficult sub- problem. – Repeat until you solve the “big” hard problem Other Strategies – Divide and Conquer – Brute Force – Greedy

3. Design Strategies Dynamic Programming is not divide and conquer. Consider Floyd’s algorithm – At no point did we break the input into two parts – What is Floyd’s algorithm really doing?

4. Design Strategies What is Floyd’s algorithm really doing? – STEP 1: Find all the shortest paths allowing a hop through vertex A, store the shortest paths – STEP 2: Now, use that answer to find the shortest paths allowing a hop through vertex B The algorithm exploits what was already computed, so STEP 2 really finds the shortest paths allowing hops through vertex A and B.

5. More General Design Strategies Top Down – See the big picture first – Break it into parts – Analyze each part – Continue breaking down sub-parts into solvable tasks Quicksort is a classic example.

6. More General Design Strategies Top Down - Quicksort – See the big picture first Need to put items in the correct sorted position – Break it into parts Put the pivot in the correct position and partition the list into two parts – Analyze each part Pick a pivot for each part… – Continue breaking down sub-parts into solvable tasks Continue recursively until sub-parts are lists of size 1

7. More General Design Strategies Bottom Up – Use the solution to larger and larger problems to solve the BIG problem and see the big picture – Use solution to small tasks to solve larger problems – Identify easily solvable tasks Mergesort is a classic example

8. More General Design Strategies Bottom Up - Mergesort – Use the solution to larger and larger problems to solve the BIG problem and see the big picture Merging the final two sorted list – Use solution to small tasks to solve larger problems Merging sorted lists – Identify easily solvable tasks Sorting lists of size 2

9. More General Design Strategies Is Floyd’s Algorithm Top Down or Bottom Up?

10. More General Design Strategies Divide and Conquer can be both Top Down or Bottom Up Dynamic Programming tends to only be Bottom Up.

11. More General Design Strategies Consider Bottom-up Strategies – Divide and Conquer usually merges two smaller sub-problems into a large problem (N/2 + N/2)  N – Dynamic Programming usually extends the solution in some way N-2  N-1 N_simple_version  N_harder_version

12. More about Dynamic Programming How does Floyd’s Algorithm extend the solution? – N-2  N-1 Does it consider a smaller graph and then extends the solution to a larger graph? – N_simple_version  N_harder_version Does it consider a simpler shortest path problem and extend it to a more complex shortes path problem?

13. More about Dynamic Programming In a graph, it is really easy to solve the shortest path problem if you do not allow any hops (intermediate vertices) – The adjacency matrix stores all the shortest paths (direct hop)

14. More about Dynamic Programming It is also easy to solve the problem if you only allow a hop through vertex x if (M[a][x] + M[x][b] < M[a][b]) then – update the distance O(N 2 ) is required to update all the cells Then, just repeat this process N times; one for each intermediate vertex. O(N 3 ) total time

15. Top Down vs. Bottom Up Top Down – Rethinking the design of existing ideas/inventions – Managing projects that are underway – Works really good in a Utopian world Bottom Up – Designing totally new ideas – Putting together projects from scratch – Seen more often in the real world.

16. Bottom-up Design Top Down – Lets build a flying carriage; what are the parts? – Lift, propulsion, steering, etc. Lets build a steering mechanism; what are the parts? – We need a steering control – Umm? Wait, we need to know how the other parts work first. Lets build a lift mechanism; how do we do this? – ??? Lets build a propulsion mechanism

17. Bottom-up Design Bottom UP – Discoveries: This shape produces lift A spinning propeller creates propulsion in the air Canvas with a wood frame is light enough – Next Step: Perhaps we can build an stable, controllable flying thing.

18. Bottom-up Design Before we can analyze the big picture We have to – Look at some of the initial smaller problems – See how they were solved – See how they led to new discoveries

19. Another Dynamic Programming Algorithm Problem: – Find The Edit Distance Between Two Strings Solutions: – Brute Force – O(K N ) – Greedy – No Optimal Algorithms yet – Divide & Conquer – None discovered yet – Dynamic Programming – O(N 2 )

20. Edit Distance How many edits are needed to exactly match the Target with the Pattern Target:TCGACGTCA Pattern: TGACGTGC

21. Edit Distance How many edits are needed to exactly match the Target with the Pattern Target:TCGACGT CA Pattern: T GACGTGC Three: – By Deleting C and A from the target, and by Deleting G from the Pattern

22. Edit Distance Applications: – Approximate String Matching – Spell checking – Google – finding similar word variations – DNA sequence comparison – Pattern Recognition

23. Edit Distance – Dynamic Programming ACGTCGCAT A C G T G T G C 01234567891 2 3 4 5 6 7 8 0 12345678 1 21234567232123456323212345432321234543432123654543234765655323 Optimal edit distance for TG and TCG Optimal edit distance for TG and TCGA Optimal edit distance for TGA and TCG Final Answer Optimal edit distance for TGA and TCGA

24. Edit Distance int matrix[n+1][m+1]; for (x = 0; x <= n; x++) matrix[x][0] = x; for (y = 1; y <= m; y++) matrix [0][y] = y; for (x = 1; x <= n; x++) for (y = 1; y <= m; y++) if (seq1[x] == seq2[y]) matrix[x][y] = matrix[x-1][y-1]; else matrix[x][y] = max(matrix[x][y-1] + 1, matrix[x-1][y] + 1); return matrix[n][m]; 01234567891 2 3 4 5 6 7 8 012345678 121234567 232123456 323212345 432321234 543432123 654543234 765655323

25. Edit Distance int matrix[n+1][m+1]; for (x = 0; x <= n; x++) matrix[x][0] = x; for (y = 0; y <= m; y++) matrix [0][y] = y; for (x = 1; x <= n; x++) for (y = 1; y <= m; y++) if (seq1[x] == seq2[y]) matrix[x][y] = matrix[x-1][y-1]; else matrix[x][y] = max(matrix[x][y-1] + 1, matrix[x-1][y] + 1); return matrix[n][m]; How many times is this comparison performed? How many times is this assignment performed?

26. Edit Distance – Dynamic Programming ACGTCGCAT A C G T G T G C 01234567891 2 3 4 5 6 7 8 0 12345678 1 21234567232123456323212345432321234543432123654543234765655323 To derive the value 7, we need to know that we can match two T’s n=8 In the worst case, this may take n comparisons To derive the value 6, we need to know that we can match two C’s after matching two T’s To derive this value 5, we need to know that we can match two G’s after already matching two C’s and previously matching two T’s

27. Edit Distance – Dynamic Programming ACGTCGCAT A C G T G T G C 01234567891 2 3 4 5 6 7 8 0 12345678 1 21234567232123456323212345432321234543432123654543234765655323 Given our previous matches, there is no way we can match two A’s Thus, the edit distance is increased Luckily, we can match these two C’s But now we’ve matched the last symbol We can’t do any more matching (period!)

28. Lesson to learn There is no way to compute the optimal (minimum) edit distance without considering all possible matching combinations. The only way to do that is to consider all possible sub-problems. This is the reason the entire table must be considered. If you can compute the optimal (minimum) edit distance using less than O(nm) computations. Then you will be renown!