Dynamic Programming Min Edit Distance Longest Increasing Subsequence Climbing Stairs Minimum Path Sum

Min Edit Distance Given two words X and Y, find the minimum number of steps required to convert X to Y. You have the following 3 operations permitted on a word: a) Insert a character b) Delete a character c) Replace a character

Min Edit Distance For two strings X of length n Y of length m We define D(i,j) the minimum edit distance between X[1..i] and Y[1..j] i.e., the first i characters of X and the first j characters of Y the min edit distance between X and Y is thus D(n, m)

Min Edit Distance Initialization D(i, 0) = i converting X[1..i] to Y[1..0] (empty string) requires at least i Delete steps D(0, j) = j converting X[1..0] (empty string) to Y[1..j] requires at least j Insert steps Recurrent Relation (i,j>0) D(i, j) = min( D(i – 1, j) + 1, D(i, j – 1) + 1, D(i – 1, j – 1) + (1 if X(i) != X(j), or 0 if X(i) == X(j)) )

Min Edit Distance Recurrent Relation D(i, j) = the minimum value of the three D(i – 1, j) + 1 Converting X[1..i] to Y[1..j] by Delete step D(i, j – 1) + 1 Converting X[1..i] to Y[1..j] by Insert step D(i – 1, j – 1) + (1 if X(i) != X(j), or 0 if X(i) == X(j)) Converting X[1..i] to Y[1..j] by Replace step if X(i) != X(j) Or, do nothing if X(i) == X(j)

Min Edit Distance Initialization Recurrent Relation Termination D(n, m) is the minimum edit distance

Min Edit Distance How to Implement? Top-down: Recursion Bottom-up: Dynamic programming

Min Edit Distance Top-down solution int MED(string X, string Y, int i, int j) if i == 0 return j if j == 0 return i min_val = MED(X, Y, i – 1, j – 1) if X.charAt(i-1) != Y.charAt(j-1): min_val += 1 A = MED(X, Y, i – 1, j) + 1 B = MED(X, Y, i, j – 1) + 1 if a < min_val: min_val = a if b < min_val: min_val = b return min_val

Min Edit Distance Top-down solution The time complexity of this solution is exponential In worst case, we may end up doing O(3^m) operations The worst case happens when none of characters of two strings match Worst case example X=“abc” Y=“xyz"

Min Edit Distance Top-down solution Worst case example X=“abc” Y=“xyz"

Min Edit Distance Dynamic Programming Bottom-up solution int MED(String X, String Y) m = int[X.length + 1][Y.length + 1] for i = 0 to X.length + 1: for j = 0 to Y.length + 1: if i == 0: m[0][j] = j else if j == 0: m[i][0] = i else: min_val = m[i – 1][j – 1] if X.charAt(i – 1) != Y.charAt(j – 1): min_val += 1 if m[i – 1][j] + 1 < min_val: min_val = m[i – 1][j] + 1 if m[i][j – 1] + 1 < min_val: min_val = m[i][j – 1] + 1 m[i][j] = min_val return m[X.length][Y.length]

Min Edit Distance Example X=“PARK” Y=“SPAKE”

Min Edit Distance Example X=“PARK” Y=“SPAKE”

Min Edit Distance Question How to trace the solution? R(i, j) = {“R”, “B”, “R-B”}

Longest Increasing Subsequence For example, Given K=[10, 9, 2, 5, 3, 7, 101, 18], The longest increasing subsequence is [2, 3, 7, 101], therefore the length is 4. Note that there may be more than one LIS combination, it is only necessary for you to return the length. We define F(i) The length of the longest increasing subsequence ending with K(i) The length of the longest increasing subsequence thus is the maximum value in array F

Longest Increasing Subsequence Initialization F(i) = 1, i=1..n Recurrent Relation (0<=j<i) F(i) = max ( F(j) + 1, if K(i)>K(j) )

Longest Increasing Subsequence LIT(int [] K) F = int[K.length] for i = 0 to K.length: F[i] = 1 for i = 1 to K.length: for j = 0 to i: if K[j] F[i]: F[i] = F[j] + 1 int max_l = 0 for i = 0 to F.length: if F[i] > max_l: max_l = F[i] return max_l

Climbing Stairs (Fibonacci Sequence) You are climbing a stair case. It takes n steps to reach to the top. Each time you can either climb 1 or 2 steps. In how many distinct ways can you climb to the top? We define F(i) the number of distinct ways we can climb to the i-th step

Climbing Stairs (Fibonacci Sequence) Initialization F(1) = 1 F(2) = 2 Recurrent Relation (i>2) F(i) = F(i – 1) + F(i – 2) int climbStairs(int n) if n == 1: return 1 if n == 2: return 2 int a = 1, b = 2; for i = 2 to n: b = a + b a = b – a return b

Minimum Path Sum (Practice) Given a n x m grid filled(G) with non-negative numbers, find a path from top left to bottom right which minimizes the sum of all numbers along its path. Note: You can only move either down or right at any point in time.

Minimum Path Sum Given a n x m grid filled(G) with non-negative numbers, find a path from top left to bottom right which minimizes the sum of all numbers along its path. Note: You can only move either down or right at any point in time. We define K(i, j) The minimum path sum along the path from G(0, 0) to G(i, j) (0 <=i<n, 0<=j<m) The minimum path sum thus is K(n - 1, m - 1)

Minimum Path Sum Initialization K(0, 0) = G(0, 0) K(0, j) = sum(G(0, 0)...G(0, j)), (0 < j < m) K(i, 0) = sum(G(0, 0)…G(i, 0)), (0 < i < n) Recurrent Relation (0 < i < n, 0 < j < m) K(i, j) = min ( K(i – 1, j) + G(i, j), K(i, j – 1) + G(i, j) )

Minimum Path Sum int MPS(int [][] G): n = G.rows, m = G.cols K = int[n][m] K[0][0] = G[0][0] for i = 1 to n: K[i][0] = K[i – 1][0] + G[i][0] for j = 1 to m: K[0][j] = k[0][j – 1] + G[0][j] for i = 1 to n: for j = 1 to m: K[i][j] = min(K[i – 1][j], K[i][j – 1]) + G[i][j] return K[n – 1][m – 1]

Conclusion Dynamic programming is a method for solving a complex problem by breaking it down into a collection of simpler subproblems solving each of those subproblems just once, and storing their solutions - ideally, using a memory-based data structure The next time the same subproblem occurs, instead of recomputing its solution, one simply looks up the previously computed solution, thereby saving computation time at the expense of a (hopefully) modest expenditure in storage space.