1 Dynamic programming algorithms for all-pairs shortest path and longest common subsequences We will study a new technique—dynamic programming algorithms (typically for optimization problems) Ideas: –Characterize the structure of an optimal solution –Recursively define the value of an optimal solution –Compute the value of an optimal solution in a bottom- up fashion (using matrix to compute) –Backtracking to construct an optimal solution from computed information.

2 Floyd-Warshall algorithm for shortest path: Use a different dynamic-programming formulation to solve the all-pairs shortest-paths problem on a directed graph G=(V,E). The resulting algorithm, known as the Floyd- Warshall algorithm, runs in O (V 3 ) time. – negative-weight edges may be present, –but we shall assume that there are no negative- weight cycles.

3 The structure of a shortest path: We use a new characterization of the structure of a shortest path Different from that for matrix-multiplication-based all-pairs algorithms. The algorithm considers the “intermediate” vertices of a shortest path, where an intermediate vertex of a simple path p= is any vertex in p other than v 1 or v k, that is, any vertex in the set {v 2,v 3,…,v k-1 }

4 Continue: Let the vertices of G be V={1,2,…,n}, and consider a subset {1,2,…,k} of vertices for some k. For any pair of vertices i,j  V, consider all paths from i to j whose intermediate vertices are all drawn from {1,2,…,k},and let p be a minimum-weight path from among them. The Floyd-Warshall algorithm exploits a relationship between path p and shortest paths from i to j with all intermediate vertices in the set {1,2,…,k-1}.

5 Relationship: The relationship depends on whether or not k is an intermediate vertex of path p. Case 1: k is not an intermediate vertex of path p –Thus, a shortest path from vertex i to vertex j with all intermediate vertices in the set {1,2,…,k-1} is also a shortest path from i to j with all intermediate vertices in the set {1,2,…,k}. Case 2: k is an intermediate vertex of path p –we break p down into i k j as shown Figure 2. –p1 is a shortest path from i to k with all intermediate vertices in the set {1,2,…,k-1}, so is p2.

6 i j k p1 p2 All intermediate vertices in {1,2,…,k-1} P:all intermediate vertices in {1,2,…,k} Figure 2. Path p is a shortest path from vertex i to vertex j,and k is the highest-numbered intermediate vertex of p. Path p1, the portion of path p from vertex i to vertex k,has all intermediate vertices in the set {1,2,…,k-1}.The same holds for path p2 from vertex k to vertex j.

7 A recursive solution to the all- pairs shortest paths problem: Let d ij (k) be the weight of a shortest path from vertex i to vertex j with all intermediate vertices in the set {1,2,…,k}. A recursive definition is given by d ij (k) = w ij if k=0, min(d ij (k-1),d ik (k-1) +d kj (k-1) ) if k 1. The matrix D (n) =(d ij (n) ) gives the final answer-d ij (n) = for all i,j V-because all intermediate vertices are in the set {1,2,…,n}.

8 Computing the shortest-path weights bottom up: FLOYD-WARSHALL(W) n rows[W] D (0) W for k 1 to n do for i 1 to n do for j 1 to n d ij (k) min(d ij (k-1),d ik (k-1) +d kj (k-1) ) return D (n)

9 Example: Figure 3 6

10 D (0) = (0) = D (1) = (1) =

11 D (2) = (2) = D (3) = (3) =

12 D (4) = (4) = D (5) = (5) =

13 Longest common subsequence Definition 1: Given a sequence X=x 1 x 2...x m, another sequence Z=z 1 z 2...z k is a subsequence of X if there exists a strictly increasing sequence i 1 i 2...i k of indices of X such that for all j=1,2,...k, we have x ij =z j. Example 1: If X=abcdefg, Z=abdg is a subsequence of X. X=abcdefg, Z=ab d g

14 Definition 2: Given two sequences X and Y, a sequence Z is a common subsequence of X and Y if Z is a subsequence of both X and Y. Example 2: X=abcdefg and Y=aaadgfd. Z=adf is a common subsequence of X and Y. X=abc defg Y=aaaadgfd Z=a d f

15 Definition 3: A longest common subsequence of X and Y is a common subsequence of X and Y with the longest length. (The length of a sequence is the number of letters in the seuqence.) Longest common subsequence may not be unique. Example: abcd acbd Both acd and abd are LCS.

16 Longest common subsequence problem Input: Two sequences X=x 1 x 2...x m, and Y=y 1 y 2...y n. Output: a longest common subsequence of X and Y. A brute-force approach Suppose that m  n. Try all subsequence of X (There are 2 m subsequence of X), test if such a subsequence is also a subsequence of Y, and select the one with the longest length.

17 Charactering a longest common subsequence Theorem (Optimal substructure of an LCS) Let X=x 1 x 2...x m, and Y=y 1 y 2...y n be two sequences, and Z=z 1 z 2...z k be any LCS of X and Y. 1. If x m =y n, then z k =x m =y n and Z[1..k-1] is an LCS of X[1..m-1] and Y[1..n-1]. 2. If x m  y n, then z k  x m implies that Z is an LCS of X[1..m-1] and Y. 2. If x m  y n, then z k  y n implies that Z is an LCS of X and Y[1..n-1].

18 The recursive equation Let c[i,j] be the length of an LCS of X[1...i] and X[1...j]. c[i,j] can be computed as follows: 0 if i=0 or j=0, c[i,j]= c[i-1,j-1]+1 if i,j>0 and x i =y j, max{c[i,j-1],c[i-1,j]} if i,j>0 and x i  y j. Computing the length of an LCS There are n  m c[i,j]’s. So we can compute them in a specific order.

19 The algorithm to compute an LCS 1. for i=1 to m do 2. c[i,0]=0; 3. for j=0 to n do 4. c[0,j]=0; 5. for i=1 to m do 6. for j=1 to n do 7. { 8. if x[I] ==y[j] then 9. c[i,j]=c[i-1,j-1]=1; 10 b[i,j]=1; 11. else if c[i-1,j]>=c[i,j-1] then 12. c[i,j]=c[i-1,j] 13. b[i,j]=2; 14. else c[i,j]=c[i,j-1] 15. b[i,j]=3; 14 }

20 Example 3: X=BDCABA and Y=ABCBDAB.

21 Constructing an LCS (back-tracking) We can find an LCS using b[i,j]’s. We start with b[n,m] and track back to some cell b[0,i] or b[i,0]. The algorithm to construct an LCS 1. i=m 2. j=n; 3. if i==0 or j==0 then exit; 4. if b[i,j]==1 then { i=i-1; j=j-1; print “xi”; } 5. if b[i,j]==2 i=i-1 6. if b[i,j]==3 j=j-1 7. Goto Step 3. The time complexity: O(nm).

22 Shortest common supersequence Definition: Let X and Y be two sequences. A sequence Z is a supersequence of X and Y if both X and Y are subsequence of Z. Shortest common supersequence problem: Input: Two sequences X and Y. Output: a shortest common supersequence of X and Y. Example: X=abc and Y=abb. Both abbc and abcb are the shortest common supersequences for X and Y.

23 Recursive Equation: Let c[i,j] be the length of an LCS of X[1...i] and X[1...j]. c[i,j] can be computed as follows: j if i=0 i if j=0, c[i,j]= c[i-1,j-1]+1 if i,j>0 and x i =y j, min{c[i,j-1]+1,c[i-1,j]+1} if i,j>0 and x i  y j.

24