Algorithms: Design and Analysis Summer School 2013 at VIASM: Random Structures and Algorithms Lecture 4: Dynamic Programming Phan Th ị Hà D ươ ng 1

Lecture 4: Dynamic Programming 0. Introduction 1. Matrix-chain multiplication 2* Longest common subsequence 3. All-Pairs shortest paths

0. Introduction Compare to Divide and Conquer algorithm: - Divide to sub problems (independently) - Conquer sub problems recursively - Combine results of sub problems to resolve initial problem. Problems: If the sub problems are not independent, there are common sub sub problem Then we use a table to stock results of common sub sub problems

Divide and Conquer Algorithm: From up to bottom. See the big problem, divide it and conquer the sub problems Dynamic Programming: From bottom to up. Beginning with small problem, construct greater problem, up to the original problem.

A very simple example Problem: Calculate C(n,k) = n!/(n-k)!k! Devide and conquer algorithm: function C(n,k) if (k=0 ou k=n) then return 1; else return C(n-1,k)+C(n-1,k-1); Question: What is the complexity of this algorithm ? Exercise: Write an dynamic programming algorithm to solve this problem. What is its complexity ?

Pascal’s Triangle Write an algorithm to the problem by using Pascal’s triangle.

1. Matrix-chain multiplication Problem: given a chain A1, A2,..., An of n matrices, where for i = 1, 2,..., n, matrix Ai has dimension p(i-1) × p(i). Fully parenthesize the product A1* A2*…* An in a way that minimizes the number of scalar multiplications.

Example: M=ABCD, A=(10,2); B=(2,100); C=(100,3); D=(3,20) Number of multiplications (with different way to parenthesize)

Analyzing the Problem Example: M=ABCD, A=(10,2); B=(2,100); C=(100,3); D=(3,20) Number of multiplications (with different way to parenthesize) ((AB)C)D: 10x2x100 +2x100x3 +10x3x20=3200 muls. (AB)(CD): 10x2x x3x20 +10x100x20 = (A(BC))D: 2x100x3 +10x2x3 +10x3x20 = 1260 A((BC)D): 2x100x3 +2x3x20 +10x2x20 = 1120 A(B(CD)): 100x3x20 +2x100x20 +10x2x20 = 10400

MATRIX-MULTIPLY(A, B) 1 if columns[A] <> rows[B] 2 then error "incompatible dimensions" 3 else for i ← 1 to rows[A] 4 do for j ← 1 to columns[B] 5 do C[i, j] ← 0 6 for k ← 1 to columns[A] 7 do C[i, j] ← C[i, j] + A[i, k] * B[k, j] 8 return C

The number of ways to parenthesize If there n matrix, this number is the number of Catalan C(n) = 1/n x C(2n-2,n-1) = Ω (4^n/n^2). We can not consider all the case to make decision. Idea: If for computing M, a cut at position i is optimal, then an minimum cost is composed of this cut and minimum cost for computing Ai … Aj and minimum cost for computing A(i+1) … An Use a table m(i,j) to stock the minimum cost for computing Ai … Aj.

Computing the minimum cost Computing m(i,j): if we cut at k, then m[i,j]=m[i,k]+m[k+1,j]+p(i-1) p(k) p(j). So the formula for m(i,j) is We compute m(i,j) by diagonal: l = j-i+1, the length of the chain of matrix. We compute m(I,j) for l from 2 to n.

Example M=ABCD, A=(10,2); B=(2,100); C=(100,3); D=(3,20). L=2:a[12]=2000,a[23]=600,a[34]=18000 … j = 1234 i = ? ? ?

Algorithm MATRIX-CHAIN-ORDER(p) 1 n ← length[p] 2 for i ← 1 to n 3 do m[i, i] ← 0 4 for l ← 2 to n ▹ l is the chain length. 5 do for i ← 1 to n - l do j ← i + l – 1 ; m[i, j] ← ∞ 8 for k ← i to j do q←m[i,k]+m[k+1,j]+p(i-1) p(k) p(j) 10 if q < m[i, j] 11-12then m[i, j] ← q; s[i, j] ← k 13return m and s

3.All-Pairs shortest paths Problem: Given a graph G= (V, E) where each edge having a length. Find the shortest path for all pairs of V.

Floyd-Warshall algorithm Notation: V={1, 2,.., n} Length of edges: l[i,i] = 0, l[i,j] = l(e) if (i,j) in E l[i,j] = ∞ otherwise Distance: d(i,j) is the temporal shortest length from i to j.

Idea of FW’s algorithm If k is on a shortest path form I to j, then the sub paths from I to k and from k to j are shortest. Algorithm – Beginning: d = l – After step k, d(i,j) is the shortest path from i to j (path containing only vertices 1, 2, …, k) – After n steps, d(i,j) is the shortest path from i to j

18 Example 0 5 ∞ ∞ D0=L= ∞ ∞ ∞ ∞ D1= D2= D3= D4=

FW’s algorithm Floyd(L) 1array D = L 2 for (k=1 to n) 3 do for (i=1 to n) 4 do for (j=1 to n) 5 do D[i,j]= min(D[i,j],D[i,k]+D[k,j]); 6 return D;

Exercise 1.What is the complexity of the FW’s algorithm ? 2.Prove the correctness of the algorithm. 3.Write an algorithm which return not only the length of shortest path but also a shortest path for each pair of vertices. 4.Write an algorithm to determine if there exist a path from each pair of vertices.