CSC 252 Algorithms Haniya Aslam DYNAMIC PROGRAMMING: Matrix Chain Multiplication & Optimal Triangulation CSC 252 Algorithms Haniya Aslam

Presentation Overview Understanding dynamic programming Dynamic programming vs. Recursion and Demand & Conquer Matrix chain multiplication Optimal polygon triangulation Acknowledgements

Dynamic Programming A generalization of iteration and recursion. “Dynamic Programming is recursion’s somewhat neglected cousin. …(It) is the basis of comparison and alignment routines. Bottom-up design: Start at the bottom Solve small sub-problems Store solutions Reuse previous results for solving larger sub-problems

Dynamic Programming cont. Fibonacci: function Fibonacci(n: integer) : integer; var i : index; sum, interm1, interm2: integer; begin interm1:= 0; {F0} interm2:= 1; {F1} for i:=3 to n do sum :=interm1 + interm2; interm1:=interm2; interm2:= sum; end {for} Fibonacci := sum; end {Fibonacci}

Dynamic Programming vs. Recursion and Divide & Conquer In a recursive program, a problem of size n is solved by first solving a sub-problem of size n-1. In a divide & conquer program, you solve a problem of size n by first solving a sub-problem of size k and another of size k-1, where 1 < k < n. In dynamic programming, you solve a problem of size n by first solving all sub-problems of all sizes k, where k < n.

Matrix Chain Multiplication Given : a chain of matrices {A1,A2,…,An}. Once all pairs of matrices are parenthesized, they can be multiplied by using the standard algorithm as a sub-routine. A product of matrices is fully parenthesized if it is either a single matrix or the product of two fully parenthesized matrix products, surrounded by parentheses. [Note: since matrix multiplication is associative, all parenthesizations yield the same product.]

Matrix Chain Multiplication cont. For example, if the chain of matrices is {A, B, C, D}, the product A, B, C, D can be fully parenthesized in 5 distinct ways: (A ( B ( C D ))), (A (( B C ) D )), ((A B ) ( C D )), ((A ( B C )) D), ((( A B ) C ) D ). The way the chain is parenthesized can have a dramatic impact on the cost of evaluating the product.

Matrix Chain Multiplication Optimal Parenthesization Example: A[30][35], B[35][15], C[15][5] minimum of A*B*C A*(B*C) = 30*35*5 + 35*15*5 = 7,585 (A*B)*C = 30*35*15 + 30*15*5 = 18,000 How to optimize: Brute force – look at every possible way to parenthesize : Ω(4n/n3/2) Dynamic programming – time complexity of Ω(n3) and space complexity of Θ(n2).

Matrix Chain Multiplication Structure of Optimal Parenthesization For n matrices, let Ai..j be the result of AiAi+1….Aj An optimal parenthesization of AiAi+1…An splits the product between Ak and Ak+1 where 1  k < n. Example, k = 4 (A1A2A3A4)(A5A6) Total cost of A1..6 = cost of A1..4 plus total cost of multiplying these two matrices together.

Matrix Chain Multiplication Overlapping Sub-Problems Overlapping sub-problems helps in reducing the running time considerably. Create a table M of minimum Costs Create a table S that records index k for each optimal sub-problem Fill table M in a manner that corresponds to solving the parenthesization problem on matrix chains of increasing length. Compute cost for chains of length 1 (this is 0) Compute costs for chains of length 2 A1..2, A2..3, A3..4, …An-1…n Compute cost for chain of length n A1..n Each level relies on smaller sub-strings

Optimal Polygon Triangulation A triangulation of a polygon is a set of T chords of the polygon that divide the polygon into disjoint triangles. In a triangulation, no chords intersect (except at end-points) and the set T of chords is maximal: every chord not in T intersects some chord in T. The sides of triangles produced by the triangulation are either chords in the triangulation or sides of the polygon. Every triangulation of an n-vertex convex polygon has n-3 chords and divides the polygon into n-2 triangles. a. b.

Optimal Polygon Triangulation cont. In the optimal polygon triangulation problem , we are given a polygon P = {v0, v1, v2, …., vn-1} and a weight function w defined on triangles formed by sides and chords of P. The problem is to find a triangulation that minimizes the sum of the weights of the triangles in the triangulation. This problem, like matrix chain multiplication, uses parenthesization. A full parenthesization corresponds to a full binary tree also called a parse tree.

Parenthesization in Triangulation Above is the parse tree for the triangulation of a polygon.The internal nodes of the parse tree are the chords of the triangulation plus the side v0v6, which is the root.

Triangulation and Matrix Chain Multiplication Since a fully parenthesized product of n matrices corresponds to a parse tree with n leaves, it therefore also corresponds to a triangulation of an (n+1)-vertex polygon. Each matrix Ai in a product of A1A2…An corresponds to side vi-1vi of an (n+1)-vertex polygon. The matrix chain is actually a special case of the optimal triangulation problem.

Triangulation and Matrix Chain Multiplication cont. Given a matrix chain product A1A2….AN, we define an (n+1)-vertex convex polygon p = {v0,v1,….,vn}. If matrix Ai has dimensions pi-1 x pi, for I = 1,2,…,n, the weight function for the triangulation is defined as w(Δvivjvk) = pipjpk An optimal triangulation of P with respect to this weight function gives the parse tree for an optimal parenthesization of A1A2….An.

Substructure of an optimal triangulation Given: an optimal triangulation T of an (n+1)-vertex polygon P that includes the triangle Δv0vkvn, where 1  k  n-1. The weight of T is the sum of the weights of Δv0vkvn and triangles in the triangulation of the two sub-polygons {v0,v1,….vk} and {vk, vk+1,….vn}. The triangulation of the sub-polygons determined by T , therefore, must be optimal, since a lesser-weight triangulation of either sub-polygon would contradict the minimality of the eight of T.

T[i,j] = min {t[i,k] + t[k+1,j] + w(Δvi-1vkvj)} A Recursive Solution Given, for 1  I  j  n: T[i,j] is the weight of an optimal triangulation of the polygon {vi-1, vi, …..vj} [Since m[i,j] is the minimum cost of computing the matrix chain sub-product AiAi+1….Aj] In the case of a 2-vertex polygon: t[i,i] = 0 for i = 1,2,…,n. To minimize over all vertices vk, where k = i,i+1,…j-1, the weight of Δvi-1vkvj plus the weights of the optimal triangulation of the polygons {vi-1, vi,….,vk} and {vk,vk+1,…,vj}. The recursive solution then is: If i < j, T[i,j] = min {t[i,k] + t[k+1,j] + w(Δvi-1vkvj)} i k j-1

Acknowledgments http://www-cse/uta/edu/~holder/courses/cse5311/lectures/18/node18.html http://www.middlebury.edu/~dickerso/ccsc/ugcg.html http://www.eecs.harvard.edu/~nr/cs152/readings/dynamic.html http://www.catalase.com/dprog.htm http://mail.informs.org/classes/dynamic/node1.html http://cse.hanyang.ac.kr/~jmchoi/c…6-2/algorithm/classnote/node6.html http://people.bu.edu/rlynch/cs566/sld002.htm