Divide and Conquer (Part II) Multiplication of two numbers Let U = (u 2n-1 u 2n-2 … u 1 u 0 ) 2 and V = (v 2n-1 v 2n-2 …v 1 v 0 ) 2, and our goal is to find U times V. Ordinary multiplication requires execution time α (n 2 ). Alternatively, let U = 2 n U 1 + U 0 and V = 2 n V 1 + V 0 where U 1 = (u 2n-1 u 2n-2 … u n ) 2 and U 0 = (u n-1 u n-2 … u 0 ) 2 Likewise, V 1 = (v 2n-1 v 2n-2 … v n ) 2 and V 0 = (v n-1 v n-2 … v 0 ) 2 U X V = 2 2n U 1 V n (U 1 V 0 + U 0 V 1 ) + U 0 V 0. Apparently there is no saving, this multiplication also requires execution time proportional to n 2. 2/16/2016

Divide and Conquer (Part II) Multiplication: Divide and Conquer U X V = (2 2n + 2 n )U 1 V n (U 1 – U 0 )( V 0 – V 1 ) + (2 n + 1) U 0 V 0. which requires only three multiplications and some extra addition. If T(n) denotes time to multiply two binary integers of size n each, then T(2n) = 3 T(n) + cn, and T(1) =1, where cn denotes the cost associated with additions and shifting of binary integers etc.. 2/16/2016

Divide and Conquer (Part II) Multiplication: Divide and Conquer Solution of this recurrence equation is (by Master Theorem) or repeated substitution, for n = 2 m, is T(2 m ) = 3 m T(1) + c(3 m -2 m ) In general, T(n)  3  lg n  T(1) + c(3  lg n  - 2  lg n  )  3c X 3 lg n = 3c X n lg 3  3c X /16/2016

Divide and Conquer (Part II) Matrix Multiplication Let C = A B = (c ij ) be the product of two n x n matrices A and B, where a 11 a 12 … a 1n b 11 b 12 … b 1n A =a 21 a 22 … a 2n and B = b 21 b 22 … b 2n… … … … a n1 a n2 … a nn b n1 b n2 … b nn We calculate n 2 terms, c ij for i =1,2,…,n,, j =1,2,…,n. Each c ij requires n scalar multiplications. Therefore, total complexity of this matrix multiplication is of the order n 3. 2/16/2016

Divide and Conquer (Part II) Suppose n = 2 k for some integer value of k. Let r … sa …be … g C = … … … = A X B = … … … X … … … t … uc …df … h Where r,s,t,u are all n/2 X n/2 matrices In this way of obtaining C, by first partitioning A and B in submatrices, we calculate 4 matrices r,s,t, and u. 2/16/2016

Divide and Conquer (Part II) Suppose n = 2 k for some integer value of k. Let Each submatrix has two matrix multiplications, of size n/2 X n/2 each. Hence, the recurrence equation associated with this divide and conquer approach would satisfy: T(n) = 8T(n/2) + (n 2 ) T(n) =  (n 3 ) Thus, there is no saving. 2/16/2016

Divide and Conquer (Part II) Matrix Multiplication: Strassen’s Algorithm P 1 = a (g-h) = ag - ah P 2 = (a+b)h = ah + bh P 3 = (c+d) e = ce + de P 4 = d (f-e) = df - de P 5 = (a+d)(e+h)= ae + ah + de + dh P 6 = (b-d)(f+h) = bf + bh - df – dh and P 7 = (a-c) (e+g) = ae + ag – ce - cg 2/16/2016

Divide and Conquer (Part II) Then, we can obtain r,s,t, and u as follows: r = P 5 +P 4 - P 2 +P 6 s = P 1 + P 2 t = P 3 +P 4 u = P 5 +P 1 - P 3 -P 7 For example, P 5 +P 4 - P 2 +P 6 = (ae+ah+de+dh)+(df - de) - (ah + bh) + (bf + bh - df -dh) = ae+bf = r 2/16/2016

Divide and Conquer (Part II) Hence, by Master Theorem, T(n) = 7T(n/2) +  (n 2 ) =  (n lg 7 ) =  (n 2.81 ) In other words, it is possible to multiply two matrices at a rate faster than  (n 3 ). 2/16/2016

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) Method 2: 1.Choose x = A[1]. 2.Set i  2, and set j  n. 3.If A[j] > x, then j  j -1, else go to step 4. 4.If A[i] < x, then i  i +1, else go to step 5. 5.If i < j, then interchange A[i] and A[j], otherwise set q  j and interchange A[1] and A[q]. 2/16/2016

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) Method 3: 1.To partition A[1],…,A[n] this method proceeds as follows: 2.Set i =1, j = n, X = A[1] 3.Compare A[i] and A[j], and exchange if A[i]  A[j]. 4.If no exchange is required then decrease j by 1 and continue. If exchange is required then increase i by 1 after the exchange and continue. If i = j, then stop, else 5.Go to step 2. To see how this method works suppose we wish to partition 503, 087, 512, 061, 908, 170, and /16/2016

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) Quick-Sort A[1], …, A[n] algorithm is: 1.Find a partition of A[1], …, A[n] such that A[1], …, A[q] are all smaller than (new) A[q+1], …, A[n] for 1  q  (n -1) 2.Quick-Sort A[1], …,A[q]. 3.Quick-Sort A[q + 1], …, A[n]. 2/16/2016

Divide and Conquer (Part II) Comments: 1.All comparisons are made with the same element. Amount of data movement is reasonable. This makes these partition procedures reasonable for large values of n. 2.Other choices for X are A[n], A[  n/2  ] and randomly chosen value i.e., A[i]. 3.It makes sense to sort an array by special procedures such as Bubble-Sort, for small values of n, say n  M. Some analyses and other supporting arguments suggest that M  9 is a reasonable bound for shifting from Quick-Sort to Bubble-Sort. 2/16/2016

Divide and Conquer (Part II) Analysis of Quick-sort: Exact analysis of Quick-sort is not possible --- we cannot control the size of the sub-arrays. We will consider the average analysis of Quick-sort. Assumptions: A[1], …, A[n] are all distinct. A[i]'s are randomly distributed. A[i]  {1,2, …, n} for i = 1, …, n. The partition Method #3 is used. 2/16/2016

Divide and Conquer (Part II) Observations: 1.All n! permutations of A[1], …, A[n] are equally likely. 2.Suppose A[i] = k. Then after one application of the partition method, size of the left sub-array is (k - 1), 3.size of the right sub-array is (n - k) and k is in its correct position. 4.Pr (A[i] = k) = 1/n. 5.Exactly (n + 1) comparisons are made in arranging the given array A[1], …, A[n] in partition method 2. [Reason - Each time i is increased by 1 or j is decreased by 1 a comparison is made until i becomes equal to j.] 2/16/2016

Divide and Conquer (Part II) 2/16/2016 Suppose AC(n) = Average number of comparisons required to sort A[1], …, A[n] by Quick-sort. Then, AC(n) = (n + 1) + (1/n){ [AC(k - 1) + AC(n - k)]} = (n + 1) + (1/n){ [AC(k) + AC(n - k - 1)]} = (n + 1) + AC(k) Consequently, n AC(n) = n (n + 1) + 2 [AC(0) + … + AC(n - 1)]. Substituting (n + 1) for n in the above equation gives, (n + 1) AC(n + 1) = (n + 1) (n + 2) + 2 [AC(0) + … + AC(n - 1) + AC(n)],

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) 2/16/2016

Divide and Conquer (Part II) Selection Problem 2/16/2016 Find the i-th smallest element of a given array A[1], A[2], …,A[n]} The algorithm described below borrows an idea from Quick-Sort--- how to partition an array into two subarrays. Algorithm 1.Call A[1] the pivotal element. 2.Use a partition procedure, as in Quick-Sort, to find the location of the pivotal element, k, in the partitioned array (in which all elements to the left of the pivotal element are smaller than it and to the right are larger than it). 3.If i = k, then we have found the i-th smallest element; the answer is (old) A[1].

Divide and Conquer (Part II) 2/16/ If 1  i  k-1, then find the i-th smallest element among (new) A[1], …, A[k-1], else 5.if k<i  n, then find the i-th smallest element among new A[k+1], …,A[n]. (This case can be seen as equivalent to finding the (i-k)-th smallest element out of (n-k) of the right subarray after renaming A[k+1], …, A[n] as A[1], …,A[n-k]). Analysis of the above algorithm Please see the book

Divide and Conquer 2/16/2016 Delete sample document icons and replace with working document icons as follows: From Insert Menu, select Object... Click “Create from File” Locate File name in “File” box Make sure “Display as Icon” is checked Click OK Select icon From Slide Show Menu, Select “Action Settings” Click “Object Action” and select “Edit” Click OK Questions