Unit-2 Divide and Conquer Analysis and design of algorithm

Outline Overview Structure of divide-and-conquer algorithms Applications- binary search quick sort Strassen multiplication

Overview The Divide and Conquer Paradigm, is a method of designing algorithms, a general approach to construct an efficient solution to a problem. The Divide and Conquer Paradigm is an algorithm design paradigm which uses this simple process: It Divides the problem into smaller sub- parts until these sub-parts become simple enough to be solved, and then the sub parts are solved recursively, and then the solutions to these sub-parts can be combined to give a solution to the original problem.

this paradigm follows three basic steps; Step 1: Break down the problem into smaller sub-problems, Step 2: Solve these separate sub-problems, and Step 3: Combine the solutions of the sub-problems.

Advantages Solving difficult problems : Divide and conquer is a powerful tool for solving conceptually difficult problems: all it requires is a way of breaking the problem into sub-problems, of solving the trivial cases and of combining sub-problems to the original problem Algorithm efficiency : The divide-and-conquer paradigm often helps in the discovery of efficient algorithms. It was the key, for example, to Karatsuba's fast multiplication method, the quicksort and mergesort algorithms, the Strassen algorithm for matrix multiplication, and fast Fourier transforms. Parallelism : Divide and conquer algorithms are naturally adapted for execution in multi-processor machines, especially shared-memory systems where the communication of data between processors does not need to be planned in advance, because distinct sub-problems can be executed on different processors.

Mechanism Divide : divide the problem into smaller sub problems. Conquer : the sub problem solving them recursively. If they are small enough solve the sub problem as base case. Combine : merge the solution of sub problem to find solution of actual problem.

Applications Merge Sort Quick Sort Binary Search Strassen's Matrix Multiplication Closest pair (points)

Application-1Binary Search This search algorithm works on the principle of divide and conquer. For this algorithm to work properly the data collection should be in sorted form. Binary search search a particular item by comparing the middle most item of the collection. If match occurs then index of item is returned. If middle item is greater than item then item is searched in sub-array to the right of the middle item other wise item is search in sub-array to the left of the middle item. This process continues on sub-array as well until the size of subarray reduces to zero.

Algorithm/ pseudo code Procedure binary_search A ← sorted array n ← size of array x ← value ot be searched Set lowerBound = 1 Set upperBound = n while x not found if upperBound < lowerBound EXIT: x does not exists. set midPoint = lowerBound + ( upperBound - lowerBound ) / 2 if A[midPoint] < x set lowerBound = midPoint + 1 if A[midPoint] > x set upperBound = midPoint - 1 if A[midPoint] = x EXIT: x found at location midPoint end while end procedure

Example: The below given is our sorted array and assume that we need to search location of value 31 using binary search. First, we shall determine the half of the array by using this formula − mid = low + (high - low) / 2 Here it is, 0 + (9 - 0 ) / 2 = 4 (integer value of 4.5). So 4 is the mid of array

Application-2 Quick Sort Quick sort is a highly efficient sorting algorithm and is based on partitioning of array of data into smaller arrays. A large array is partitioned into two arrays one of which holds values smaller than specified value say pivot based on which the partition is made and another array holds values greater than pivot value. The quick sort partitions an array and then calls itself recursively twice to sort the resulting two subarray. This algorithm is quite efficient for large sized data sets as its average and worst case complexity are of O(nlogn) where n are no. of items.

Steps Divide: Rearrange the elements and split the array into two subarrays and an element in between such that so that each element in the left subarray is less than or equal the middle element and each element in the right subarray is greater than the middle element. Conquer: Recursively sort the two subarrays. Combine: None

Basic Idea Pick one element in the array, which will be the pivot. Make one pass through the array, called a partition step, re-arranging the entries so that: the pivot is in its proper place. entries smaller than the pivot are to the left of the pivot. entries larger than the pivot are to its right. Recursively apply quicksort to the part of the array that is to the left of the pivot,  and to the right part of the array. Here we don't have the merge step, at the end all the elements are in the proper order.

Algorithm STEP 1. Choosing the pivot Choosing the pivot is an essential step.  Depending on the pivot the algorithm may run very fast, or in quadric time.: Some fixed element: e.g. the first, the last, the one in the middle This is a bad choice - the pivot may turn to be the smallest or the largest element,  then one of the partitions will be empty. Randomly chosen (by random generator ) - still a bad choice. The median of the array (if the array has N numbers, the median is the [N/2] largest number. This is difficult to compute - increases the complexity. The median-of-three choice: take the first, the last and the middle element.  Choose the median of these three elements.

Example: 8, 3, 25, 6, 10, 17, 1, 2, 18, 5 The first element is 8, the middle - 10, the last - 5. The median of [8, 10, 5] is 8

STEP 2. Partitioning Partitioning is illustrated on the above example. 1. The first action is to get the pivot out of the way - swap it with the last element 5, 3, 25, 6, 10, 17, 1, 2, 18, 8 2. We want larger elements to go to the right and smaller elements to go to the left. Two "fingers" are used to scan the elements from left to right and from right to left:

STEP 3. Recursively quicksort the left and the right parts [5, 3, 25, 6, 10, 17, 1, 2, 18, 8] ^ ^ i j While i is to the left of j, we move i right, skipping all the elements  less than the pivot. If an element is found greater then the pivot, i stops While j is to the right of i, we move j left, skipping all the elements  greater than the pivot. If an element is found less then the pivot, j stops When both i and j have stopped, the elements are swapped. When i and j have crossed, no swap is performed, scanning stops,  and the element pointed to by i is swapped with the pivot . In the example the first swapping will be between 25 and 2, the second between 10 and 1. 3. Restore the pivot. After restoring the pivot we obtain the following partitioning into three groups: [5, 3, 2, 6, 1] [ 8 ] [10, 25, 18, 17] STEP 3. Recursively quicksort the left and the right parts

Quick Sort Pivot Algorithm Step 1 − Choose the highest index value as pivot Step 2 − Take two variables to point left and right of the list excluding pivot Step 3 − left points to the low index Step 4 − right points to the high Step 5 − while value at left is less than pivot move right Step 6 − while value at right is greater than pivot move left Step 7 − if both step 5 and step 6 does not match swap left and right Step 8 − if left ≥ right, the point where they met is new pivot

Quick Sort Algorithm Step 1 − Make the right-most index value pivot Step 2 − partition the array using pivot value Step 3 − quicksort left partition recursively Step 4 − quicksort right partition recursively

Example

Application-3 Strassen Multiplication Basic Mathod: void multiply(int A[][N], int B[][N], int C[][N]) {     for (int i = 0; i < N; i++)     {         for (int j = 0; j < N; j++)         {             C[i][j] = 0;             for (int k = 0; k < N; k++)             {                 C[i][j] += A[i][k]*B[k][j];             }         }     } } Time Complexity of above method is O(N3).

Let A, B be two square matrices. Calculate the matrix product C as If the matrices A, B are not of type 2n × 2n we fill the missing rows and columns with zeros. We partition A, B and C into equally sized block matrices With then

Strassen showed that 2x2 matrix multiplication can be accomplished in 7 multiplication and 18 additions or subtractions. .(2log27 =22.807) How???

 = This reduction can be done by Divide and Conquer Approach A  B = R   Divide matrices into sub-matrices: A0 , A1, A2 etc Use blocked matrix multiply equations Recursively multiply sub-matrices  =

Strassan Algorithm P1 = (A11+ A22)(B11+B22) P2 = (A21 + A22) * B11 P3 = A11 * (B12 - B22) P4 = A22 * (B21 - B11) P5 = (A11 + A12) * B22 P6 = (A21 - A11) * (B11 + B12) P7 = (A12 - A22) * (B21 + B22) C11 = P1 + P4 - P5 + P7 C12 = P3 + P5 C21 = P2 + P4 C22 = P1 + P3 - P2 + P6

C11 = P1 + P4 - P5 + P7 = (A11+ A22)(B11+B22) + A22 C11 = P1 + P4 - P5 + P7 = (A11+ A22)(B11+B22) + A22 * (B21 - B11) - (A11 + A12) * B22+ (A12 - A22) * (B21 + B22) = A11 B11 + A11 B22 + A22 B11 + A22 B22 + A22 B21 – A22 B11 - A11 B22 -A12 B22 + A12 B21 + A12 B22 – A22 B21 – A22 B22 = A11 B11 + A12 B21 Complexity: