Lecture 6 Sorting II Divide-and-Conquer Algorithms

Sorting Algorithms Quadratic O(n 2 ): Selection Sort, Insertion Sort, Bubble Sort. Divide-and-Conquer: Merge Sort, Quick Sort O(n log n) Using Special Data Structures: HeapSort O(n log n) Non-comparison based (Domain dependent): Radix Sort (complexity depends on domain size of numbers being sorted)

Merge Sort Pseudo code and applet, see class web page We did analysis last week. Remaining: algorithm for Merging two sorted lists in O(n) time

Merging two sorted lists using additional space Algorithm: –Compare the top elements of each list –Place the smallest in a new array –Continue until one list becomes empty Analysis: each operation (comparison) is charged to the element being moved to the new list. Total elements moved: n

Divide-and-Conquer Paradigm General technique for algorithm design Often leads to very efficient algorithms Paradigm: –Divide the problem into 2 or more smaller subproblems –Solve the subproblems recursively –Combine the solutions to the subproblems to obtain solution of original problem

Examples of Divide-and- Conquer Algorithms Binary Search Merge Sort Quick Sort Certain Convex Hull algorithms in Computational Geometry Many parallel algorithms

QuickSort Pseudocode and applet from Dominique’s page. Algorithm:Algorithm: –Choose an element of the array (the pivot). For example, the first element. –Scan the array and partition the elements into two subarrays: those smaller and those larger than the pivot, and place them in the array with the pivot between them. –Recursively sort the two subarrays.

Worst case analysis of Quick Sort Happens when one of the two subarrays is very small, the other very large: 1 and n-1 elements each. Recurrence: T(n)=T(1)+T(n-1)+n Solution to the recurrence: T(n)=O(n 2 )

Average case analysis of Quick Sort I won’t do the formal analysis, need background from probability theory (some done in Discrete) Idea: choose the pivot at random. On the average, half of the elements are larger, half are smaller than it. Hence on the average it splits the array in half, which leads to the known recurrence: T(n) = 2 T(n/2) + n Hence on average T(n)= O(n log n)

Other Sorting Algorithms next time (Heap Sort) Now: more divide-and-conquer algorithms

Selection Finding the minimum: n-1 steps Finding the second largest element: n-1+n-2=2n-3. Can you do better? Finding the third largest: 3n-6. Better? Median: n/2th element: O(n 2 ). Better? There exist O(n) time algorithms based on divide-and-conquer

Matrix Multiplication a11a12a13a14 a21a22a23a24 a31a32a33a34 a41a42a43a44 b11b12b13b14 b21b22b23b24 b31b32b33b34 b41b42b43b44 c11c12c13c14 c21c22c23c24 c31c32c33c34 c41c42c43c44 = x C ij =  n k=1 a ik b kj O(n 3 ) multiplications

Divide and Conquer I Pp textbook Divide the matrix in 2 by rows and columns (4 parts) Multiply –T(n)=8T(n/2) –T(1)=1 Solution? In-class exercise

Strassen’s Matrix Multiplication Algorithm m1=(a12-a22)(b21+b22) m2=(a11+a22)(b11+b22) m3=(a11-a21)(b11+b12) m4=(a11+a12)b22 m5=a11(b12-b22) m6=(a21+a22)b11 c11=m1+m2-m4+m6 c12= m4+m5 c21=m6+m7 c22=m2-m3+m5-m7 Time: T(n)=7T(n/2) T(1)=1 Solution? In-class exercise.

Convex Hull Algorithm Show applet pp textbook