CS223 Advanced Data Structures and Algorithms 1 Sorting and Master Method Neil Tang 01/21/2009

CS223 Advanced Data Structures and Algorithms 2 Class Overview  Review of sorting algorithms: Insertion, merge and quick  The master method

CS223 Advanced Data Structures and Algorithms 3 Insertion Sort  Start from the second element and select an element (key) in each step.  Insert the key to the current sorted subsequence such that the new subsequence is in the sorted order.

CS223 Advanced Data Structures and Algorithms 4 Insertion Sort ji key ji ji ji

CS223 Advanced Data Structures and Algorithms Insertion Sort Insertion-Sort(A[1..n]) 1for j := 2 to n do 2key := A[j] i := j -1 5while i > 0 and A[i] > key do 6A[i+1] := A[i] 7i := i - 1 8A[i+1] := key 5

Insertion Sort CS223 Advanced Data Structures and Algorithms 6  Best case: the array is sorted each while loop only takes constant time, which is repeated (n-1) times, so time complexity is Θ(n).  Worst case: the array is in reverse order each while loop takes cj time. So time complexity is

CS223 Advanced Data Structures and Algorithms 7 Merge Sort  Divide: Divide the N-element sequence into 2 subsequences of N/2 each.  Conquer: Sort each subsequence recursively using merge sort.  Combine: Merge two sorted subsequences to produce a single sorted sequence.  The time complexity is  O(NlogN)

CS223 Advanced Data Structures and Algorithms 8 Merge Sort

CS223 Advanced Data Structures and Algorithms 9 Quick Sort  Divide: Divide the sequence into 2 subsequences, s.t. each element in the 1st subsequence is less than or equal to each element in the 2nd subsequence.  Conquer: Sort each subsequence recursively using quick sort.  Combine: no work is needed.

CS223 Advanced Data Structures and Algorithms 10 Quick Sort age 25 People age < 25 people age  25 age 23 people age < 23 age 30 people age  30 people age  23 people age < 30

CS223 Advanced Data Structures and Algorithms 11 Master Method  Recurrence in general form: T(N) = aT(N/b) + f(N), a  1 and b > 1.  Case 1: If f(N) = O(N log b a -  ) for some constant  > 0, then T(N) =  (N log b a ).  Case 2: If f(N) =  (N log b a ), then T(N) =  (N log b a logN).  Case 3: If f(N) =  (N log b a +  ) for some constant  > 0, and a*f(N/b)  c*f(N) for some constant c < 1 and all sufficiently large N, then T(N) =  (f(N)).

CS223 Advanced Data Structures and Algorithms 12 Example: Binary Search  T(N) = T(N/2) + 1  a=1, b=2, f(N)=1  log 2 1 = 0  So we have case 2, T(N) =  (logN).

CS223 Advanced Data Structures and Algorithms 13 Example: Merge Sort  T(N) = 2T(N/2) + N  a=2, b=2, f(N)=N  log 2 2 = 1  So we have case 2, T(N) =  (NlogN).

CS223 Advanced Data Structures and Algorithms 14 Example: Matrix Multiplication  T(N) = 8T(N/2) + N 2  a=8, b=2, f(N)=N 2  log 2 8 = 3    = 0.5  So we have case 1, T(N) =  (N 3 ).

CS223 Advanced Data Structures and Algorithms 15 More Example  T(N) = 3T(N/4) + NlogN  a=3, b=4, f(N) = NlogN  log 4 3 =    =0.2, c=3/4  So we have case 3, T(N) =  (NlogN).

CS223 Advanced Data Structures and Algorithms 16 Limitations  Does the master method work all the time?  Quick sort: T(N) = T(N-1) + N  Fibonacci number: T(N) = T(N-1) + T(N-2) + 1  Does it work for all the cases in the format of T(N) =aT(N/b) + f(N)?