1. Design and Analysis of Algorithms Quicksort Haidong Xue Summer 2012, at GSU

2. Review of insertion sort and merge sort Insertion sort – Algorithm – Worst case number of comparisons = O(?) Merge sort – Algorithm – Worst case number of comparisons = O(?)

3. Sorting Algorithms AlgorithmWorst TimeExpected TimeExtra Memory Insertion sortO(1) Merge sortO(n) Quick sortO(1) Heap sortO(1)

4. Quicksort Algorithm Input: A[1, …, n] Output: A[1,.., n], where A[1]<=A[2]…<=A[n] Quicksort: 1.if(n<=1) return; 2.Choose the pivot p = A[n] 3.Put all elements less than p on the left; put all elements lager than p on the right; put p at the middle. (Partition) 4. Quicksort(the array on the left of p) 5. Quicksort(the array on the right of p)

5. Quicksort Algorithm Quicksort example 28713 5 64 2 8 7 1 3 5 6 4 28 7 13 5 6 4 Current pivots Previous pivots Quicksort Hi, I am nothing Nothing Jr. Nothing 3rd 2 8 7 1 3 5 6 4 2 8 7 1 3 5 6 4 2 8 7 1 3 5 6 4

6. Quicksort Algorithm More detail about partition Input: A[1, …, n] (p=A[n]) Output: A[1,…k-1, k, k+1, … n], where A[1, …, k-1] A[k], A[k]=p Partition: 1.t = the tail of smaller array 2.from i= 1 to n-1{ if(A[i]<p) { exchange A[t+1] with A[i]; update t to the new tail; } 3.exchange A[t+1] with A[n];

7. Quicksort Algorithm Partition example 28713 5 64 28713 5 64 28713 5 64 tail 28713 5 64 Exchange 2 with A[tail+1] Do nothing

8. Quicksort Algorithm Partition example 28713 5 64 2 8 713 5 64 2 8 71 3 5 64 tail Exchange 1 with A[tail+1] tail Exchange 3 with A[tail+1] Do nothing 2 8 7 1 3 5 64 tail Do nothing 2 8 7 1 3 5 6 4 Final step: exchange A[n] with A[tail+1]

9. Quicksort Algorithm The final version of quick sort: Quicksort(A, p, r){ if(p<r){ //if(n<=1) return; q = partition(A, p, r); //small ones on left; //lager ones on right Quicksort(A, p, q-1); Quicksort(A, q+1, r); }

10. Analysis of Quicksort Time complexity – Worst case – Expected Space complexity - extra memory – 0 = O(1)

11. Analysis of Quicksort

12. Strict proof of the worst case time complexity

14. Strict proof of the expected time complexity

15. Java implementation of Quicksort Professional programmers DO NOT implement sorting algorithms by themselves We do it for practicing algorithm implementation techniques and understanding those algorithms Code quicksort – Sort.java // the abstract class – Quicksort_Haydon.java // my quicksort implementation – SortingTester.java // correctness tester

16. Design and Analysis of Algorithms Review on time complexity, “in place”, “stable” Haidong Xue Summer 2012, at GSU

17. Time complexity of algorithms Execution time? – Pros: easy to obtain; Cons: not accurate Number Instructions? – Pros: very accurate; Cons: calculation is not straightforward Number of certain operations? – Pros: easy to calculate, generally accurate; Cons: not very calculate Asymptotic Notations Pros? Cons?

18. Time complexity of algorithms In the worst case for(int i=0; i<A.length; i++) C1: cost of “assign a value ” C2: cost of “<” C3: cost of “increment by 1” C4: cost of “==” C5: cost of “return” C1 + (length+1)*C2+ length*C3 length*C4 0*C5 C5 Assuming C4 >> C1, C2, C3, C5 Worst case T(n) = n*C4 =  (n) Worst case T(n) = C1+C2+ C5 + n(C2+C3+C4)+ =  (n)

19. “in place” and “stable” in sorting algorithms AlgorithmWorst TimeExpected TimeExtra MemoryStable Insertion sortO(1) (in place)Can be Merge sortO(n)Can be Quick sortO(1) (in place)Can be Heap sortO(1) (in place)No 23313 5 23313 5 2 3 31 3 5 Stable Not stable

20. Design and Analysis of Algorithms Heapsort Haidong Xue Summer 2012, at GSU

21. Max-Heap A complete binary tree, and … Yes No every level is completely filled, except possibly the last, which is filled from left to right

22. Max-Heap Satisfy max-heap property: parent >= children 16 1410 8793 2 Since it is a complete tree, it can be put into an array without lose its structure information.

23. Max-Heap 16 1410 8793 2 123 4 5 6 78

24. Max-Heap Use an array as a heap 16 1410 8793 2 123 4 5 6 78 16 14 10 8 7932 1 23 456 7 8 For element at i: Parent index =parent(i)= floor(i/2); Left child index = left(i)=2*i; Right child index =right(i)=2*i +1 Last non-leaf node = floor(length/2) i=3 2*i = 6 2*i+1=7 floor(i/2)=floor(1.5)=1 floor(length/2)=4

25. Max-Heapify Input: A compete binary tree rooted at i, whose left and right sub trees are max-heaps; last node index Output: A max-heap rooted at i. Algorithm: 1. Choose the largest node among node i, left(i), right(i). 2. if(the largest node is not i){ – Exchange i with the largest node – Max-Heapify the affected subtree }

26. Max-Heapify Example 2 1610 14793 8

27. Heapsort for a heap Input: a heap A Output: a sorted array A Algorithm: 1. Last node index l = A’s last node index 2. From the last element to the second{ exchange (current, root); l--; Max-Heapify(A, root, l); }

28. Heapsort example 16 1410 87 9 3 2

29. Heapsort example 14 810 27 9 3 16

30. Heapsort example 10 89 27 3 14 16

31. Heapsort example 9 83 27 10 14 16

32. Heapsort example 8 73 29 10 14 16

33. Heapsort example 7 23 89 10 14 16

34. Heapsort example 3 27 89 10 14 16

35. Array -> Max-Heap Input: a array A Output: a Max-Heap A Algorithm: Considering A as a complete binary tree, from the last non-leaf node to the first one{ Max-Heapify(A, current index, last index); }

36. Build Heap Example 16 14 10 8 7 14 8 16 107

37. Heapsort Input: array A Output: sorted array A Algorithm: 1. Build-Max-Heap(A) 2. Last node index l = A’s last node index 3. From the last element to the second{ exchange (current, root); l--; Max-Heapify(A, root, l); } Let’s try it

38. Analysis of Heapsort Input: array A Output: sorted array A Algorithm: 1. Build-Max-Heap(A) 2. Last node index l = A’s last node index 3. From the last element to the second{ exchange (current, root); l--; Max-Heapify(A, root, l); } O(n) or O(nlgn) O(nlgn) O(n)

39. Design and Analysis of Algorithms Non-comparison sort (sorting in linear time) Haidong Xue Summer 2012, at GSU

40. Comparison based sorting Algorithms that determine sorted order based only on comparisons between the input elements AlgorithmWorst TimeExpected TimeExtra MemoryStable Insertion sortO(1) (in place)Can be Merge sortO(n)Can be Quick sortO(1) (in place)Can be Heap sortO(1) (in place)No What is the lower bound?

41. Lower bounds for comparison based sorting <? Y N Y …… ………………………….. Done N Y For n element array, how many possible inputs are there? Factorial of n ----- n! What is the shortest tree can have n! leaves? As a result ……

42. Sorting in linear time Can we sort an array in linear time? Yes, but not for free E.g. sort cards with 13 slots What if there are more than one elements in the same slot?

43. Counting Sort Input: array A[1, …, n]; k (elements in A have values from 1 to k) Output: sorted array A Algorithm: 1.Create a counter array C[1, …, k] 2.Create an auxiliary array B[1, …, n] 3.Scan A once, record element frequency in C 4.Calculate prefix sum in C 5.Scan A in the reverse order, copy each element to B at the correct position according to C. 6.Copy B to A

44. Counting Sort 25362373 A: C: 1 2 3 4 5 6 02 3 011 7 1 1 2 3 45 678 B: 1 2 3 45 678 2 5 5 76 8 Position indicator: 0 3 4 7 7 3 3 2 1 6 3 52 26 5 0

45. Analysis of Counting Sort Input: array A[1, …, n]; k (elements in A have values from 1 to k) Output: sorted array A Algorithm: 1.Create a counter array C[1, …, k] 2.Create an auxiliary array B[1, …, n] 3.Scan A once, record element frequency in C 4.Calculate prefix sum in C 5.Scan A in the reverse order, copy each element to B at the correct position according to C. 6.Copy B to A Time O(n) O(k) O(n) O(n+k)=O(n) (if k=O(n)) Space O(k) O(n) O(n+k)=O(n) (if k=O(n))

46. Radix-Sort Input: array A[1, …, n]; d (number of digit a element has) Output: sorted array A Algorithm: for each digit{ use a stable sort to sort A on a digit } T(n)=O(d(n+k))

47. Summary AlgorithmWorst TimeExpected TimeExtra MemoryStable Insertion sortO(1) (in place)Yes Merge sortO(n)Yes Quick sortO(1) (in place)Yes Heap sortO(1) (in place)No Counting sortYes Design strategies: Divide and conquer Employ certain special data structure Tradeoff between time and space

48. Knowledge tree Algorithms Analysis Design Algorithms for classic problems Classic data structure Asymptotic notations Probabilisti c analysis Sorting Shortest path Matrix multiplication … Divide & Conquer Greedy Dynamic Programming O(), o(),  (),  (),  () Heap, Hashing, Binary Tree, RBT, …. Quicksort, Heapsort, Mergesort, … ……… … … … ………… … … … … … ……… …