HKOI 2006 Intermediate Training Searching and Sorting 1/4/2006

Sorting - Introduction Sorting is the ordering of your data in a consistent manner. Each element is usually part of a collection of data called a record. Each record contains a key, which is the value to be sorted

Sorting - Introduction Ordering--We require the existence of the “ ” operators, which can be used to place a consistent ordering on the input. Eg. Sort the elements: Dog, Cat, Mouse, Tiger, Turtoise

Sorting - Introduction There are several easy algorithms to sort in O(n 2 ) There are slightly more complicated O(n log n) sorting algorithms. Any binary comparison based sorting algorithm has complexity of at least O(n log n)

Sorting - Introduction Different Sorting Methods: Bubble Sort Insertion Sort Selection Sort Quick Sort Heap Sort Shell Sort Merge Sort Radix Sort

Bubble Sort This is probably the simplest way sort an array of objects. The basic idea is to compare two neighboring objects, and to swap them if they are in the wrong order In each pass, the largest key in the list will be bubbled to the end, but the earlier keys may still be out of order.

Bubble Sort Example Original # of swaps After p = 1, After p = 2, After p = 3, After p = 4, After p = 5, After p = 6,

Bubble Sort Algorithm: for i = 1 to N-1 do for j = 1 to N-i do if A[j].key > A[j+1].key then swap(A[j],A[j+1])

Bubble Sort Improved version: for i = 1 to N-1 do begin for j = 1 to N-i do if A[j].key > A[j+1].key then swap(A[j],A[j+1]) if no swapping was done, then stop here end

Bubble Sort Complexity? O(n 2 ) If the file is initially sorted in the reverse order, the total number of comparisons is (n-1) + (n-2) = (n-1) * n/2

Bubble Sort Advantage: It requires little additional space (one additional record to swap and few simple integer variables to process) It is O(n) in the case that the file is completely sorted (or almost completely sorted)

Insertion Sort One of the simplest methods An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence

Insertion Sort Example Original # of Moves After p = 1, After p = 2, After p = 3, After p = 4, After p = 5,

Insertion Sort Algorithm for p = 2 to N temp = A[p]; for j = p downto 2 if A[j-1] > temp A[j] = A[j-1]; j = j – 1; A[j] = temp;

Insertion Sort Complexity? O(n 2 )

Insertion Sort Advantages: Similar to those mentioned in Bubble Sort

Additional Information on Insertion Sort: Improvement: using a binary search. This reduces the total number of comparisons from O(n 2 ) to O(n log n). However, the moving operation still requires O(n 2 ) time. So the binary search does not significantly improves the overall time bound. Another improvement is to use the linked-list insertion. This reduces the time required for insertion but not the time for searching for the proper position.

Selection Sort It is also called Straight Selection or Push- Down Sort. A selection sort is one in which successive elements are selected in order and placed into their proper sorted positions. The elements of the input may have to be preprocessed to make the ordered selection possible.

Selection Sort Example Original # of swaps After p = 1, After p = 2, After p = 3, After p = 4, After p = 5,

Selection Sort Algorithm: for i = 1 to N-1 lowindex = i; lowkey = A[i]; for j = i+1 to N if A[j] < lowkey lowindex = j; lowkey = A[j] swap (A[i], A[lowindex]);

Selection Sort Complexity? O(n 2 ) Advantage?

Stability A stable sorting algorithm is any sorting algorithm that preserves the relative ordering of items with equal values Before: (a,3),(b,5),(c,2),(d,5),(e,4) After: (c,2),(a,3),(e,4),(b,5),(d,5)

Stability Which of the discussed sorting methods are stable? It depends on your implementation

Merge Sort Merge sort is good for external sorting (data are stored in disks or tapes) The basic operation of merge sort is to merge 2 sorted lists Technique of divide-and-conquer

Merge Sort Merge 2 sorted list: Aptr Bptr Cptr

Merge Sort Array A is exhausted! Copy remainder of array B

Merge Sort Merge sort follows the divide-and-conquer approach Divide: Divide the n-element sequence into two (n/2)-element subsequences Conquer: Sort the two subsequences recursively Combine: Merge the two sorted subsequence to produce the answer

Merge Sort Complexity? O(n log n)

Quick Sort Divide-and-conquer process for sorting an array A[p..r] Divide: A[p..r] is partitioned into two nonempty subarrays A[p..q] and A[q+1..r] such that each element of A[p..q] is less than each element of A[q+1..r] Conquer: The two subarrays A[p..q] and A[q+1..r] are sorted by recursive calls to quicksort. Combine: Since the subarrays are sorted in place, no work is needed to combine them

Quick Sort Quicksort(S) { if size of S is 0 or 1 then return; pick a pivot v divide S - {v} into S 1 and S 2 such that every element in S 1 is <= v every element in S 2 is >= v return { Quicksort(S 1 ) v Quicksort(S 2 ) } }

Quick Sort Questions: How to pick a pivot? How to divide S into S 1 and S 2 ? S v S1S1 S2S2 v

Quick Sort How to pick a pivot? First element in S For example: Median-of-Three median of first, center and last element For example: Pivot = median of 42, 6 and 1 = 6 Pivot = 42

Quick Sort How to divide S into S 1 and S 2 ? Based on the value of pivot, we put elements that are <= pivot to the left of pivot elements that are >= pivot to the right of pivot For example, On the left of pivot: 10, 11, 9, 15 are <= 25 On the right of pivot: 31, 43, 62, 81 are >= Pivot

Quick Sort One of the methods Steps: (1) Swap the first, center and last element in S such that first element <= center element <= last element (2) The pivot v is the center element, swap it with the element before the last element (3) Make two pointers i, j to point to first element and the element before the last elements respectively (4) Move i pointer in right direction to point to a number >= pivot v (5) Move j pointer in left direction to point to a number <= pivot v (6) If i pointer is right of j pointer, goto step (7), else swap the elements pointed by i and j and goto step (4) (7) Swap the element pointed to i with the pivot v (the element before the last element in S)

Quick Sort Example: Step 1: Swap the first, center and last element in S such that first element <= center element <= last element

Quick Sort Example: Step 1: After swapping. Therefore the pivot is 8

Quick Sort Example: Step 2: Swap the pivot with the element before the last element.

Quick Sort Example: Step 3: Make two pointers i, j to point to first element and the element just before the last elements respectively ij

Quick Sort Example: Step 4: Move i pointer in right direction to point to a number >= pivot v ij

Quick Sort Example: Step 5: Move j pointer in left direction to point to a number <= pivot v ij

Quick Sort Example: Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) ij

Quick Sort Example: Step 4: Move i pointer in right direction to point to a number >= pivot v i j

Quick Sort Example: Step 5: Move j pointer in left direction to point to a number <= pivot v i j

Quick Sort Example: Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) i j

Quick Sort Example: Step 4: Move i pointer in right direction to point to a number >= pivot v i j

Quick Sort Example: Step 5: Move j pointer in left direction to point to a number <= pivot v ij

Quick Sort Example: Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) ij

Quick Sort Example: Step 4: Move i pointer in right direction to point to a number >= pivot v ij

Quick Sort Example: Step 5: Move j pointer in left direction to point to a number <= pivot v ij

Quick Sort Example: Step 6: As i pointer is right of j pointer, goto step (7) ij

Quick Sort Example: Step 7: Swap the element pointed to i with the pivot (the element before the last element in S) ij

Quick Sort Example: Done!!! You can see that all elements in S 1 is = pivot v Pivot v S1S1 S2S2

Quick Sort Complexity? Best: O(n log n) Worst: O(n 2 ) Average: O(n log n) Not a stable sort

Bucket Sort Assume 0<A[i]<=M Algorithm: initialize an array Count of size M to all 0's for i=1 to N inc(Count[A[i]]) Scan the Count array, and printout the sorted list

Bucket Sort Complexity: O(M+N) It can be used for small integers and limited by M

Radix Sort It is also called card sort. Algorithm: initialize an array of 10 buckets to empty for i=1 to N read A[i] and place it into the bucket the last digit Use the same process to sort the second last digit repeat until the first digit

Radix Sort Sort into buckets on the least significant digit first, then proceed to the most significant digit Initial: 64, 8, 216, 512, 27, 729, 0, 1, 343, By the least significant digit: After First Pass: 0, 1, 512, 343, 64, 125, 216, 27, 8, 729

Radix Sort 1st Pass Result: 0, 1, 512, 343, 64, 125, 216, 27, 8, By the tens digit: After Second Pass: 0, 1, 8, 512, 216, 125, 27, 729, 343, 64 Second Pass:

Radix Sort 2nd Pass Result: 0, 1, 8, 512, 216, 125, 27, 729, 343, By the most significant digit: 0, 1, 8, 27, 64, 125, 216, 343, 512, 729 Third Pass: After Last Pass:

Radix Sort Complexity: O(dn), d is the number of digits Stable sort