1. COMP 103 SORTING Lindsay Groves 2016-T2 Lecture 26
Marcus Frean, Thomas Khuene, Lindsay Groves
Lindsay Groves
School of Engineering and Computer Science, Victoria University of Wellington
2016-T2 Lecture 26

2. RECAP-TODAY 2
RECAP
Algorithms and costs for various set/bag/map implementations
Binary Search, BST and heap have O(log n) cost
Made possible by ordering properties
TODAY
Introduction to sorting
Two algorithms for sorting a list:
Selection Sort
Insertion Sort
Announcements:

3. Why Sort? May want to present information in different ways3
May want to present information in different ways
Eg list telephone subscribers by name, phone number or address.
We can find items in an array much faster if they are sorted
finding1-in-a-billion takes as long as the ArraySet takes to find1-in-30
But... constructing a sorted array one item at a time is very slow : O(n2)
for 10 million items you need  50,000,000,000,000 steps at 10nS per step ⇒ 500,000 seconds = 58 days .
There are sorting algorithms that are much faster if you sort the whole array at once: O(n log n)
for 10 million items you need  250,000,000 steps ⇒ 2.5 seconds

4. Why study sorting algorithms?4
Important practical problem – default utility isn’t always the best option.
Illustrates:
Important algorithm design strategies
Interesting uses of data structures
Algorithm analysis techniques
Introduces important algorithms that every computer science or software engineering student should know

5. Ways of sorting Selection-based sorts: Insertion-based sorts:5
Selection-based sorts:
find the next largest/smallest item and put in place
build the correct list in order incrementally
Insertion-based sorts:
for each item, insert it into an ordered sublist
build a sorted list, but keep changing it
Compare-and-Swap-based sorts:
find two items that are out of order, and swap them
keep “improving” the list

6. Ways to rate sorting algorithms6
Efficiency
What is the (worst-case) order of the algorithm?
How does the algorithm deal with border cases?
Requirements on Data
Does the algorithm need random-access to data?
Does it need anything more than “compare” and “swap”?
Space Usage
Can the algorithm sort in-place, or does it need extra space?
Stability
Will the algorithm ever reverse the order of “equal” items?
Important when sorting records/objects with multiple fields;
E.g. employee name, address, phone number, employee number, …

7. Selection-based Sorts7
search for minimum here
Selection Sort (slow)
HeapSort (fast) 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

8. Insertion-based Sorts8
Insertion Sort (slow)
Shell Sort (pretty fast)
Merge Sort (fast) (Divide and Conquer) 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

9. Compare and Swap Sorts Bubble Sort (easy but terrible performance)9
Bubble Sort (easy but terrible performance)
QuickSort (very fast) (Divide and Conquer)
...and so on... 1 2 3 4 5 6 7 8 9 10 11
things bubble up quickly,
but bubble down slowly

10. Other Sorts 10
Radix Sort (only works with certain data types) (eg sort names on 1st character, then 2nd, …)
Permutation Sort (very slow)
Random Sort (Generate and Test) 1 2 3 4 5 6 7 8 9 10 11
Radix Sort requires data that can be mapped to integers as it sorts by grouping keys by the individual digits which share the same significant position and value

11. Coding it: some design decisions...11
We could sort Lists ⇒ general and flexible
but efficiency depends on how the List is implemented
or just sort Arrays ⇒ less general
efficiency is well-defined
NB: method toArray() converts any Collection to an array
May sort the actual array (or file), or an index into it
We could require items to be Comparable
i.e. any item can call compareTo(otherObj) on another ("natural ordering")
OR provide a Comparator
i.e. the sorter can call compare(obj1, obj2) on any two items.
We will sort an Array, using a Comparator:
public void …Sort(E[] data, int size, Comparator<E> comp)
array
number of items
comparator

12. Selection Sort Find the smallest item and move it to position 012
Find the smallest item and move it to position 0
Find the next smallest item and move it to position 1 …
What do we to with the item that was there?
Move all the items before it up one (easy for linked list)
Swap with the item to be moved
cost (comparisons)
= (n-1) + (n-2) + (n-3) + …
= n(n-1)/2 = O(n2)
= best, worst, and average - always does the same comparisons and swaps. 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

13. Selection Sort 13
public void selectionSort(E[ ] data, int size, Comparator<E> comp) {
// for each position, from 0 up, find the next smallest item
// and swap it into place
for (int i=0; i<size-1; i++) {
int minPos = i;
for (int j=i+1; j<size; j++)
if (comp.compare(data[j], data[minPos]) < 0)
minPos=j;
swap(data, i, minPos); }
cost (comparisons)
= (n-1) + (n-2) + (n-3) + …
= n(n-1)/2 = O(n2)
= best, worst, and average - always does the same comparisons and swaps. 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

14. Efficiency of Selection Sort14
Cost:
step?
best case:
what is it?
cost:
worst case:
average case:
all cases the same: have to scan all remaining items every time (n-1)+(n-2)+(n-3)+…1 = n(n-1)/2 = O(n2)

15. Selection Sort Analysis15
Efficiency
worst-case: O(n2)
average case exactly the same.
Requirements on Data
Needs random-access data, but easy to modify for files
Needs compare and swap
Space Usage
in-place
Stability ??
Performance on Nearly Sorted Data
No faster when given nearly sorted data

16. Insertion Sort 16
Place first item into the output list (bottom part of the array)
Add next item to the output list so that it is still sorted. …
How do we add each item to the output list?
Find required position and move others up to make a space (easy with link list)
Swap with the item to the left as long as it is larger
Move items up one as long as they are larder, then insert the new item 1 2 3 4 5 6 7 8 9 10 11

17. Insertion Sort for (int i=1; i<size; i++) { E item = data[i];17
public void insertionSort(E[] data, int size, Comparator<E> comp) {
// for each item, from 0, insert into place in the sorted region (0..i-1)
for (int i=1; i<size; i++) {
E item = data[i];
int place = i;
while (place > 0 && comp.compare(item, data[place-1]) < 0) { 
data[place] = data[place-1];
place--; }
data[place]= item; 1 2 3 4 5 6 7 8 9 10 11

18. Efficiency of Insertion Sort18
first: what is the step?
best case:
what is it?
cost:
worst case:
average case:
almost sorted cost:
best case: already sorted: don’t have to move anything, just n-1 comparisons. O(n)
worst case: reverse sorted: have to move everything down to the bottom: …n-1 = n(n-1)/2 = O(n2)
Averace case: random order: have to move half way down on average: n(n-1)/4 = O(n2), but twice as fast as selection or bubble sort
almost sorted case: only have to move a few items into place. or only move items a short way: O(n)
ie, Insertion sort is very good when you know the list is almost sorted already.

19. Insertion Sort Analysis19
Efficiency
worst-case: O(n2)
average case: half the cost of worst case.
Requirements on Data
Needs random-access data
Needs compare and swap
Space Usage
in-place
Stability ??
Performance on Nearly Sorted Data
Very fast (O(n)) when given nearly sorted data

20. BubbleSort 20
Step through the array comparing successive pairs of items and swapping if they are out of order.
Continue till no pairs are swapped.
Can improve by:
Recording where the last swap occurred
Bubbling up and down in alternate passes (shaker sort)

21. Can we do better? 21
InsertionSort, SelectionSort and BubbleSort are all slow (apart from Insertion on nearly-sorted lists)
Problems
Selection Sort
compares every pair of items, i.e., it ignores the results of previous comparisons.
Insertion Sort (and Bubble Sort)
only compares adjacent items
only moves items one step at a time
Can we do better? How? How much better?