1. Linear Sorting
Sorting in O(n)
Jeff Chastine

2. Previously
Previous sorts had the same property: they are based only on comparisons
Any comparison-based sorting algorithm must run in Ω(n log n)
Thus, MERGESORT and HEAPSORT are asymptotically optimal
Obviously, this lower bound does not apply to linear sorting
Jeff Chastine

3. Lower Bound for Sorting
Assume (for simplicity) that all elements in an input sequence a1, a2, ..., an are distinct
We can view comparison-based sorting using a decision tree
Any correct sorting algorithm must be able to handle any of the n! permutations
Each permutation must appear as a leaf in the tree
Jeff Chastine

4. The Decision Tree a1:a2 ≤ a2:a3 a1:a3 ≤ ≤ a1:a3 a2:a3 1,2,3 2,1,3
>
a2:a3
a1:a3 ≤
> ≤
>
a1:a3
a2:a3
1,2,3
2,1,3 ≤ ≤
>
>
1,3,2
3,1,2
2,3,1
3,2,1
a1 = 6, a2 = 8, a3 = 5
Jeff Chastine

5. Proof Consider a decision tree with height h with l reachable leaves
n!  l
Since a binary tree of height h has no more than 2h leaves:
n!  l  2h
Taking the log of both sides:
h  lg (n!) = (n lg n)
Jeff Chastine

6. Counting Sort Each element is an integer in the range from 1 to k
For each element, determine the number of elements less than it
Works well on small ranges
Does not sort in place
Jeff Chastine

7. How do you construct C's size?1 2 3 4 5 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 C 2 2 3 1
Number of 1's, 2's..
How do you construct C's size?
How do you fill C's data?
Jeff Chastine

8. Modify C Such that C now tells how many elements are less than i1 2 3 4 5 6 C 2 2 3 1
Number of 1's, 2's.. 1 2 3 4 5 6 C' 2 2 4 7 7 8
Number of slots
How do you construct C'?
Jeff Chastine

9. Move into B from A A 3 6 4 1 3 4 1 4 Original B Sorted C 2 2 4 7 7 85 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B
Sorted 1 2 3 4 5 6 C 2 2 4 7 7 8
Number of slots.
Jeff Chastine

10. Move into B from A A 3 6 4 1 3 4 1 4 Original B 4 Sorted C 2 2 4 6 7 85 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 4
Sorted 1 2 3 4 5 6 C 2 2 4 6 7 8
Number of slots.
Jeff Chastine

11. Move into B from A A 3 6 4 1 3 4 1 4 Original B 4 Sorted C 2 2 4 6 7 85 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 4
Sorted 1 2 3 4 5 6 C 2 2 4 6 7 8
Number of slots.
Jeff Chastine

12. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 4 Sorted C 1 2 4 6 75 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 4
Sorted 1 2 3 4 5 6 C 1 2 4 6 7 8
Number of slots.
Jeff Chastine

13. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 4 Sorted C 1 2 4 6 75 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 4
Sorted 1 2 3 4 5 6 C 1 2 4 6 7 8
Number of slots.
Jeff Chastine

14. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 4 4 Sorted C 1 2 4 57 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 4 4
Sorted 1 2 3 4 5 6 C 1 2 4 5 7 8
Number of slots.
Jeff Chastine

15. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 4 4 Sorted C 1 2 4 57 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 4 4
Sorted 1 2 3 4 5 6 C 1 2 4 5 7 8
Number of slots.
Jeff Chastine

16. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 3 4 4 Sorted C 1 2 35 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 3 4 4
Sorted 1 2 3 4 5 6 C 1 2 3 5 7 8
Number of slots.
Jeff Chastine

17. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 3 4 4 Sorted C 1 2 35 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 3 4 4
Sorted 1 2 3 4 5 6 C 1 2 3 5 7 8
Number of slots.
Jeff Chastine

18. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 Sorted C 2 35 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4
Sorted 1 2 3 4 5 6 C 2 3 5 7 8
Number of slots.
Jeff Chastine

19. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 Sorted C 2 35 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4
Sorted 1 2 3 4 5 6 C 2 3 5 7 8
Number of slots.
Jeff Chastine

20. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 4 Sorted C 25 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4 4
Sorted 1 2 3 4 5 6 C 2 3 4 7 8
Number of slots.
Jeff Chastine

21. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 4 Sorted C 25 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4 4
Sorted 1 2 3 4 5 6 C 2 3 4 7 8
Number of slots.
Jeff Chastine

22. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 4 6 Sorted C2 3 4 5 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4 4 6
Sorted 1 2 3 4 5 6 C 2 3 4 7 7
Number of slots.
Jeff Chastine

23. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 4 4 4 6 Sorted C2 3 4 5 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 4 4 4 6
Sorted 1 2 3 4 5 6 C 2 3 4 7 7
Number of slots.
Jeff Chastine

24. Move into B from A A 3 6 4 1 3 4 1 4 Original B 1 1 3 3 4 4 4 6 Sorted2 3 4 5 6 7 8 A 3 6 4 1 3 4 1 4
Original 1 2 3 4 5 6 7 8 B 1 1 3 3 4 4 4 6
Sorted 1 2 3 4 5 6 C 2 2 4 7 7
Number of slots.
Jeff Chastine

25. The Code COUNTING-SORT (A, B, k) for i ←1 to k // Init C
do C[i] ←0 // (k)
for j ←1 to length[A] // Build C
do C[A[j]] ← C[A[j]] + 1 // (n)
for i ← 2 to k // Make C'
do C[i] ← C[i] + C[i-1] // (k)
for j ← length[A] downto 1 // Copy info
do B[C[A[j]]] ← A[j] // (n)
C[A[j]] ← C[A[j]] -1
Jeff Chastine

26. Stable Sorting
An important property (for later) is that counting sort is stable:
numbers with the same value appear in the output array in same order they did in the input array
This is important for our next sorting algorithm: radix sort
Jeff Chastine

27. Radix Sort Sorting by "column" A d-digit number would create d columns
Start with least-significant row
Usually requires counting sort to be used on the columns
Jeff Chastine

28. Example (by hand) H H H F F F 331 429 190 127 982 784 318 190 331 982
Jeff Chastine

29. The Code RADIX-SORT (A, d) for i ← 1 to d
do use a stable sort to sort A on digit i
Have we created d more times work than counting sort?
If so, why do we do this?
Jeff Chastine

30. Analysis Each digit is in the range 0 – (k-1)
Takes k time to construct C
Each pass over n d-digit numbers takes (n+k)
Thus, the total running time is (d(n+k))
Jeff Chastine

31. Bucket Sort Once again, not any greater than counting sort
Assumes uniform distribution of random numbers [0, 1)
"Chunk" numbers into equal-sized buckets, based on first digit
Sort the buckets (with what?)
Jeff Chastine

32. Example →
.06 / 1
.06 / 2 →
.43
.37 3
.48 →
.43 →
.44 →
.48 4
.37 / 5
.91 / 6
.44 / 7 /
.98 8 → 9
.98
Jeff Chastine

33. The Code
BUCKET-SORT (A) n ← length[A] for i ← 1 to n do insert A[i] into list B[A[i]] for i ← 0 to n - 1 do sort list B[i] with insertion sort concatenate the lists together
Jeff Chastine

34. Proof of Bucket Sort Consider two elements A[i] and A[j]
Assume A[i]  A[j]
Then, A[i] is placed into either the same bucket, or a bucket with a lower index.
The sort of each bucket guarantees A[i] and A[j] are ordered correctly
Jeff Chastine

35. Of Interest (only to me)
“As long as the input has the property that the sum of the squares of the bucket sizes is linear in the total number of elements”... “bucket sort will run in linear time”
In other words, each bucket should get the square root of the number of bucket elements.
Jeff Chastine