CS200: Algorithm Analysis

SORTING IN LINEAR TIME Non-comparison Sorts Counting Sort : no comparisons, but algorithm depends on assumption about numbers to be sorted. Numbers to be sorted must be in range from 1 …k, k <= n. REMEMBER THIS!

θ(k) for i=1 to k do {init aux.storage} C[i] = 0 Input: A[1 … n] ,A[i] is in [1,2,...,k], for i in [1 … n]. Output: B[1 … n] in sorted order. Uses: C[1 … k] auxillary storage. Counting Sort(A,B,k) θ(k) for i=1 to k do {init aux.storage} C[i] = 0 θ(n) for i=1 to n do {C[i]=#elements = i} C[A[i]] = C[A[i]] + 1 θ(k) for i=2 to k do {C[i]=#elements ≤i} C[i] = C[i] + C[i-1] θ(n) for i=n down to 1 do{Sort into B.Update C} B[C[A[i]]] = A[i] C[A[i]] = C[A[i]] - 1

Do trace of algorithm using Counting Sort Animation at https://www.cs.usfca.edu/~galles/visualization/ CountingSort.html Runtime analysis of algorithm is ? 􏰀θ (n + k), which is 􏰀(n) if k = O(n). Note that constants are quite large for this algorithm. Is this a stable sort? (keys with same value appear in same order in output as they did in input) because of how the last loop works.

Practicality of Counting Sort How big a k is practical? Good for 32-bit values? 16-bit? 8-bit? 4-bit? No. Probably not. Maybe, , depending on n. Probably (unless n is really small).

RADIX SORT Motivation for the importance of algorithm design. How did IBM get rich originally? By designing a punched card reader for census tabulations in the early 1900’s. This is the card sorting machine described in section 8.3 of text.

Card Reader Each card has 80 cols. Each col. has 12 places for punched holes.The reader examines a col. on a card and places it into one of 12 bins depending on where hole is punched. Operator collects cards from bins and orders them so that cards with 1st hole punched is on top of cards with 2nd hole punched, etc. Machine sorts one col. at a time. Algorithm for using machine makes multi-col. sorting possible. The human operator was part of the algorithm!

Show example trace of algorithm to get intuition of execution, next slide. Correctness: Inductive Sketch Induction is on # of passes of algorithm. Assume lower order digits are sorted. Show that sorting on next digit leaves lower digits sorted (using part of example trace).

1. If two digits in higher order position are different, ordering the numbers by that position is correct because lower order digits are irrelevant. 2. If two digits in higher order position are equal then the numbers are already in sorted order, since the numbers are already sorted on lower order digits. Correctness depends on a stable sort, one that maintains order of elements in each pass over a col. of digits. The stable sort that is used is the Counting Sort; used to sort each col., if digits are in range from 1..k and k is not too large.

Analysis Each pass over n, d-digit numbers takes Q(n+k) time. There are d passes of the algorithm => Q(dn+dk) time When d is constant and k = O(n) then Radix sort takes θ (n) time.

SKIP

Radix sort: ⌈32/20⌉ = 2 passes. SKIP So, to sort 216 32-bit numbers, use r = lg 216 = 16 bits. ⌈b/r⌉ = 2 passes. Compare radix sort to merge sort and quicksort for 1 million (220) 32-bit integers. Radix sort: ⌈32/20⌉ = 2 passes. Merge sort/quicksort: lg n = 20 passes. Remember, though, that each radix sort “pass” is really 2 passes— one to take census, and one to move data. How does radix sort violate the ground rules for a comparison sort? Using counting sort allows us to gain information about keys by means other than directly comparing 2 keys. Uses keys as array indices.

Summary Counting Sort algorithm and analysis Stable Sorts Radix Sort