What is the runtime of the best possible (comparison based) sorting algorithm? 1.O(log n) 2.O(n) 3.O(n log n) 4.O(n 2 ) 5.None of the above

Comparison-Based Sorting is a: 1.o(n ) problem 2.  (n 2 ) problem 3.O(log n) problem 4.  (n log n) problem 5.None of the above

What are the limitations of using comparisons? What are the alternatives?

We know sorting is O(n log n) Proved by the existence of Merge Sort How do we how sorting is  (n log n)? Write out every possible algorithm and show that each is  (n log n)? That might take a while Better idea: Find a computation model that we can analyze

Any sorting algorithms is a series of decisions Start with list: i 1,i 2,i 3,…,i n Compare i j and i k –Value of j and k depend on sort –Swap if needed Compare i j’ and i k’ –Value of j’ and k’ depend on sort and previous answer –Swap if needed Continue…

i 1,i 2,i 3,i 4,…,i n i 1 < i 2 ? i 1,i 2,i 3,i 4,…,i n i 2 <i 3 ? Yes i 2,i 1,i 3,i 4,…,i n i 1 <i 3 ? No i 1,i 2,i 3,i 4,…,i n i 3 < i 4 i 1,i 3,i 2,i 4,…,i n i 2 <i 4 ? Yes No i 2,i 1,i 3,i 4,…,i n i 3 <i 4 ? i 2,i 3,i 1,i 4,…,i n i 2 <i 4 ? YesNo

A tree leaf indicates where we stopped asking questions The ordering of the leaf at which we end up must be the sorted order The depth of the leaf is determined by the number of comparisons leading to it

How many leaves? Every possible ordering of i 1,i 2,…,i n is a solution for some possible input

How many leaves must the tree have? 1.  (n 2 ) 2.  (2 n ) 3.  (n!) 4.  (n n ) 5.None of the above

How many leaves? Every possible ordering of i 1,i 2,…,i n is a solution for some possible input There are n! possible orderings Any decision tree has n! leaves

Best we can hope for is: Balanced tree n! leaves What is the height of a balanced tree? log 2 (number of leaves) Easy to prove from previous results

What is the height of the tree? 1.  (log n!) 2.  (log 2 n ) 3.  (log n 2 ) 4.  (log n) 5.None of the above

Our height/ runtime is log 2 n! Not useful -- how bad is this? Stirling’s Approximation: n! ~ (2  ) 1/2 n n+1/2 /e n Follows: log n! ~ c 1 + c 2 n log n - c 3 n =  (n log n)

Basic Idea: Look at the list Count the number of times each element appears in the list Reconstruct list from the count Notice: Only works on integers –Or sets that can be mapped to the integers For example: Letters (A=0, B=1, C=2, …) Never compares two elements -- previous proof is not applicable!!!

List to be sorted Array for all possible values

What is the runtime of Counting Sort? 1.O(log n) 2.O(n) 3.O(n log n) 4.O(n 2 ) 5.None of the above

Consider the steps: –Allocate memory for the counting array: O(1) Run through the list: O(n) Anything else? All elements of the array must initially be set to 0: O(M) –Where M is the largest element of the array Finding M: O(n) Transverse array at the end: O(M) Total: O(n+M)

Sorting Social Security Numbers O(n + 999,999,999)=O(n) Sorting by Birth Year O(n ) = O(n) Sorting n random numbers from (1,n 2 ) O(n + n 2 ) = O(n 2 ) Sorting n random numbers O(n +  ) = O(  )

As implemented: We cannot carry any information with the sorted values Input: (name, birth year) pairs Question: How do we sort names by birth year using counting sort?

Abbott, H.1980 Bell, K.1979 Bones, S.1980 Creevy, C.1981 Diggory, C.1978 Fawcett, S.1980 Jordan, L.1978 Patil, P.1980 Wood, O Wood, O.Diggory, C.Abbott, H.Creevy, C.Bell, K.Jordan, L.Bones, S.Fawcett, S.Patil, P.

Abbott, H.1980 Bell, K.1979 Bones, S.1980 Creevy, C.1981 Diggory, C.1978 Fawcett, S.1980 Jordan, L.1978 Patil, P.1980 Wood, O.1976 Notice: This is a stable sort.

Sort by digit Intuitively: Sort from most significant to least significant Then you would need to keep track of the “break” points -- irritating overhead Instead: Start with the least significant digit Sorting technique must be stable… …like counting sort

Radix Sort

Assume base 10 numbers (for now) Each step: Sort n numbers Range from 0 to 9 O(n+10)

How many steps are required for a m digit number? 1.O(log m) 2.O(m) 3.O(m log m) 4.O(m 2 ) 5.None of the above

Assume base 10 numbers (for now) Each step: Sort n numbers Range from 0 to 9 O(n+10) Number of steps: One per digit: log 10 M (M = max number) O((n + 10)log 10 M)=O(n log M)

Sorting binary numbers: O((n + 2)log 2 n) = O(n log M) Sorting hex numbers: O((n + 16)log 16 M) = O(n log M) Sorting English words O((n + 26) log 26 M) = O(n log M)

Sorting binary numbers: O((n + 2)log 2 n) = O(n log M) Sorting hex numbers: O((n + 16)log 16 M) = O(n log M) Sorting English words O((n + 26) log 26 M) = O(n log M) Sorting base b numbers O((n + b) log b M) = O(n log M + b log M)

Use: Numbers evenly distributed over the interval (0,1) Sort by tossing in buckets: Step 1: Create n buckets Step 2: Toss items into buckets Step 3: Sort each bucket with quicksort (or other)

Example [0.31, 0.70, 0.96, 0.23, 0.04, 0.56, 0.92, 0.89, 0.53, 0.32] [ ) [ ) [ ) [ ) [ ) [ ) [ ) [ ) [ ) [ )

Assumption: Numbers uniformly distributed There will not be too many numbers in any one bucket Sorting each bucket will be quick O(n) average runtime

Given a set of points on the plane, which points define the borders of the convex hull? Suppose each point was a pin sticking into the paper Suppose we surrounded them with a rubber band and let it contract Which points would it touch?

Sample Convex Hull Convex: We cannot draw a line that intersects the hull at more than two points.

Input: An (unordered) set of n points on the plane (given as coordinates) Output: A list of the points on the convex hull in counter-clockwise order Algorithms: Jarvis March: O(n 2 ) Graham’s Scan: O(n log n)

We now know the problem is O(n log n) Can we show that is it  (n log n)? –What other lower bounds do we know? –Can we shows that an o(n log n) CH algorithm would lead to an o(n log n) sorting algorithm?

How do we sort with a Convex Hull algorithm? –(Sorting) Input: A sequence of numbers i 1,i 2,…,i n –Translation: Create a CH input: (i 1,i 1 2 ), (i 2,i 2 2 ), …, (i n,i n 2 )

List: 4, 1, 9, 3, 2, 6, 8, 7 CH Input: (4,16), (1,1), (9,81), (3,9), (2,4), (6,36), (8,64), (7,49) Return (1,1) (2,4) (3,9) (4,16) (6,36) (7,49) (8,64) (9,81)

If CH takes f(n) time: –Translates sorting input to CH input O(n) –Run CH algorithm O(f(n)) –Process CH results O(n) –Runtime: O(n + f(n)) If CH = o(n log n) then we can sort in o(n log n) –We can’t sort in o(n log n) –Hence: CH is  (n log n)