Sorting

Input: A sequence of n numbers a 1, …, a n Output: A reordering a 1 ’, …, a n ’, such that a 1 ’ < … < a n ’

Insertion Sort INSERTION-SORT(A) // sort the array A for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key j

Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key

Pseudo-Code Not really a program, just an outline Enough details to establish the correctness and running time Even pseudo-code is too complicated Note that for a simple algorithm it obscures what is going on A simpler version of Insertion Sort, using an auxiliary array:  Go over the numbers one-by-one, starting from the first, copy to new array  Each time copy to the correct place in the new array  In order to create empty space, shift the numbers that are larger than the current number one cell to the right Credits: Serge Plotkin

Analysis of an algorithm Correctness: given a legal input, the algorithm terminates and produces the desired output Running Time: depends on input size, input properties  Worst case: max T(n), on any input of size n  Expected: E(T(n)), where inputs are taken from a distribution  Best case: min T(n) – not really interesting, except to show that an algorithm works well on input with certain properties

Correctness of Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Loop Invariant: At the start of each iteration of the for loop, A[1…j-1] consists of the original first j-1 elements, but sorted

Loop Invariants Help prove that an algorithm is correct To prove loop invariant, need to show three properties: Initialization: Invariant is true before 1 st iteration Maintenance: If invariant true before iteration k, it remains true right before iteration k+1 Termination: When the loop terminates, the invariant implies some useful property

Initialization INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Show that when j = 2, invariant holds “ A[1] consists of the original first 1 elements, but sorted” Clear: A[1…j-1] is just A[1]

Maintenance INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Show that each iteration maintains the invariant Informally, the relative order of A[1…j-1] is not affected by the body of the loop; A[j] is inserted in its proper place.

Termination INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key What does the invariant imply at loop termination? A[1…length(A)] is sorted!

Running Time INSERTION-SORT(A) # times executed for j = 2 to length(A) n = length(A) key = A[j] n – 1 i = j – 1 n – 1 while i > 0 and A[i] > key  j=2…n t j A[i+1] = A[i]  j=2…n (t j – 1) i--  j=2…n (t j – 1) A[i+1] = key n – 1  Assume each operation costs 1  Let t j = # times while loop is executed T(n) = n + 3(n – 1) +  j=2…n t j + 2  j=2…n (t j – 1)

Running Time T(n) = n + 3(n – 1) +  j=2…n t j + 2  j=2…n (t j – 1) Best case: Input A is already sorted Then, t j = 1, for all j T(n) = 4n – 3 + (n-1) + 0 = 5n - 4 Worst case: Input A is in reverse order: A[1] > … > A[n] Then, t j = j, for all j T(n) = 4n – 3 + n(n+1)/2 + n(n-1) = 3/2 n 2 + 7/2 n –

Merge Sort Divide A into two sub-arrays of half the size Recursively, sort each sub-array Merge the two sub-arrays into a sorted array  Merge by successively picking & erasing the smallest element from the beginning of the two sub-arrays

The divide-and-conquer approach Divide the problem into a number of subproblems Conquer the subproblems by solving recursively Combine the solutions to the sub-problems into the solution for the original problem

Merge Sort MERGE-SORT(A, p, r) if p < r then q =  (p+r)/2  MERGE-SORT(A, p, q) MERGE-SORT(A, q+1, r) MERGE(A, p, q, r) What is left is to define the MERGE subroutine DIVIDE CONQUER COMBINE

Merge Sort MERGE(A, p, q, r) create arrays L[1…q-p+1] and R[1…r-q] L[1…q-p] = A[p…q] R[1…r-q-1] = A[q+1…r] L[n 1 +1] = R[n 1 +1] =  i = j = 1 For k = p to r if L[i] < R[i] then A[k] = L[i]; i++ else A[k] = R[j]; j q pr

Merge Sort – Example