1. Sorting

2. Input: A sequence of n numbers a 1, …, a n Output: A reordering a 1 ’, …, a n ’, such that a 1 ’ < … < a n ’ 14326816 23141668

3. Insertion Sort INSERTION-SORT(A) // sort the array A for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 14326816

4. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 14326816 j

5. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 31426816 j

6. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 31426816 j

7. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 38261416 j

8. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 38261416 j

9. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 38261416 j

10. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 23166814 j

11. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 23166814 j

12. Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key 23141668

13. Pseudo-Code Not really a program, just an outline Enough details to establish the correctness and running time Even pseudo-code is too complicated Note that for a simple algorithm it obscures what is going on A simpler version of Insertion Sort, using an auxiliary array:  Go over the numbers one-by-one, starting from the first, copy to new array  Each time copy to the correct place in the new array  In order to create empty space, shift the numbers that are larger than the current number one cell to the right Credits: Serge Plotkin

14. Analysis of an algorithm Correctness: given a legal input, the algorithm terminates and produces the desired output Running Time: depends on input size, input properties  Worst case: max T(n), on any input of size n  Expected: E(T(n)), where inputs are taken from a distribution  Best case: min T(n) – not really interesting, except to show that an algorithm works well on input with certain properties

15. Correctness of Insertion Sort INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Loop Invariant: At the start of each iteration of the for loop, A[1…j-1] consists of the original first j-1 elements, but sorted

16. Loop Invariants Help prove that an algorithm is correct To prove loop invariant, need to show three properties: Initialization: Invariant is true before 1 st iteration Maintenance: If invariant true before iteration k, it remains true right before iteration k+1 Termination: When the loop terminates, the invariant implies some useful property

17. Initialization INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Show that when j = 2, invariant holds “ A[1] consists of the original first 1 elements, but sorted” Clear: A[1…j-1] is just A[1]

18. Maintenance INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key Show that each iteration maintains the invariant Informally, the relative order of A[1…j-1] is not affected by the body of the loop; A[j] is inserted in its proper place.

19. Termination INSERTION-SORT(A) for j = 2 to length(A) key = A[j] i = j – 1 while i > 0 and A[i] > key A[i+1] = A[i] i-- A[i+1] = key What does the invariant imply at loop termination? A[1…length(A)] is sorted!

20. Running Time INSERTION-SORT(A) # times executed for j = 2 to length(A) n = length(A) key = A[j] n – 1 i = j – 1 n – 1 while i > 0 and A[i] > key  j=2…n t j A[i+1] = A[i]  j=2…n (t j – 1) i--  j=2…n (t j – 1) A[i+1] = key n – 1  Assume each operation costs 1  Let t j = # times while loop is executed T(n) = n + 3(n – 1) +  j=2…n t j + 2  j=2…n (t j – 1)

21. Running Time T(n) = n + 3(n – 1) +  j=2…n t j + 2  j=2…n (t j – 1) Best case: Input A is already sorted Then, t j = 1, for all j T(n) = 4n – 3 + (n-1) + 0 = 5n - 4 Worst case: Input A is in reverse order: A[1] > … > A[n] Then, t j = j, for all j T(n) = 4n – 3 + n(n+1)/2 + n(n-1) = 3/2 n 2 + 7/2 n – 3 23141668 143286

22. Merge Sort Divide A into two sub-arrays of half the size Recursively, sort each sub-array Merge the two sub-arrays into a sorted array  Merge by successively picking & erasing the smallest element from the beginning of the two sub-arrays

23. The divide-and-conquer approach Divide the problem into a number of subproblems Conquer the subproblems by solving recursively Combine the solutions to the sub-problems into the solution for the original problem

24. Merge Sort MERGE-SORT(A, p, r) if p < r then q =  (p+r)/2  MERGE-SORT(A, p, q) MERGE-SORT(A, q+1, r) MERGE(A, p, q, r) What is left is to define the MERGE subroutine DIVIDE CONQUER COMBINE

25. Merge Sort MERGE(A, p, q, r) create arrays L[1…q-p+1] and R[1…r-q] L[1…q-p] = A[p…q] R[1…r-q-1] = A[q+1…r] L[n 1 +1] = R[n 1 +1] =  i = j = 1 For k = p to r if L[i] < R[i] then A[k] = L[i]; i++ else A[k] = R[j]; j++ 38121416610 38141612610 q pr 128 361416

26. Merge Sort – Example 1432681611014381626110261 143816 26110143816 1102681631438 1612610128 361416