Insertion Sort Introduction to Algorithms Insertion Sort CSE 680 Prof. Roger Crawfis

Insertion Sort Overview How do we know where to place the next card? What assumptions do we make at each step? What is the algorithm?

Insertion Sort Algorithm Given: a set U of unsorted elements Algorithm: 1. Foreach element e in U 2. remove e from U 3. place e in the sorted sequence S at the correct location. Step 3 needs some refinement. What is it’s Problem Statement? What are the Data Structures?

Inserting an Element Any ideas on how to insert element e into our sorted sequence S?

Insertion Sort Algorithm Your book’s pseudo-code: What are the differences? Why go backwards?

Insertion Sort Example The following slides are from David Luebke when he taught the course at the University of Virginia: Thanks David! Unfortunately, he flipped the indices i and j from the book.

David Luebke 7 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

David Luebke 8 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i =  j =  key =  A[j] =  A[j+1] = 

David Luebke 9 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 1key = 10 A[j] = 30 A[j+1] = 10

David Luebke 10 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 1key = 10 A[j] = 30 A[j+1] = 30

David Luebke 11 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 1key = 10 A[j] = 30 A[j+1] = 30

David Luebke 12 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 0key = 10 A[j] =  A[j+1] = 30

David Luebke 13 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 0key = 10 A[j] =  A[j+1] = 30

David Luebke 14 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 2j = 0key = 10 A[j] =  A[j+1] = 10

David Luebke 15 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 0key = 10 A[j] =  A[j+1] = 10

David Luebke 16 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 0key = 40 A[j] =  A[j+1] = 10

David Luebke 17 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 0key = 40 A[j] =  A[j+1] = 10

David Luebke 18 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40

David Luebke 19 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40

David Luebke 20 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40

David Luebke 21 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 40 A[j] = 30 A[j+1] = 40

David Luebke 22 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40

David Luebke 23 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40

David Luebke 24 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 3key = 20 A[j] = 40 A[j+1] = 20

David Luebke 25 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 3key = 20 A[j] = 40 A[j+1] = 20

David Luebke 26 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40

David Luebke 27 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40

David Luebke 28 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40

David Luebke 29 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40

David Luebke 30 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40

David Luebke 31 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 30

David Luebke 32 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 2key = 20 A[j] = 30 A[j+1] = 30

David Luebke 33 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 1key = 20 A[j] = 10 A[j+1] = 30

David Luebke 34 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 1key = 20 A[j] = 10 A[j+1] = 30

David Luebke 35 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 1key = 20 A[j] = 10 A[j+1] = 20

David Luebke 36 1/15/2015 An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } i = 4j = 1key = 20 A[j] = 10 A[j+1] = 20 Done!

Memory Analysis The algorithm is said to be in-place if it uses only a constant amount of memory in accomplishing its solution. Clearly, the book’s implementation is better than mine. Clearly, the book’s algorithm is better than mine. That is, do we consider my implementation to be a different algorithm or the same?

Correctness We can use a property known as loop-invariance to reason about the correctness of the algorithm. For Insertion-Sort, we can say that: The subarray A[1..i-1] elements are in sorted order A[1..i-1] is the set of numbers from the original A[1..i-1]

Insertion Sort Example

Loop Invariant Initialization True at the beginning of the loop. Termination The loop terminates. True, when the loop exists. Maintenance True at the end of each iteration of the loop. This allows us to prove the invariant similarly to proof-by-induction.

David Luebke 41 1/15/2015 Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } What is the precondition for this loop?

Bubble-Sort Can you come up with loop invariants? Aka, can you prove it is correct? How expensive is it? When is it most expensive? When is it least expensive?

Selection Sort What about selection sort? void selectionSort(int[] a) { for (int i = 0; i < a.length - 1; i++) { int min = i; for (int j = i + 1; j < a.length; j++) { if (a[j] < a[min]) { min = j; } if (i != min) { int swap = a[i]; a[i] = a[min]; a[min] = swap; }