26 Sep 2014Lecture 3 1

Last lecture: Experimental observation & prediction Cost models: Counting the number of executions of Every single kind of command Only some important commands (e.g., array accesses) We also assumed: Each command executes in 1 time unit Only the highest-order term of calculations counts (T(n) ~ 5n 2 ) Today: One more cost model: Counting the number of executions Lines of Code * Best case / Worst case * / Average case A correctness argument – Loop Invariants 26 Sep 2014Lecture 3 2

How can we sort a deck of cards? 26 Sep 2014Lecture 3 3 wikipedia

InsertionSort Video: What is the algorithm? 26 Sep 2014Lecture 3 4 wikipedia

Specification of an Algorithm What are the inputs? What are assumed properties of the inputs? What is the output? What is the important property of the output? 26 Sep 2014Lecture 3 5

InsertionSort – Specification First let’s make sure we know what we want to do: The specification of the algorithm AKA the problem we are trying to solve 26 Sep 2014Lecture 3 6 Input: sequence of n numbers A=(a 1, … a n ) Output: a permutation (reordering) of the input (a’ 1, … a’ n ) Such that a’ 1 ≤ a’ 2 ≤ … ≤ a’ n

Expression We can express the algorithm at different levels of detail: In English: imprecise Can convey the main idea of the algorithm But hides the details – some of them are important! We cannot use it to analyse the algorithm In a programming language: very precise Necessary for implementing the algorithm Does express the details Sometimes too much detail – can confuse the idea In pseudocode: happy medium Resembles a programming language Has a “good amount” of detail 26 Sep 2014Lecture 3 7 Modern high-level programming languages try to be more like pseudocode by hiding details.

InsertionSort Algorithm (in English) 1. Start from 1 st element of the array (optimisation: start from 2 nd ) 2. Shift element back until its right position 3. Continue to next element 4. Repeat (2) and (3) until the end of the array 26 Sep 2014Lecture 3 8

InsertionSort Algorithm (in pseudocode) 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 9

InsertionSort Algorithm (in Java) left as an exercise. 26 Sep 2014Lecture 3 10

Assumptions We use unspecified time units (tu) Each command takes 1tu Numerical data are stored in binary format Size of an int is 1 memory word. Of an array A[0..n-1] is n words. Program variables can store arbitrarily large numbers no overflow Simple numerical operations takes 1tu (+,-,*,/,mod,exp,..) 26 Sep 2014Lecture for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A

Best Case costno of times 1. for (j = 1; j<A.length; j++) {1 2. //shift A[j] into the sorted A[0..j-1] 3. i=j while i>=0 and A[i]>A[i+1] {1 5. swap A[i], A[i+1]1 6. i=i }} 8. return A1 26 Sep 2014Lecture 3 12

Best Case costno of times 1. for (j = 1; j<A.length; j++) {1 2. //shift A[j] into the sorted A[0..j-1] 3. i=j while i>=0 and A[i]>A[i+1] {1 5. swap A[i], A[i+1]1 6. i=i }} 8. return A1 In the best case the array is already sorted. 26 Sep 2014Lecture 3 13

Best Case costno of times 1. for (j = 1; j<A.length; j++) {1n 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-11n-1 4. while i>=0 and A[i]>A[i+1] {1 (1+1+…+1) n-1 times 5. swap A[i], A[i+1]10 6. i=i }} 8. return A11 In the best case the array is already sorted. The time (as a function of the input size n): T(n) = n + n-1 + n = 3n Sep 2014Lecture 3 14

Worst Case costno of times 1. for (j = 1; j<A.length; j++) {1 2. //shift A[j] into the sorted A[0..j-1] 3. i=j while i>=0 and A[i]>A[i+1] {1 5. swap A[i], A[i+1]1 6. i=i }} 8. return A1 26 Sep 2014Lecture 3 15

Worst Case costno of times 1. for (j = 1; j<A.length; j++) {1 2. //shift A[j] into the sorted A[0..j-1] 3. i=j while i>=0 and A[i]>A[i+1] {1 5. swap A[i], A[i+1]1 6. i=i }} 8. return A1 In the worst case the array is in reverse sorted order. 26 Sep 2014Lecture 3 16

26 Sep 2014Lecture 3 17

Worst Case 26 Sep 2014Lecture 3 18 costno of times 1. for (j = 1; j<A.length; j++) {1n 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-11n-1 4. while i>=0 and A[i]>A[i+1] {12+…+n 5. swap A[i], A[i+1]11+…+(n-1) 6. i=i-111+…+(n-1) 7. }} 8. return A11 In the worst case the array is in reverse sorted order. T(n) = n + n-1 + Sum x=2..n (x) + 2Sum x=1..n-1 (x-1) + 1 = n + n-1 + (n(n+1)/2 - 1) + 2n(n-1)/2 + 1 = (3/2)n 2 + (3/2)n - 1

Average Case 26 Sep 2014Lecture 3 19 costno of times 1. for (j = 1; j<A.length; j++) {1n 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-11n-1 4. while i>=0 and A[i]>A[i+1] {1(2+…+n)/2 5. swap A[i], A[i+1]1(1+…+(n-1))/2 6. i=i-11(1+…+(n-1))/2 7. }} 8. return A11 In the average case we shift each A[j] about j/2 positions to the left T(n) = n + n-1 + Sum x=2..n (x)/2 + Sum x=1..n-1 (x-1)/2 + 1 = …

Exercises Estimate the worst case running time cost of 2SUM by counting the number of times each line of code is executed. Assume each line takes 1 time unit to execute Now give this estimate using the tilde notation Estimate the worst case running time cost of InsertSort by counting the number of swap operations Now give this estimate using the tilde notation 26 Sep 2014Lecture 3 20

Correctness 26 Sep 2014Lecture 3 21

Why is our algorithm correct? We will make an argument of its correctness using a loop invariant. A loop invariant is a property which is true: At the beginning of the algorithm At the end of the algorithm Before each iteration of the algorithm 26 Sep 2014Lecture 3 22

Loop exit: InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 23

Loop exit: j==n InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 24 n=A.length

Loop exit: j==n sorted A[0..n-1] InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 25 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 26 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 27 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0] InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 28 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 29 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] After line 1: InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 30 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] After line 1:1≤j<n InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 31 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] After line 1: 1≤j<n sorted A[0..j-1] InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 32 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] After line 1: 1≤j<n sorted A[0..j-1]unsorted A[j..n-1] InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 33 n=A.length

Loop exit: j==n sorted A[0..n-1] Loop entry: j==1 sorted A[0..0]unsorted A[1..n-1] After line 1: 1≤j<n sorted A[0..j-1]unsorted A[j..n-1] Invariant: property that holds at the beginning of the loop at every iteration InsertionSort – loop invariant 1. for (j = 1; j<A.length; j++) { 2. //shift A[j] into the sorted A[0..j-1] 3. i=j-1 4. while i>=0 and A[i]>A[i+1] { 5. swap A[i], A[i+1] 6. i=i-1 7. }} 8. return A 26 Sep 2014Lecture 3 34 n=A.length

Exercise: Loop invariant of 1SUM? 26 Sep 2014Lecture int count = 0; 2. for (int i = 0; i < N; i++) 3. if (a[i] == 0) 4. count++; Properties: When we exit the loop (and the code)? At the start of the code? Immediately after line 1 is executed?