1 Jeff Edmonds York University COSC 2011 Lecture 3 Contracts Assertions Loop Invariants The Sum of Objects Insertion and Selection Sort Binary Search Like Examples Bucket (Quick) Sort for Humans Reverse Polish Notation (Stack) Whose Blocking your View (Stack) Parsing (Stack) Data Structure Invariants Stack and Queue in an Array Linked Lists Contracts, Assertions, and Invariants

2 PreconditionPostcondition On Step At a Time next I implored you to not worry about the entire computation. next It can be difficult to understand where computation go. i-1 i i i 0 T+1 i-1 i i i 0 T+1 Trust who passes you the baton and go around once

3 Contracts, Assertions, and Invariants Clear pre and post conditions separate out the responsibility of a routine or class and its user. Assertions state assumptions about what is true at a point in a computation. System, Data Structure, and Loop Invariants draw a picture of what must be maintained through out time while making progress.

4 A Routine, Class, or Data Structure: Implementer: Coder of the routine. Three Players User: Coder using routine. End User: Runs the program Not part of this course. This is us in CSE 3101 Complicated details are hidden from him. Correctness & efficiency. Contracts

5 Max( a,b,c ) if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) At least tell me what the algorithm is supposed to do. Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns. “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” Contracts

Modularity Each module has a well-specified job. specified by pre & post conditions of the abstract interface Allows the programmer to implement it as she wishes. Allows the user to reuse the module. preCond: Input has n numbers postCond: returns max or class Node { E element; Node next; } Node age = Node(5); Node name = Node(“Jeff”’);

Contracts Modularity Each module has a well-specified job. specified by pre & post conditions of the abstract interface Allows the programmer to implement it as she wishes. Allows the user to reuse the module. Hierarchical Design Each level depends on but does not micro manage the lower level

Contracts Modularity Each module has a well-specified job. specified by pre & post conditions of the abstract interface Allows the programmer to implement it as she wishes. Allows the user to reuse the module. Hierarchical Design Each level depends on but does not micro manage the lower level Encapsulation Internal details are kept private. And lack of trust. private class Node { E element; Node next; } Outside of the stack ADT no one can touch the nodes. Goal of C is to write efficient code In some ways close to machine code But highly prone to hard to find bugs. And computer security can be compromised by programs intentionally accessing memory in unintended ways. Goal of Java is encapsulation (and lack of trust) the modal validates everything the user sends.

9 Contracts, Assertions, and Invariants Clear pre and post conditions separate out the responsibility of a routine or class and its user. Assertions state assumptions about what is true at a point in a computation.

10 Paradigm Shift Is the black the form and the green the background? Is the green the form and the black the background? It is helpful to have different ways of looking at it.

11 A Sequence of Actions Max( a,b,c ) if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) If you are describing an alg to a friend, is this what you say? Do you think about an algorithm as a sequence of actions? Do you explain it by saying: “Do this. Do that. Do this”? Do you get lost about where the computation is and where it is going? What if there are many paths through many ifs and loops? How do you know it works for every path on every input?

12 A Sequence of Actions Max( a,b,c ) if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) How can you possibly understand this algorithm without knowing what is true when the computation is here? “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}”

13 A Sequence of Actions Max( a,b,c ) if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) How can you possibly understand this algorithm without knowing what is true when the computation is here? Tell me! “assert: m is max in {a,b}” “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” “assert: m is max in {a}” “assert: m is max in {a,b,c}”

14 A Sequence of Actions “preCond: Input has 3 numbers.” “assert: m is max in {a}” “assert: m is max in {a,b}” “assert: m is max in {a,b,c}” Max( a,b,c ) if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) “postCond: return max in {a,b,c}” vs A Sequence of Assertions It is helpful to have different ways of looking at it.

15 Purpose of Assertions Useful for – thinking about algorithms – developing – describing – proving correctness

16 Definition of Assertions An assertion is a statement about the current state of the data structure that is either true or false. eg. the amount in your bank account is not negative.

17 Definition of Assertions It is made at some particular point during the execution of an algorithm. It should be true independent of the path followed through the code and independent of the input. If it is false, then something has gone wrong in the logic of the algorithm.

18 Definition of Assertions An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader.

19 Definition of Assertions The current state of the computation –Everything that is true at a particular instant in time. –Imagine flying in from Mars, what do you need to know about the present to be able to continue. –It does not say anything about the past, i.e. how the computation got here. –Eg

20 Definition of Assertions The current state of the computation An assertion, A –is a function that takes as input the current state and that outputs –Eg A = “x is odd” A( ) = True A( ) = False True or False  Computation is on the path  Something has gone wrong!

21 Algorithm Termination You need to define some Measure of progress to prove that your algorithm eventually terminates. An Measure of Progress, M –is a function that takes as input the current state and that outputs –Eg Measure = “value of y” M( ) = 3 –We must prove that this number goes down (up) each iteration. a real number

22 Don't Be Frightened An assertion need not consist of formal mathematical mumble jumble Use an informal description and a picture

23 Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns. Specifying a Computational Problem Max( a,b,c ) “preCond: Input has 3 numbers.” if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) “postCond: return max in {a,b,c}”

24 Definition of Correctness &  If the input meets the preconditions, then the output must meet the postconditions. If the input does not meet the preconditions, then nothing is required.

25 if( ) then code else code end if if( ) then code else code end if … any code How is this proved? Definition of Correctness A Sequence of Assertions

26 if( ) then code else code end if if( ) then code else code end if … Must check 2 r different settings of paths through the code. Is there a faster way? A Sequence of Assertions any code Definition of Correctness

27 if( ) then code else code end if if( ) then code else code end if Step 1 code ¬ code … A Sequence of Assertions

28 if( ) then code else code end if if( ) then code else code end if … Step 2 code ¬ code A Sequence of Assertions

29 Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” A Sequence of Assertions Example code

30 A Sequence of Assertions Example code ¬ code Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}”

31 A Sequence of Assertions Example code ¬ code Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}”

32 Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” A Sequence of Assertions Example code

33 Contracts, Assertions, and Invariants Clear pre and post conditions separate out the responsibility of a routine or class and its user. Assertions state assumptions about what is true at a point in a computation. Loop Invariants draw a picture of what must be maintained through out time while making progress.

34 A good way to structure many programs: Store the key information you currently know in some data structure. In the main loop, take a step forward towards destination by making a simple change to this data. Iterative Algorithms loop (until done) take step end loop

35 Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Any assumptions that must be true about the state of computation every time it is at the top of the loop.

36 Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Any assumptions that must be true about the state of computation every time it is at the top of the loop.

37 Not just to please the prof. Not just to prove correctness. Use them to think about your algorithm! Before you start coding. What do you want to be true in the middle of your computation? –You need to know. –Let your reader know. Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]”

38 Not just to please the prof. Not just to prove correctness. Use them to think about your algorithm! Before you start coding. What do you want to be true in the middle of your computation? –You need to know. –Let your reader know. Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” loop “loop-invariant: m is max in A[1,i]” exit when (i=n) “postCond: m is max in A[1,n]”

39 codeA loop exit when codeB endloop codeC Definition of Correctness code How is this proved? Iterative Algorithms Loop Invariants We don’t know how many times it will iterate.

40 Iterative Algorithms Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Step 0 codeA Example

41 Step 1 ¬ codeB Iterative Algorithms Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Step 2Step i Example

42 Last Step codeC Iterative Algorithms Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Example

43 Your job is only to Trust who passes you the baton Go around once Pass on the baton i-1 i i i 0 T+1 A Relay Race Iterative Algorithms Loop Invariants

44 codeA loop exit when codeB endloop codeC 0 Person 0: Carries baton region to safe region codeA Establishing Loop Invariant

45 Person i: Running around the track i-1 i codeA loop exit when codeB endloop codeC i i ¬ codeB Exit Maintaining Loop Invariant

46 T+1 codeA loop exit when codeB endloop codeC codeC Exit Clean up loose ends Last Person: Carries baton from safe region to finish.

47 Ok I know you knew Insertion Sort But hopefully you are beginning to appreciate Loop Invariants for describing algorithms

48 Maintain Loop Invariant Can we be assured that the computation will always be in a safe location? By what principle?

49 Maintain Loop Invariant By Induction the computation will always be in a safe location.

50 No Assumptions: Suppose a Martian jumped into the top of the loop. All you would know is that was true. It alone must provide enough information so that, after going around the loop, you can establish that the loop invariant is again true.

51 Ending The Algorithm Exit Define Exit Condition Termination: 0 kmExit With sufficient progress, the exit condition will be met. Exit from these we must establish the post conditions. Exit When we exit, we know exit condition is true loop invariant is true

52 Let’s Recap

53 Iterative Algorithm with Loop Invariants Precondition: What is true about input Post condition: What is true about output.

54 Iterative Algorithm with Loop Invariants Goal: Your goal is to prove that no matter what the input is, as long as it meets the precondition, and no matter how many times your algorithm iterates, as long as eventually the exit condition is met, then the post condition is guarantee to be achieved. Proves that IF the program terminates then it works & 

55 Iterative Algorithm with Loop Invariants Loop Invariant: Picture of what is true at top of loop.

56 Iterative Algorithm with Loop Invariants Establishing the Loop Invariant. Our computation has just begun. All we know is that we have an input instance that meets the Pre Condition. Being lazy, we want to do the minimum work. And to prove that it follows that the Loop Invariant is then made true. codeA Establishing Loop Invariant

57 Iterative Algorithm with Loop Invariants Maintaining the loop invariant (while making progress) 79 km75 km Exit We arrived at the top of the loop knowing only the Loop Invariant is true and the Exit Condition is not. We must take one step (iteration) (making some kind of progress). And then prove that the Loop Invariant will be true when we arrive back at the top of the loop. ¬ codeB Maintaining Loop Invariant Exit

58 codeC Obtain the Post Condition Exit We know the Loop Invariant is true because we have maintained it. We know the Exit Condition is true because we exited. We do a little extra work. And then prove that it follows that the Post Condition is then true. Obtain the Post Condition: Exit Iterative Algorithm with Loop Invariants

59 Designing an Algorithm Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit Is this sufficient?

60 Consider an Algorithm Exit 0 kmExit 79 km75 km Exit

61 Loop Invariant Maintained Exit

62 Computation can always proceed

63 Computation always makes progress 79 km75 km79 km75 km Exit

64 Computation Terminates 79 km 75 km 0 km Exit

65 Computation Terminates Exit

66 Consider an Algorithm This is sufficient! Exit 0 kmExit 79 km75 km Exit

67 Partial Correctness Proves that IF the program terminates then it works &  codeA Establishing Loop Invariant codeC Clean up loose ends Exit ¬ codeB Maintaining Loop Invariant Exit

68 General Principle Do not worry about the entire computation. Take one step at a time! next

69 Some locations are too confusing for algorithm Algorithm too lazy to define step for every location. Computation Stuck

70 Safe Locations Algorithm specifies from which locations it knows how to step.

71 Loop Invariant “The computation is presently in a safe location.” Maybe true and maybe not. If not something has gone wrong.

72 Picture from the Middle Leap into the middle of the algorithm. What would you like your data structure to look like when you are half done?

73 Flow Smoothly The loop invariant should flow smoothly from the beginning to the end of the algorithm. –At the beginning, it should follow easily from the preconditions. –It should progress in small natural steps. –Once the exit condition has been met, the postconditions should easily follow.

74 Ask for 100% A good philosophy in life is to ask for 100% of what you want, but not to assume that you will get it. –What would you like to be true in the middle of your computation?

75 Ask for 100% Pretend that a genie has granted your wish. –You are now in the middle of your computation and your dream loop invariant is true.

76 Ask for 100% Maintain the Loop Invariant: – From here, are you able to take some computational steps that will make progress while maintaining the loop invariant?

77 No Assumptions: Suppose a Martian jumped into the top of the loop. All you would know is that was true. It alone must provide enough information so that, after going around the loop, you can establish that the loop invariant is again true.

78 Know What a LI Is Not ``the LI is...'' – code –A precondition –A postcondition –A statement that is always true Eg: “1+1=2” –The steps taken by the algorithm

79 Differentiating between Iterations x=x+2 –Meaningful as code –False as a mathematical statement x’ = x i = value at the beginning of the iteration x” = x i+1 = new value after going around the loop one more time. x” = x’+2 –Meaningful as a mathematical statement

80 This completes One Higher Level Abstract View of Iterative Algorithms

81 Simple Example A sum of objects You MUST do it my way!

82 Designing an Algorithm Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

83 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 What is the loop invariant? Each iteration add in a new object. Action not picture If you fight me, I will fail you

84 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 What is the loop invariant? What do you want to be true in the middle of the computation? To have a partial sum: s = … + i 2 (and 1 ≤ i ≤ I)

85 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (0 ≤ i ≤ I) Measure of Progress i = # of objects added. 79 km75 km Exit 79 km to school Step to make progress i = i+1 Exit Condition i = I

86 The Sum Problem In order to make progress, we may head in one direction. 79 km75 km Exit Step to make progress i = i+1

87 The Sum Problem In order to make progress, we may head in one direction. 79 km75 km Exit Step to make progress i = i+1

88 The Sum Problem In order to maintain the loop invariant, we must adjust to come back to the path. Exit

89 The Sum Problem This may be how we determine the step in then next iteration.

90 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Partial Step: i = i+1 ¬ codeB Maintaining Loop Invariant Exit Let i t and s t be the values of i and s at the beginning of the iteration and let i t+1 and s t+1 be their after going around again.

91 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Partial Step: i = i+1 Use the fact that you already have the LI true and take a small step. ¬ codeB: Make the loop Invariant true!!! Solve the entire algorithm!!! Panic!! next

92 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Partial Step: i = i+1 s t = … + i t 2 i t < I i t+1 = i t + 1 s t+1 = … + i t+1 2 ¬ codeB: (codeB in loop) (math) Write down LHS/RHS of what we must prove

93 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Partial Step: i = i+1 s t = … + i t 2 i t < I i t+1 = i t + 1 s t+1 =[ … + i t 2 ]+ i t+1 2 = … + i t+1 2 = s t + i t+1 2 ¬ codeB: s t+1 = s t + i t+1 2

94 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Partial Step: i = i+1 s = s + i 2 i = i + 1 i t+1 = i t + 1 s t+1 = s t + i t+1 2 i = i + 1 s = s + i 2

95 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Step: i = i+1, s = s + i 2 codeC Clean up loose ends Exit

96 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Step: i = i+1, s = s + i 2 s = … + i 2 i = I S = s S = … + I 2 codeC: I2I2

97 The Sum Problem Precondition: The input is an integer I. Postcondition: The output is S = … + I 2 Loop Invariant: s = … + i 2 (1 ≤ i ≤ I) Step: i = i+1, s = s + i 2 codeA Establishing Loop Invariant is =14 1+4=5 1 0 i = 0 s = 0

98 Designing an Algorithm Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

99 Simple Example Insertion Sort Algorithm

100 Reconsidering Simple Examples A good martial arts student will attentively repeat each fundamental technique many times. In contrast, many college students tune out when a concept (e.g., depth first search) is taught more than once. The better students listen carefully in order to refine and develop their understanding. Repetition Is An Opportunity Rudich

101 Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }}

102 Higher Level Abstract View Representation of an Algorithm 9 km 5 km

103 Problem Specification Precondition: The input is a list of n values with the same value possibly repeated. Post condition: The output is a list consisting of the same n values in non-decreasing order ,23,25,30,31,52,62,79,88,98

104 “State” of computation determined by values of all variables. Location of Computation

105 Location of Computation ,23,25,30,31,52,62,79,88,98 “State” of computation determined by values of all variables <52 25<62 79<

,23,25,30,31,52,62,79,88,98 23 Location too confusing for algorithm Algorithm too lazy to define step for every location. Computation Stuck 88 14<52 25<62 79<

107 Algorithm specifies from which locations it knows how to step. Safe Locations ,23,25,30,31,52,62,79,88,98

,23,25,30,31,52,62,79,88, ,31,52,88 Define Loop Invariant Loop Invariant: Input numbers split to left and right. Numbers on left are sorted.

,31,52, ,52,62,79 Location Not Determined Which subset of the elements are sorted, is not specified.

110 Defining Measure of Progress ,31,52,88 6 elements to school

111 Define Step ,31,52,

112 Define Step Select arbitrary element from side. Insert it where it belongs ,31,52, ,31,52,62,

113 Making progress while Maintaining the loop invariant ,31,52, ,31,52,62, km75 km 5 elements to school 6 elements to school Exit Sorted sub-list

n elements to school 14,23,25,30,31,52,62,79,88,98 0 elements to school Beginning & Ending Exit 0 kmExit

115 Explaining Insertion Sort We maintain a subset of elements sorted within a list. The remaining elements are off to the side somewhere. Initially, think of the first element in the array as a sorted list of length one. One at a time, we take one of the elements that is off to the side and we insert it into the sorted list where it belongs. This gives a sorted list that is one element longer than it was before. When the last element has been inserted, the array is completely sorted. But don’t do this. I want separate paragraphs for the separate steps.

116 Running Time Inserting an element into a list of size i takes  (i) time. Total = …+n ,31,52, ,31,52,62, = ?

n = number of white dots n-1+n=S Adding Made Easy

n-1+n=S n+ +n =S n = number of yellow dots n = number of white dots Adding Made Easy

n-1+n=S n+ +n =S = number of white dots = number of yellow dots n+1 n+1 n+1 n+1 n+1 n n n n n n = n(n+1) dots in the grid 2S dots S = n (n+1) 2 Adding Made Easy

120 Adding Made Easy

121 Running Time Inserting an element into a list of size i takes  (i) time. Total = …+n ,31,52, ,31,52,62, =  (n 2 )

122 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

,31,52,88 Loop Invariant: Input numbers split to left and right. Numbers on left are sorted. Select the smallest Insertion Sort Selection What changes? This was on an exam and most got it Action not picture WRONG

,23,25,30 Loop Invariant: Input numbers split to left and right. Numbers on left are sorted. All left ≤ All right Insertion Sort Selection What changes? This was on an exam and most got it ≤ WRONG

125 Define Step ,23,25,30 ≤

126 Define Step ,23,25,30 ≤ ,23,25,30,31 ≤ Select the smallest on right and put it at the end of the left.

127 Maintaining Loop Invariant ,23,25,30 ≤ ,23,25,30,31 ≤ ¬ codeB Exit Numbers now on left are sorted. Numbers on left were sorted. New number added to end came from right. All left ≤ All right

128 Maintaining Loop Invariant ,23,25,30 ≤ ,23,25,30,31 ≤ ¬ codeB Exit Now All left ≤ All right Was All left ≤ All right New number is smallest from right

129 Ok I know you knew Insertion Sort But hopefully you are beginning to appreciate Loop Invariants for describing algorithms

130 Define Problem: Binary Search PreConditions –Key 25 –Sorted List PostConditions –Find key in list (if there)

131 Define Loop Invariant Maintain a sub-list Such that key

132 Define Loop Invariant Maintain a sub-list. If the key is contained in the original list, then the key is contained in the sub-list. key

133 Define Step Make Progress Maintain Loop Invariant key

134 Define Step Cut sub-list in half. Determine which half the key would be in. Keep that half. key

135 Define Step Cut sub-list in half. Determine which half the key would be in. Keep that half. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

136 Define Step It is faster not to check if the middle element is the key. Simply continue. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

137 Make Progress The size of the list becomes smaller km75 km 79 km75 km Exit

138 Initial Conditions key If the key is contained in the original list, then the key is contained in the sub-list. The sub-list is the entire original list. n km

139 Ending Algorithm If the key is contained in the original list, then the key is contained in the sub-list. Sub-list contains one element. Exit km If the key is contained in the original list, then the key is at this location. key 25 Check this location. If it’s the key, then you have found it. If it’s not, then it was not in the original list.

140 Running Time The sub-list is of size n, n / 2, n / 4, n / 8,…,1 Each step  (1) time. Total =  (log n) key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

141 Code

142 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

143 Study: Many experienced programmers were asked to code up binary search. 80% got it wrong Good thing is was not for a nuclear power plant.

144 Make Precise Definitions Maintain a sub-list with end points i & j. key 25 ij

145 Make Precise Definitions Maintain a sub-list with end points i & j. key 25 ij Does not matter which, but you need to be consistent.

146 If the sub-list has even length, which element is taken to be mid? key Make Precise Definitions

147 If the sub-list has even length, which element is taken to be mid? key Should not matter Choose right. Make Precise Definitions mid

148 Common Bugs Cut sub-list in half. Determine which half the key would be in. Keep that half. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

149 Common Bugs If the middle element is the key, it can be skipped over. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. Exit

150 Common Bugs Fix the bug. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

151 Common Bugs Second fix, by making the left half slightly bigger. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. New bug?

152 Common Bugs Second fix, by making the left half slightly bigger. key New bug? No progress is made. Loop for ever! 79 km75 km Exit

153 Why is Binary Search so Easy to Get Wrong? 1 2 How many possible algorithms? How many correct algorithms? Probability of success by guessing? 3 i = mid else j = mid -1 i = mid + 1 else j = mid

154 Moral Use the loop invariant method to think about algorithms. Be careful with your definitions. Be sure that the loop invariant is always maintained. Be sure progress is always made.

155 Loop Invariants for Iterative Algorithms A second Binary Search like example

Binary Search Tree –Key 17 –A binary search tree. –Find key in BST (if there). key 17?

Binary Search Tree ≤ ≤ Its left children ≤ Any node ≤ Its right children Data Structure Invariant key 17?

Binary Search Tree ≤ ≤ We have maintained a sub-tree. If the key is contained in the original tree, then the key is contained in this sub-tree. Search Loop Invariant key 17?

159 Cut sub-tree in half. Determine which half the key would be in. Keep that half If key < root, then key is in left half. If key > root, then key is in right half. If key = root, then key is found Binary Search Tree ≤ ≤ key 17?

Binary Search Tree key 17? We have maintained a sub-tree. If the key is contained in the original tree, then the key is contained in this sub-tree. Search Loop Invariant

161 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

162 Loop Invariants for Iterative Algorithms A third Binary Search like example

163 A volunteer, please. Card Trick

164 Pick a Card Done

165 Loop Invariant: The selected card is one of these.

166 Which column? left

167 Loop Invariant: The selected card is one of these.

168 Selected column is placed in the middle.

169 I will rearrange the cards.

170 Relax Loop Invariant: I will remember the same about each column.

171 Which column? right

172 Loop Invariant: The selected card is one of these.

173 Selected column is placed in the middle.

174 I will rearrange the cards.

175 Which column? left

176 Loop Invariant: The selected card is one of these.

177 Selected column is placed in the middle.

178 Here is your card. Wow!

179 Bucket (Quick) Sort for Humans Input: A pile of things to sort. (Ex: Exams) Output: Sort them Requirements: Easy to execute by humans Only can have 7 piles on desk Can move one exam (or pile) at a time. Fast – O(nlog(n)) time. Likely the only algorithm in this course which you will execute by hand yourself.

180 Bucket (Quick) Sort for Humans [A-Z] Denotes an unsorted pile of exams with last names starting in the range [A-Z] Input:

181 Bucket (Quick) Sort for Humans [A-Z] Psychology study: Humans can only think about 5 things at once. Is the first letter of the name on the first exam [A-E] [F-K] [L-O] [P-T] [U-Z] in

182 Bucket (Quick) Sort for Humans [A-Z] [A-E][F-K] [L-O] [P-T] [U-Z] Bucket Sort Put exams one at a time in the correct bucket.

183 Bucket (Quick) Sort for Humans [A-E] [F-K] [L-O] [P-T] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down) [] sorted

184 [A-E] [F-K] [L-O] [P-T] [U-Z] [] sorted [A][B] [C] [D] [E] Bucket Sort Put exams one at a time in the correct bucket. Bucket (Quick) Sort for Humans

185 A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down) [] sorted [F-K] [L-O] [P-T] [U-Z] [A] [B] [C] [D] [E] Bucket (Quick) Sort for Humans

186 [] sorted [F-K] [L-O] [P-T] [U-Z] [A] [B] [C] [D] [E] Bucket (Quick) Sort for Humans [AA-AE][AF-AK] [AL-AO] [AP-AT] [AU-AZ]

187 [] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AA-AE] [AU-AZ] … A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

188 [] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AA-AE] [AU-AZ] … [AA-AE]

189 [] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AF-AK] [AU-AZ] … [AA-AE] sorted When sufficiently small sort by hand

190 [AA-AE] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AF-AK] [AU-AZ] … A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

191 [AA-AE] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AF-AK] [AU-AZ] … [AF-AK] sorted

192 [AA-AK] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AL-AO] [AU-AZ] … A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

193 [AA-AK] sorted [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AL-AO] [AU-AZ] … [AL-AO] [AP-AT] [AU-AZ] sorted

194 [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [AA-AZ] sorted

195 [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [A] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down) sorted

196 [A] [F-K] [U-Z] [B] [E] Bucket (Quick) Sort for Humans … … [BA-BE][BF-BK] [BL-BO] [BP-BT] [BU-BZ] sorted

197 [A] [F-K] [U-Z] [C] [E] Bucket (Quick) Sort for Humans … … [BA-BE] [BU-BZ] … A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down) sorted

198 [A-ZT] Bucket (Quick) Sort for Humans [ZU-ZZ] sorted

199 [A-Z] Bucket (Quick) Sort for Humans Exit sorted

200 Bucket (Quick) Sort for Humans Fast – O(nlog(n)) time. Each time you touch an exam, the size of its pile goes down by a factor of 5 Hence I touch it only log 5 n times. (Assuming names distributed randomly through alphabet) [A-Z] [A-E][F-K] [L-O] [P-T] [U-Z] [A-Z] [A-E][F-K] [L-O] [P-T] [U-Z]

201 Reverse Polish Notation Calculator Many calculators in used what we called Reverse Polish Notation Input: A string such as   Output: Determine the output of the calculator. Numbers are pushed on the stack. Operations pop two numbers and push back the result  = Stack  

202 Whose Blocking Your View? Input: A string buildings Output: For each, who is blocking its left view. For each building, must search back. May have to search back O(n) buildings. Time = O(n 2 ). We need a better loop invariant / stack algorithm!

203 Whose Blocking Your View? Input: A string buildings Output: For each, who is blocking its left view. A prefix of the buildings are done. The stack contains the sorted list of candidates for the next buildings. Pop all building that are too short for the next building, finding his blocker. Then push this next building. Stack

204 Whose Blocking Your View? Input: A string buildings Output: For each, who is blocking its left view. A prefix of the buildings are done. The stack contains the sorted list of candidates for the next buildings. Pop all building that are too short for the next building, finding his blocker. Then push the next building. Stack

205 Whose Blocking Your View? Input: A string buildings Output: For each, who is blocking its left view. Stack For each building, must pop the stack searching back. May have to pop O(n) buildings. Time = O(n 2 ). Oops. No better! But each building, is popped at most once. Time = O(n).

Computing Spans (not in book) Using a stack as an auxiliary data structure in an algorithm Given an array X, the span S[i] of X[i] is the maximum number of consecutive elements X[j] immediately preceding X[i] and such that X[j]  X[i] Spans have applications to financial analysis Example: stock at 52-week high Andranik Mirzaian X S Whose Blocking Your View?

Quadratic Algorithm Last Update: Oct 1, 2014 Andranik Mirzaian207 Algorithm spans1(X, n) // O(n 2 ) time Input: array X of n integers Output: array S of spans of X # of steps S  new array of n integers n for i  0.. n  1 don s  1n while s  i and X[i - s]  X[i] …+ (n  1) s  s …+ (n  1) S[i]  s n return S 1 Algorithm spans1(X, n) // O(n 2 ) time Input: array X of n integers Output: array S of spans of X # of steps S  new array of n integers n for i  0.. n  1 don s  1n while s  i and X[i - s]  X[i] …+ (n  1) s  s …+ (n  1) S[i]  s n return S 1 Whose Blocking Your View?

Computing Spans with a Stack Stack holds indices of the elements visible when “looking back” Scan X left to right  i  current index  pop stack until top index j satisfies X[j] > X[i]  set S[i]  i  j  push X[i] onto stack Last Update: Oct 1, 2014 Andranik Mirzaian208 Whose Blocking Your View?

Linear Time Algorithm Last Update: Oct 1, 2014 Andranik Mirzaian209 Algorithm spans2(X, n) // O(n) timeS  new array of n integers A  new empty stack for i  0.. n  1 do while (! A.isEmpty() && X[A.top()]  X[i] ) do A.pop() if A.isEmpty() then S[i]  i + 1 else S[i]  i - A.top() A.push(i) return S Algorithm spans2(X, n) // O(n) timeS  new array of n integers A  new empty stack for i  0.. n  1 do while (! A.isEmpty() && X[A.top()]  X[i] ) do A.pop() if A.isEmpty() then S[i]  i + 1 else S[i]  i - A.top() A.push(i) return S Analysis: Each index of X o Is pushed into the stack exactly once o Is popped from the stack at most once while-loop iterates at most n times, since each iteration has to pop. Algorithm spans2 runs in O(n) time. Whose Blocking Your View?

210 Parsing with a Stack Input: A string of brackets. Output: Each “(”, “{”, or “[” must be paired with a matching “)”, “}”, or “[”. [( )(( ))}{([( )])}] incorrect [()(()){([()])}]

211 Parsing with a Stack Input: A string of brackets. Output: Each “(”, “{”, or “[” must be paired with a matching “)”, “}”, or “[”. Loop Invariant: Prefix has been read. Matched brackets are matched and removed. Unmatched brackets are on the stack. [()(() Stack [(

212 Parsing with a Stack Input: A string of brackets. Output: Each “(”, “{”, or “[” must be paired with a matching “)”, “}”, or “[”. Loop Invariant: Prefix has been read. Matched brackets are matched and removed. Unmatched brackets are on the stack. Stack [( Opening Bracket: Push on stack. Closing Bracket: If matches that on stack pop and match. else return(unmatched) [()(()){ (({ }] )

public static boolean isMatched (String expression) { final String opening = "({["; // opening delimiters final String closing = ")}]"; // respective closing delimiters Stack buffer = new LinkedStack<>( ); for (char c : expression.toCharArray( )) { if (opening.indexOf(c) != −1) // this is a left delimiter buffer.push(c); else if (closing.indexOf(c) != −1) { // this is a right delimiter if (buffer.isEmpty( )) // nothing to match with return false; if (closing.indexOf(c) != opening.indexOf(buffer.pop( ))) return false; // mismatched delimiter } return buffer.isEmpty( ); // were all opening delimiters matched? } public static boolean isMatched (String expression) { final String opening = "({["; // opening delimiters final String closing = ")}]"; // respective closing delimiters Stack buffer = new LinkedStack<>( ); for (char c : expression.toCharArray( )) { if (opening.indexOf(c) != −1) // this is a left delimiter buffer.push(c); else if (closing.indexOf(c) != −1) { // this is a right delimiter if (buffer.isEmpty( )) // nothing to match with return false; if (closing.indexOf(c) != opening.indexOf(buffer.pop( ))) return false; // mismatched delimiter } return buffer.isEmpty( ); // were all opening delimiters matched? } Andranik Mirzaian213 Parsing with a Stack

214 begin Alg x = 5 begin loop x = ( [ { 3 } ] ) exit loop return x end Alg Linked Parsing with a Stack

215  Op   If  if (  Boolean  ) then  Op   Equ  =  Equ   Term   Term  +  Equ   Factor  Term   Term   Factor  y x 7 3 Parsing with a Stack

216 Contracts, Assertions, and Invariants Clear pre and post conditions separate out the responsibility of a routine or class and its user. Assertions state assumptions about what is true at a point in a computation. System, Data Structure, and Loop Invariants draw a picture of what must be maintained through out time while making progress.

217 Dude! You have been teaching 3101 too long. This is not an course on Algorithms, but on Data Structures! Data Structure Invariants The importance of invariants is the same. Differences: 1.An algorithm must terminate with an answer, while systems and data structures may run forever. 2.An algorithm gets its full input at the beginning, while data structures gets a continuous stream of instructions from the user. Both have invariants that must be maintained.

218 Data Structure Invariants A Data Structure: Has data organized in ways dictated by its invariants. Has allowed operations on that data. Implementer: Coder of the data structure. Three Players User: Coder using data structure. End User: Runs the program Not part of this course. This is us in CSE 2011 Complicated details are hidden from him. Correctness & efficiency.

219 Data Structure Invariants An Abstract (Public) Data Structure: The minimal needed by the User so he can understand how best to visualize the data and what each operation does. Eg of Public Invariants: Bank: The amount is not negative. Stack: Contains an ordered list of objects which cannot be changed except at the top via the explicit push and pop operations. Drug Allocation Data Base: A patient is not simultaneously prescribed a set of drugs that interact poorly with each other. The implementer must make sure that each operation maintains these invariants. User Implementer Important in recursion and parsing.

220 Data Structure Invariants An Implemented (Private) Data Structure: The details need to be worked out so that operations are correct and efficient. This is all hidden from the User. Eg of Private Invariants: Bank: Stack: Drug Allocation Data Base: User Implementer Charge extra fees. For each patient give name address and list of drugs. or

221 Constructor  preCond Constructor   Invariants Data Struc  Establishing Loop Invariant Data Structure Invariants When a user wants to construct an instance of data structure the implementer must make sure that its constructor establishes all of its loop-invariants. Initially the stack contains no objects.

222  Invariants Data Struc  Destructor  preCond Destructor   postCond Data Struc  Clean up loose ends Exit Data Structure Invariants A destructor for the data structure must clean up loose ends.

223 Data Structure Invariants Assume we fly in from Mars and  Invariants Data Struc t  is true:  Invariants Data Struc t+1   postCond Push  Maintaining Loop Invariant Exit  Invariants Data Struc t  Push Operation  preCond Push  Assume the user correctly calls the Push Operation:  preCond Push  The input is info for a new element. Implementer must ensure:  postCond Push  The element is pushed on top of the stack.  Invariants Data Struc t+1 

224 Data Structure Invariants Do not worry about the entire computation. Take one step at a time! top = top + 1; A[top] = info;

225 Data Structure Invariants  postCond Pop  The element is popped off top of the stack.  Invariants Data Struc t+1  info = A[top]; top = top - 1;

226 Data Structure Invariants Stack: Add and Remove from same end. Queue: Add and Remove from opposite ends.

227 Data Structure Invariants Queue: Add and Remove from opposite ends. How would we remove an element from the front end? Shifting all these elements to the beginning of the array time  # of elements instead of constant

228 Data Structure Invariants Queue: Add and Remove from opposite ends. How would we remove an element from the front end? Leave the elements in place. Change the loop invariant!

229 Data Structure Invariants Queue: Add and Remove from opposite ends. How would we remove an element from the front end? Leave the elements in place. Change the loop invariant!

230 Data Structure Invariants Queue: Add and Remove from opposite ends. How would we remove an element from the front end? Change the loop invariant! Algorithm dequeue() if isEmpty() then throw EmptyQueueException else info  A[bottom] bottom  (bottom + 1) mod N return e

231 Data Structure Invariants Queue: Add and Remove from opposite ends. Limitations of an array implementation. –The maximum size must be defined a priori and cannot be changed ie could be too much or too little. –Trying to push a new element into a full stack causes an exception or you need to create a bigger array and copy everything over.

232 Data Structure Invariants An Implemented (Private) Data Structure: The details need to be worked out so that operations are correct and efficient. This is all hidden from the User. Eg of Private Invariants: Bank: Stack: Drug Allocation Data Base: User Implementer Charge extra fees. For each patient give name address and list of drugs. or

233 Constructor  preCond Constructor   Invariants Data Struc  Establishing Loop Invariant Data Structure Invariants When a user wants to construct an instance of data structure the implementer must make sure that its constructor establishes all of its loop-invariants. Initially it contains no objects.

234 Data Structure Invariants Assume we fly in from Mars and  Invariants Data Struc t  is true:  Invariants Data Struc t+1   postCond Push  Maintaining Loop Invariant Exit  Invariants Data Struc t  Push Operation  preCond Push  Assume the user correctly calls the Push Operation:  preCond Push  The input is info for a new element. Implementer must ensure:  postCond Push  The element is pushed on top of the stack.  Invariants Data Struc t+1 

235 Data Structure Invariants  Invariants Data Struc t   preCond Push   postCond Push   Invariants Data Struc t+1  Don’t panic. Just draw the pictures and move the pointers.

236 Data Structure Invariants  Invariants Data Struc t   preCond Push   postCond Push   Invariants Data Struc t+1 

237 Data Structure Invariants  Invariants Data Struc t   preCond Push   postCond Push   Invariants Data Struc t+1  Special Case: Empty

238 Data Structure Invariants Assume we fly in from Mars and  Invariants Data Struc t  is true:  Invariants Data Struc t+1   postCond Pop  Maintaining Loop Invariant Exit  Invariants Data Struc t  Pop Operation  preCond Pop  Assume the user correctly calls the Pop Operation:  preCond Pop  No inputs Implementer must ensure:  postCond Pop  The top element is Popped off stack and returned.  Invariants Data Struc t+1 

239 Data Structure Invariants  Invariants Data Struc t   preCond Pop   postCond Pop   Invariants Data Struc t+1  Don’t panic. Just draw the pictures and move the pointers.

240 Data Structure Invariants  Invariants Data Struc t   preCond Pop   postCond Pop   Invariants Data Struc t+1 

241 Data Structure Invariants  Invariants Data Struc t   preCond Pop   postCond Pop   Invariants Data Struc t+1  Special Case: Empty

242 Data Structure Invariants  Invariants Data Struc t   preCond Add Rear   postCond Add Rear   Invariants Data Struc t+1  How about adding a new element to the rear? Not hard. Left as homework.

243 Data Structure Invariants  Invariants Data Struc t   preCond Remove Rear   postCond Remove Rear   Invariants Data Struc t+1  How about removing an element from the rear? Is it so easy??? last must point at the second last element. How do we find it? You have to walk there from first! time  # of elements instead of constant

244 Data Structure Invariants FrontRear Add ElementTime Constant Remove ElementTime Constant Time  n Stack: Add and Remove from same end. Actually, for a Stack the last pointer is not needed.

245 Data Structure Invariants FrontRear Add ElementTime Constant Remove ElementTime Constant Time  n Stack: Add and Remove from same end. Queue: Add and Remove from opposite ends.

246 Data Structure Invariants FrontRear Add ElementTime Constant Remove ElementTime Constant Time  n Time Constant trailer header nodes/positions elements Doubly-linked lists allow more flexible list

247 Data Structure Invariants  preCond Find  Suppose you want to find where the new element 6 would go?  Invariants Data Struc t   postCond Find  We want the location sandwiched. How do you get there? Walk!

248 Data Structure Invariants  preCond Find   Invariants Data Struc t   postCond Find  We want the location sandwiched. What would you like to be true in the middle of your computation?

249 Data Structure Invariants How do you initialize this walk? Sandwich the location before the first node.

250 Data Structure Invariants Just take one step. How do you maintain this while making progress? Exit

251 Data Structure Invariants Exit

252 Data Structure Invariants  preCond Insert   postCond Find  We want the location sandwiched. The location for the new element has been found.  PostCond Insert  The new element has been inserted where it belongs.

253 Data Structure Invariants  preCond Insert  The location for the new element has been found.  PostCond Insert  The new element has been inserted where it belongs.  postCond Insert   Invariants Data Struc t+1 

254 Data Structure Invariants  preCond Delete  The element to be deleted has been found.  PostCond Delete  The found element has been deleted  postCond Delete   Invariants Data Struc t+1 

255 Learning Outcomes From this lecture, you should be able to: –Use the loop invariant method to think about iterative algorithms. –Prove that the loop invariant is established. –Prove that the loop invariant is maintained in the ‘typical’ case. –Prove that the loop invariant is maintained at all boundary conditions. –Prove that progress is made in the ‘typical’ case –Prove that progress is guaranteed even near termination, so that the exit condition is always reached. –Prove that the loop invariant, when combined with the exit condition, produces the post-condition. –Trade off efficiency for clear, correct code.

256 End Contracts, Assertions, and Invariants