Presentation is loading. Please wait.

Presentation is loading. Please wait.

Iterative Algorithms & Loop Invariants

Similar presentations


Presentation on theme: "Iterative Algorithms & Loop Invariants"— Presentation transcript:

1 Iterative Algorithms & Loop Invariants
Assertions Home to School Problem Steps in an Iterative Algorithm Insertion Sort Designing an Algorithm Finding Errors Typical Loop Invariants Binary Search Like Examples The Partition Problem Greatest Common Divisor Bucket (Quick) Sort for Humans Radix/Counting Sort

2 We will start with Assertions

3 Paradigm Shift Is the black the form and the green the background?
Is the green the form and the black the background? It is helpful to have different ways of looking at it.

4 A Sequence of Actions If you are describing an alg to a friend, is this what you say? Do you think about an algorithm as a sequence of actions? Do you explain it by saying: “Do this. Do that. Do this”? Do you get lost about where the computation is and where it is going? What if there are many paths through many ifs and loops? How do you know it works for every path on every input? Max( a,b,c ) m = a if( b>m ) m = b endif if( c>m ) m = c endif return(m)

5 At least tell me what the algorithm is supposed to do.
A Sequence of Actions Max( a,b,c ) “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” At least tell me what the algorithm is supposed to do. m = a if( b>m ) m = b endif Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns. if( c>m ) m = c endif return(m)

6 A Sequence of Actions Max( a,b,c ) “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” m = a How can you possibly understand this algorithm without knowing what is true when the computation is here? if( b>m ) m = b endif if( c>m ) m = c endif return(m)

7 A Sequence of Actions Max( a,b,c ) “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” m = a How can you possibly understand this algorithm without knowing what is true when the computation is here? if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c endif Tell me! return(m)

8 Purpose of Assertions Useful for thinking about algorithms developing
describing proving correctness

9 Definition of Assertions
An assertion is a statement about the current state of the data structure that is either true or false. eg. the amount in your bank account is not negative.

10 Definition of Assertions
It is made at some particular point during the execution of an algorithm. If it is false, then something has gone wrong in the logic of the algorithm.

11 Definition of Assertions
An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader.

12 Don't Be Frightened An assertion need not consist of formal mathematical mumble jumble Use an informal description and a picture

13 Value Correctness Don’t tack on a formal proof of correctness after coding to make the professor happy. It need not be mathematical mumbo jumbo. Goal: To think about algorithms in such way that their correctness is transparent.

14 Paradigm Shift Is the black the form and the green the background?
Is the green the form and the black the background? It is helpful to have different ways of looking at it.

15 Don’t worry about code at the moment.
A Sequence of Actions vs A Sequence of Assertions Max( a,b,c ) “preCond: Input has 3 numbers.” “postCond: return max in {a,b,c}” if( b>m ) m = b endif m = a if( c>m ) m = c endif return(m) “assert: m is max in {a}” “assert: m is max in {a,b,c}” Don’t worry about code at the moment. Think of an algorithm as a sequence of assertions. “assert: m is max in {a,b}”

16 vs A Sequence of Assertions
A Sequence of Actions vs A Sequence of Assertions Max( a,b,c ) “preCond: Input has 3 numbers.” “assert: m is max in {a}” After we understand the algorithm as a sequence of assertions, then we can add the code back in. “assert: m is max in {a,b}” “assert: m is max in {a,b,c}” “postCond: return max in {a,b,c}”

17 vs A Sequence of Assertions
A Sequence of Actions vs A Sequence of Assertions Max( a,b,c ) “preCond: Input has 3 numbers.” Code can be created between each pair of assertions, independently, without worrying about past or future. One step at a time. “assert: m is max in {a}” “assert: m is max in {a,b}” “assert: m is max in {a,b,c}” “postCond: return max in {a,b,c}”

18 (This creates independence between the blocks.)
A Sequence of Actions vs A Sequence of Assertions Max( a,b,c ) “preCond: Input has 3 numbers.” “assert: m is max in {a}” “postCond: return max in {a,b,c}” Assume you fly in from Mars and know nothing about the state of the computation except that the ith assertion is true (This creates independence between the blocks.) “assert: m is max in {a,b}” “assert: m is max in {a,b,c}”

19 vs A Sequence of Assertions
A Sequence of Actions vs A Sequence of Assertions Create simple code/actions to change the state of the computation to satisfy the i+1st assertion “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}”

20 vs A Sequence of Assertions
A Sequence of Actions vs A Sequence of Assertions Prove: <assertioni> code <assertioni+1> Two cases: <assertioni> c>m m=c <assertioni+1> “assert: m is max in {a,b}” if( c>m ) m = c endif <assertioni> c  m no change <assertioni+1> “assert: m is max in {a,b,c}”

21 Put the pieces together.
A Sequence of Actions vs A Sequence of Assertions Put the pieces together. <assertion1> code <assertion2> <assertion3> <PostCond> <PreCond> Max( a,b,c ) “preCond: Input has 3 numbers.” if( b>m ) m = b endif “assert: m is max in {a}” m = a return(m) “postCond: return max in {a,b,c}” “assert: m is max in {a,b}” if( c>m ) m = c endif “assert: m is max in {a,b,c}” <PreCond> all code <PostCond>

22 Definition of Correctness
<PreCond> & <code> Þ <PostCond> If the input meets the preconditions, then the output must meet the postconditions. If the input does not meet the preconditions, then nothing is required.

23 Iterative Algorithms Loop Invariants
<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeB endloop codeC <postCond> Loop-Invariant: Any assumptions that must be true about the state of computation every time it is at the top of the loop. Use: To help understanding for coder and reader Proof of correctness.

24 Iterative Algorithms Loop Invariants
<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeB endloop codeC <postCond> Definition of Correctness <preCond> <any conditions> code <postCond> How is this proved? We don’t know how many times it will iterate.

25 Iterative Algorithms Loop Invariants
Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Iterative Algorithms Loop Invariants Step 0 <preCond> codeA <loop-inv> Example

26 Iterative Algorithms Loop Invariants
Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Iterative Algorithms Loop Invariants Example Step 1 Step 2 Step i <loop-invariant> ¬<exit Cond> codeB <loop-inv>

27 Iterative Algorithms Loop Invariants
Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Iterative Algorithms Loop Invariants Example Last Step <loop-invariant> <exit Cond> codeC <postCond>

28 Partial Correctness Establishing Loop Invariant
<preCond> codeA <loop-invariant> Establishing Loop Invariant <loop-invariant> ¬<exit Cond> codeB Maintaining Loop Invariant Exit <loop-invariant> <exit Cond> codeC <postCond> Clean up loose ends Exit Proves that IF the program terminates then it works <PreCond> & <code> Þ <PostCond>

29 Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.”
i = 1 m = A[1] loop exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Not just to please the prof. Not just to prove correctness. Use them to think about your algorithm! Before you start coding. What do you want to be true in the middle of your computation? You need to know. Let your reader know.

30 Loop Invariants Max( A ) “preCond: Input is array A[1,n] of n values.”
i = 1 m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Not just to please the prof. Not just to prove correctness. Use them to think about your algorithm! Before you start coding. What do you want to be true in the middle of your computation? You need to know. Let your reader know.

31 Algorithm Termination
You need to define some Measure of progress to prove that your algorithm eventually terminates.

32 Two Key Types of Algorithms
Iterative Algorithms Recursive Algorithms

33 Iterative Algorithms loop (until done) take step end loop
A good way to structure many programs: Store the key information you currently know in some data structure. In the main loop, take a step forward towards destination by making a simple change to this data.

34 I like understanding things using a story.
I will now tell one.

35 The Getting to School Problem

36 Problem Specification
Pre condition: location of home and school Post condition: traveled from home to school

37 Algorithm What is an Algorithm?

38 Algorithm The algorithm defines the computation route.

39 Algorithm Is this reasonable?

40 Complexity There are an infinite number of input instances
Algorithm must work for each.

41 Complexity Difficult to predict where computation might be in the middle of the computation.

42 Location of Computation
The current “State” of computation is determined by values of all variables.

43 Location of Computation
Suppose the computation ends up here?

44 Don’t Panic Where ever the computation might be, take best step towards school.

45 General Principle Do not worry about the entire computation.
Take one step at a time!

46 Defining Algorithm Wherever the computation might be, take best step towards school.

47 Take a step What is required of this step?

48 A Measure of Progress 75 km to school 79 km to school

49 Defining Algorithm Is this sufficient to define a working algorithm?
79 km 75 km

50 Working Algorithm Computation steadily approaches goal 79 km to school

51 Defining Algorithm Extra requirements 79 km 75 km km 79 km 0 km

52 Define a Step Wherever the computation might be, take best step towards school.

53 Computation Stuck Some location too confusing for algorithm
Algorithm too lazy to define step for every location.

54 Safe Locations Algorithm specifies from which locations it knows how to step.

55 Loop Invariant “The computation is presently in a safe location.”
Maybe true and maybe not If not something has gone wrong.

56 Defining Algorithm From every safe location, define one step towards school.

57 Take a step What is required of this step?

58 Maintain Loop Invariant
If the computation is in a safe location, it does not step into an unsafe one.

59 Maintain Loop Invariant
If the computation is in a safe location, it does not step into an unsafe one. Can we be assured that the computation will always be in a safe location?

60 Maintain Loop Invariant
If the computation is in a safe location, it does not step into an unsafe one. Can we be assured that the computation will always be in a safe location? No. What if it is not initially true?

61 Establishing Loop Invariant
From the Pre-Conditions on the input instance we must establish the loop invariant.

62 Maintain Loop Invariant
Can we be assured that the computation will always be in a safe location? By what principle?

63 Maintain Loop Invariant
By Induction the computation will always be in a safe location.

64 Ending The Algorithm Exit Define Exit Condition Termination:
0 km Exit With sufficient progress, the exit condition will be met. Exit When we exit, we know exit condition is true loop invariant is true Exit from these we must establish the post conditions.

65 Let’s Recap

66 Designing an Algorithm
Is this sufficient? Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

67 Consider an Algorithm Exit 79 km 75 km Exit 0 km Exit Exit

68 Loop Invariant Maintained
Exit

69 Computation can always proceed

70 Computation always makes progress
79 km 75 km 79 km 75 km Exit

71 Computation Terminates
79 km 75 km 0 km Exit Exit 0 km

72 Computation Terminates
Exit Exit

73 Consider an Algorithm This is sufficient! Exit 79 km 75 km Exit 0 km

74 This completes One Higher Level Abstract View of Iterative Algorithms

75 Insertion Sort Algorithm
Simple Example Insertion Sort Algorithm

76 Reconsidering Simple Examples
A good martial arts student will attentively repeat each fundamental technique many times. In contrast, many college students tune out when a concept (e.g., depth first search) is taught more than once. The better students listen carefully in order to refine and develop their understanding. Repetition Is An Opportunity Rudich

77 Code Representation of an Algorithm
class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }}

78 Higher Level Abstract View Representation of an Algorithm
9 km 5 km

79 Designing an Algorithm
Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

80 Problem Specification
Precondition: The input is a list of n values with the same value possibly repeated. Post condition: The output is a list consisting of the same n values in non-decreasing order. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98

81 Location of Computation
“State” of computation determined by values of all variables.

82 Location of Computation
“State” of computation determined by values of all variables. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98 88 14<52 31 25<62 30 23 79<98

83 Computation Stuck Location too confusing for algorithm
Algorithm too lazy to define step for every location. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98 88 14<52 31 25<62 30 23 79<98

84 Safe Locations Algorithm specifies from which locations it knows how to step. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98

85 Define Loop Invariant Some subset of the elements are sorted
The remaining elements are off to the side. 88 14 98 25 62 52 79 30 23 31 14,23,25,30,31,52,62,79,88,98 14 98 25 62 79 30 23,31,52,88

86 Location Not Determined
Which subset of the elements are sorted, is not specified. 23 98 25 88 31 30 14,52,62,79 14 98 25 62 79 30 23,31,52,88

87 Defining Measure of Progress
14 98 25 62 79 30 23,31,52,88 6 elements to school

88 Define Step 30 14 23,31,52,88 25 62 98 79

89 Define Step Select arbitrary element from side.
Insert it where it belongs. 30 14 23,31,52,88 25 62 98 79 14 98 25 79 30 23,31,52,62,88

90 Making progress while Maintaining the loop invariant
79 km 75 km Exit 30 14 23,31,52,88 25 62 98 79 14 98 25 79 30 6 elements to school 23,31,52,62,88 5 elements to school Sorted sub-list

91 Beginning & Ending n elements to school 0 elements to school 88 14 98
0 km Exit Exit 88 14 98 25 62 52 79 30 23 31 88 14 98 25 62 52 79 30 23 31 n elements to school 0 elements to school 14,23,25,30,31,52,62,79,88,98 14,23,25,30,31,52,62,79,88,98

92 Running Time Inserting an element into a list of size i takes (i) time. Total = …+n = ? 30 14 23,31,52,88 25 62 98 79 14 98 25 79 30 23,31,52,62,88

93 Running Time Inserting an element into a list of size i takes (i) time. Total = …+n = (n2) 30 14 23,31,52,88 25 62 98 79 14 98 25 79 30 23,31,52,62,88

94 Algorithm Definition Completed
Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

95 Yet Another Abstract View
T+1 A Relay Race Your job is only to Trust who passes you the baton Go around once Pass on the baton

96 Establishing Loop Invariant
Person 0: Carries baton region to safe region <preCond> codeA <loop-invariant> Establishing Loop Invariant <preCond> codeA loop <loop-invariant> exit when <exit Cond> codeB endloop codeC <postCond>

97 Maintaining Loop Invariant
Person i: Running around the track <loop-invariant> ¬<exit Cond> codeB Exit Maintaining Loop Invariant <preCond> codeA loop <loop-invariant> exit when <exit Cond> codeB endloop codeC <postCond> i-1 i i i

98 Last Person: Carries baton from safe region to finish.
<loop-invariant> <exit Cond> codeC <postCond> Exit Clean up loose ends <preCond> codeA loop <loop-invariant> exit when <exit Cond> codeB endloop codeC <postCond> T+1

99 Insertion Sort as a Relay Race
88 52 14 31 62 25 30 98 23 79

100 Establish Loop Invariant from preconditions
88 14 98 25 62 52 79 30 31 23

101 Maintaining Loop Invariant
88 52 14 31 62 25 30 98 23 79 1

102 Maintaining Loop Invariant
88 52 14 31 62 25 30 98 23 79

103 Maintaining Loop Invariant
88 52 14 31 62 25 30 98 23 79 i

104 Maintaining Loop Invariant
52 14 31,88 62 25 30 98 23 79

105 Maintaining Loop Invariant
52 14 31,88 62 25 30 98 23 79 i

106 Maintaining Loop Invariant
52 14 25,31,88 62 30 98 23 79

107 Maintaining Loop Invariant
52 14 25,31,88 62 30 98 23 79 i

108 Maintaining Loop Invariant
14 25,31,52,88 62 30 98 23 79

109 Maintaining Loop Invariant
14,23,25,30,31,52,62,79,88,98

110 Clean up loose ends T+1 14,23,25,30,31,52,62,79,88,98

111 Ok I know you knew Insertion Sort
But hopefully you are beginning to appreciate Loop Invariants for describing algorithms

112 Explaining Insertion Sort
We maintain a subset of elements sorted within a list. The remaining elements are off to the side somewhere. Initially, think of the first element in the array as a sorted list of length one. One at a time, we take one of the elements that is off to the side and we insert it into the sorted list where it belongs. This gives a sorted list that is one element longer than it was before. When the last element has been inserted, the array is completely sorted.

113 Explaining Selection Sort
We maintain that the k smallest of the elements are sorted in a list. The larger elements are in a set on the side. Initially, with k=0, all the elements are in this set. Progress is made by finding the smallest element in the remaining set of large elements and adding this selected element at the end of the sorted list of elements. This increases k by one. Stop with k=n. At this point, all the elements have been selected and the list is sorted. 88 98 62 52 79 < 14,23,25,30,31

114 Designing an Algorithm Designing algorithms is an art form.

115 Designing Loop Invariants
Coming up with the loop invariant is the hardest part of designing an algorithm. It requires practice, perseverance, and insight. Yet from it the rest of the algorithm follows easily

116 Use This Process Don't come up with the loop invariant after the fact. Use it to design your algorithm.

117 You must design a working algorithm first.
Don’t start coding You must design a working algorithm first.

118 Exemplification: Try solving the problem on small input examples.
Rudich

119 Start with Small Steps What basic steps might you follow to make some kind of progress towards the answer? Describe or draw a picture of what the data structure might look like after a number of these steps.

120 Picture from the Middle
Leap into the middle of the algorithm. What would you like your data structure to look like when you are half done?

121 The Work Completed The loop invariant should state what work has been completed towards solving the problem and what works still needs to be done. 79 km to school 75 km

122 Flow Smoothly The loop invariant should flow smoothly from the beginning to the end of the algorithm. At the beginning, it should follow easily from the preconditions. It should progress in small natural steps. Once the exit condition has been met, the postconditions should easily follow.

123 Ask for 100% What would you like to be true in the middle of your computation?

124 Pretend that a genie has granted your wish.
Ask for 100% Pretend that a genie has granted your wish. You are now in the middle of your computation and your dream loop invariant is true.

125 Ask for 100% Maintain the Loop Invariant:
From here, are you able to take some computational steps that will make progress while maintaining the loop invariant?

126 Ask for 100% If you can maintain the loop invariant, great. If not,
Too Weak: If your loop invariant is too weak, then the genie has not provided you with everything you need to move on. Too Strong: If your loop invariant is too strong, then you will not be able to establish it initially or maintain it.

127 No Assumptions: Suppose a Martian jumped into the top of the loop. All you would know is that <loop-invariant> was true. It alone must provide enough information so that, after going around the loop, you can establish that the loop invariant is again true.

128 Know What a LI Is Not ``the LI is ...'' code A precondition
A postcondition A statement that is always true Eg: “1+1=2” The steps taken by the algorithm

129 Differentiating between Iterations
x=x+2 Meaningful as code False as a mathematical statement x’ = xi = value at the beginning of the iteration x” = xi+1 = new value after going around the loop one more time. x” = x’+2 Meaningful as a mathematical statement

130 Find Errors Which of the steps to develop an iterative algorithm did the student fail to do correctly?

131 Find Errors If I = -5, s = 1-5 j = Fine: Describes input and output.

132 Find Errors Variables need to be documented.
Fine: Purpose outlined in LI

133 Find Errors Loop Invariants are pictures of current state. Not actions! Not algorithms!

134 Fine: Shows a relationship between the variables.
Find Errors Fine: Shows a relationship between the variables.

135 Find Errors Let s’ and i’ be values of s and i when at the top of the loop. Let s” and i” be values after going around again.

136 Find Errors LI  s’ = j=1i’ j. Code  s” = s’+i’ i” = i’+1.
Together: s” = (j=1i’ j) + i’. i’ is added in twice. i” should be added.

137 Find Errors i = 1  j=1i j = 1 = s.

138 Find Errors Exit Better to say: loop loop-invariant: _____
exit when i > I exit when i>I

139 Find Errors LI  s = j=1i j. Exit  i > I Together: s = j=1I+1 j.
 Post Cond. exit when i>I

140 Find Errors Test: on typical value i = 3  s=1+2+3. on special values
exit when i>I

141 Typical Types of Loop Invariants
More of the input More of the output Narrowed the search space Case Analysis Work Done

142 More of the Input Loop Invariant
The input consists of an array of objects Solution Extra We have read in the first i objects. We will pretend that this prefix is the entire input. We have a solution for this prefix plus some additional information

143 Loop Invariants and Deterministic Finite Automaton
L = {a Î {0,1}* | a has length at most three and the number of 1's is odd }.

144 More of the Input Loop Invariant
The input consists of an array of objects Solution Solution Extra Extra We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have to one for this larger for prefix. (plus some additional information)

145 More of the Input Loop Invariant
The input consists of an array of objects Solution Solution Extra Extra 79 km 75 km Exit i to i+1

146 More of the Input Loop Invariant
The input consists of an array of objects Solution Exit End the end, we have read in the entire input. The LI gives us that we have a solution for this entire input.

147 Insertion Sort The input consists of an array of integers Solution
52,23,88,31,25,30,98,62,14,79 23,31,52,88 Solution We have read in the first i objects. We will pretend that this prefix is the entire input. We have a solution for this prefix

148 Insertion Sort The input consists of an array of integers
52,23,88,31,25,30,98,62,14,79 23,25,31,52,88 23,31,52,88 We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have to one for this larger for prefix.

149 More of the Input Loop Invariant
The input consists of an array of objects A student midterm answer: Each iteration considers the next input item. Loop Invariants are pictures of current state. Not actions! Not algorithms!

150 More of the Input Loop Invariant
The input consists of an array of objects A student midterm answer: We have considered the ith input element. What about elements [1...i]?

151 More of the Input Loop Invariant
The input consists of an array of objects A student midterm answer: We have considered input elements [1..i]. So?

152 More of the Input Loop Invariant
The input consists of an array of objects Solution A student midterm answer: We have read in the first i objects. We have a solution for each. Each object does not need a separate solution The input as a whole needs a solution.

153 More of the Input Loop Invariant
The input consists of an array of objects Solution Extra A student midterm answer: We have a solution for the prefix consisting of elements [1..i]. (plus some additional information)

154 Longest Block of Ones Alg reads the digits one at a time and remembers enough about what read so that it does not need to reread anything. (i.e. a DFA)

155 Longest Block of Ones 00101111001100011 1 1 1 Largest block so far.
When it has read this much, what does it remember? Read the next character & re-determine the largest block so far.

156 Longest Block of Ones 1 1 1 When it has read this much, what does it remember? Largest block so far. Largest current block. Read the next character & re-determine the largest block so far & current largest.

157 Typical Types of Loop Invariants
More of the input More of the output Narrowed the search space Case Analysis Work Done

158 More of the Output Loop Invariant
The output consists of an array of objects I have produced the first i objects. Produce the i+1st output object. Exit 79 km 75 km Exit Done when output n objects. i to i+1

159 Selection Sort Input: The output consists of an array of integers
52,23,88,31,25,30,98,62,14,79 The output consists of an array of integers 14,23,25,30,31,52,62,79,88,98 I have produced the first i objects. Produce the i+1st output object. Exit Done when output n objects.

160 Typical Types of Loop Invariants
More of the input More of the output Narrowed the search space Case Analysis Work Done Three Binary Search like examples

161 Define Problem: Binary Search
PreConditions Key Sorted List PostConditions Find key in list (if there). 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95

162 Define Loop Invariant Maintain a sub-list Such that key 25 3 5 6 13 18
21 25 36 43 49 51 53 60 72 74 83 88 91 95

163 Define Loop Invariant Maintain a sub-list.
If the key is contained in the original list, then the key is contained in the sub-list. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95

164 Define Step Make Progress Maintain Loop Invariant key 25 3 5 6 13 18
21 25 36 43 49 51 53 60 72 74 83 88 91 95

165 Define Step Cut sub-list in half.
Determine which half key would be in. Keep that half. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95

166 Define Step If key ≤ mid, then key is in left half. If key > mid,
Cut sub-list in half. Determine which half the key would be in. Keep that half. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

167 Define Step If key ≤ mid, then key is in left half. If key > mid,
It is faster not to check if the middle element is the key. Simply continue. key 43 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

168 Make Progress The size of the list becomes smaller. 3 5 6 13 18 21 25
79 km 75 km Exit The size of the list becomes smaller. 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 79 km 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 75 km

169 Initial Conditions If the key is contained in the original list,
3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 n km If the key is contained in the original list, then the key is contained in the sub-list. The sub-list is the entire original list.

170 Ending Algorithm Exit If the key is contained in the original list,
3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 0 km If the key is contained in the original list, then the key is contained in the sub-list. Sub-list contains one element. If the key is contained in the original list, then the key is at this location. Exit

171 If key not in original list
If the key is contained in the original list, then the key is contained in the sub-list. Loop invariant true, even if the key is not in the list. key 24 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If the key is contained in the original list, then the key is at this location. Conclusion still solves the problem. Simply check this one location for the key.

172 Running Time If key ≤ mid, then key is in left half. If key > mid,
The sub-list is of size n, n/2, n/4, n/8,…,1 Each step (1) time. Total = (log n) key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

173 Code

174 Algorithm Definition Completed
Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

175 Make Precise Definitions
Maintain a sub-list with end points i & j. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 i j

176 Make Precise Definitions
Maintain a sub-list with end points i & j. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 i j Does not matter which, but you need to be consistent.

177 Make Precise Definitions
If the sub-list has even length, which element is taken to be mid? key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95

178 Make Precise Definitions
If the sub-list has even length, which element is taken to be mid? mid key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 Should not matter Choose right.

179 Common Bugs If key ≤ mid, then key is in left half. If key > mid,
Cut sub-list in half. Determine which half the key would be in. Keep that half. key 25 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

180 Common Bugs If key ≤ mid, then key is in left half. If key > mid,
If the middle element is the key, it can be skipped over. Exit key 43 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

181 Common Bugs If key ≤ mid, then key is in left half. If key > mid,
Fix the bug. key 43 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

182 Common Bugs New bug? If key ≤ mid, then key is in left half.
Second fix, by making the left half slightly bigger. New bug? key 43 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

183 Common Bugs New bug? No progress is made. Loop for ever!
79 km 75 km Exit Second fix, by making the left half slightly bigger. New bug? key 43 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 3 5 6 13 18 21 25 36 43 49 51 53 60 72 74 83 88 91 95 No progress is made. Loop for ever!

184 Why is Binary Search so Easy to Get Wrong?
How many possible algorithms? How many correct algorithms? Probability of success by guessing? 1 2 i = mid else j = mid -1 i = mid + 1 else j = mid 3

185 Moral Use the loop invariant method to think about algorithms.
Be careful with your definitions. Be sure that the loop invariant is always maintained. Be sure progress is always made.

186 Loop Invariants for Iterative Algorithms
A second Binary Search-like example

187 Define Problem: Binary Search Tree
<preCond> Key A binary search tree. <postCond> Find key in BST (if there). 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71

188 Its left children ≤ Any node ≤ Its right children
Binary Search Tree Its left children ≤ Any node ≤ Its right children 38 25 51 17 31 42 63 4 21 28 35 40 49 55 71

189 Define Loop Invariant Maintain a sub-tree.
If the key is contained in the original tree, then the key is contained in the sub-tree. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17

190 Define Step Cut sub-tree in half.
Determine which half the key would be in. Keep that half. 38 25 17 4 21 31 28 35 51 42 40 49 63 55 71 key 17 If key < root, then key is in left half. If key = root, then key is found If key > root, then key is in right half.

191 Algorithm Definition Completed
Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

192 Loop Invariants for Iterative Algorithms
A third Binary Search-like example

193 Card Trick A volunteer, please.

194 Pick a Card Done

195 The selected card is one of these.
Loop Invariant: The selected card is one of these.

196 Which column? left

197 The selected card is one of these.
Loop Invariant: The selected card is one of these.

198 Selected column is placed
in the middle.

199 I will rearrange the cards.

200 I will remember the same about each column.
Relax Loop Invariant: I will remember the same about each column.

201 Which column? right

202 The selected card is one of these.
Loop Invariant: The selected card is one of these.

203 Selected column is placed
in the middle.

204 I will rearrange the cards.

205 Which column? left

206 The selected card is one of these.
Loop Invariant: The selected card is one of these.

207 Selected column is placed
in the middle.

208 Here is your card. Wow!

209 The “Partitioning” Problem (used in quicksort)
Input: Output: p=52 88 14 98 25 62 52 79 30 23 31 88 98 62 79 14 25 30 23 31 ≤ 52 ≤

210 Define Loop Invariant or p=52 31 23 25 98 30 14 62 79 88 ≤ p p ≤ p=52
71 25 98 30 62 79 88 ≤ p p ≤

211 Defining Measure of Progress
31 23 25 98 30 14 62 79 88 4 elements not looked at

212 Define Step Make Progress Maintain Loop Invariant p=52 31 23 25 98 30
14 62 79 88 ≤ p p ≤

213 Define Step Make Progress Maintain Loop Invariant p=52 31 23 25 98 30
14 62 79 88 ≤ p p ≤ p=52 31 23 14 25 98 30 62 79 88 ≤ p p ≤

214 Define Step Four cases p=52 p=52 31 23 25 98 30 14 62 79 88 31 23 14
71 62 79 88 31 23 71 25 98 30 62 79 88 31 23 25 98 30 71 62 79 88 31 23 25 98 30 71 62 79 88

215 Beginning & Ending p=52 88 14 98 25 62 52 79 30 23 31 p=52 88 25 31 98
0 km Exit Exit Beginning & Ending p=52 88 14 98 25 62 52 79 30 23 31 p=52 88 25 31 98 62 14 30 79 23 n-1 elements not looked at p=52 23 30 25 31 14 62 98 79 88 0 elements not looked at 62 79 98 88 25 30 31 14 23 ≤ 52 ≤

216 Each iteration takes (1) time. Total = (n)
Running Time Each iteration takes (1) time. Total = (n) p=52 31 23 25 98 30 14 62 79 88 ≤ p p ≤ p=52 31 23 14 25 98 30 62 79 88 ≤ p p ≤

217 Algorithm Definition Completed
Define Problem Define Loop Invariants Define Measure of Progress Define Step Define Exit Condition Maintain Loop Inv Make Progress Initial Conditions Ending 79 km to school Exit Exit 79 km 75 km Exit 0 km Exit Exit

218 23 79 30 14 62 98 52 25 88 31 p=52 23 79 30 14 62 98 31 25 88 79 30 14 62 98 31 25 88 23 88 79 30 14 62 98 31 25 23 88 79 30 14 62 98 31 25 23 Tracing an Example 88 79 14 62 98 31 25 30 23 88 79 14 62 98 31 25 30 23 88 79 14 62 98 31 25 30 23 88 79 98 14 62 31 25 30 23 88 79 98 62 14 31 25 30 23 88 79 98 62 14 31 25 30 23

219 Bucket (Quick) Sort for Humans
Input: A pile of things to sort. (Ex: Exams) Output: Sort them Requirements: Easy to execute by humans Only can have 7 piles on desk Can move one exam (or pile) at a time. Fast – O(nlog(n)) time. Likely the only algorithm in this course which you will execute by hand yourself.

220 Bucket (Quick) Sort for Humans
Input: [A-Z] Denotes an unsorted pile of exams with last names starting in the range [A-Z]

221 Bucket (Quick) Sort for Humans
[A-Z] Psychology study: Humans can only think about things at once. [A-E] [F-K] [L-O] [P-T] [U-Z] in Is the first letter of the name on the first exam

222 Bucket (Quick) Sort for Humans
[A-Z] [A-E] [F-K] [L-O] [P-T] [U-Z] Bucket Sort Put exams one at a time in the correct bucket.

223 Bucket (Quick) Sort for Humans
[A-E] [] [F-K] [L-O] sorted [P-T] [U-Z] A sorted pile before all the rest (upside down) A stack of piles each pile unsorted all in one pile before all in next

224 Bucket (Quick) Sort for Humans
[A-E] [] [F-K] [L-O] sorted [P-T] [U-Z] [A] [B] [C] [D] [E] Bucket Sort Put exams one at a time in the correct bucket.

225 Bucket (Quick) Sort for Humans
[] [B] [C] [D] sorted [E] [F-K] [L-O] [P-T] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

226 Bucket (Quick) Sort for Humans
[D] [E] [] sorted [F-K] [L-O] [P-T] [U-Z] [AA-AE] [AF-AK] [AL-AO] [AP-AT] [AU-AZ]

227 Bucket (Quick) Sort for Humans
[AA-AE] [] [AU-AZ] [B] sorted [E] [F-K] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

228 Bucket (Quick) Sort for Humans
[AA-AE] [] [AU-AZ] [B] sorted [E] [F-K] [U-Z] [AA-AE]

229 Bucket (Quick) Sort for Humans
[AF-AK] [] [AU-AZ] [B] sorted [E] [F-K] [U-Z] [AA-AE] When sufficiently small sort by hand sorted

230 Bucket (Quick) Sort for Humans
[AF-AK] [AA-AE] [AU-AZ] [B] sorted [E] [F-K] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

231 Bucket (Quick) Sort for Humans
[AF-AK] [AA-AE] [AU-AZ] [B] sorted [E] [F-K] [U-Z] [AF-AK] sorted

232 Bucket (Quick) Sort for Humans
[AL-AO] [AA-AK] [AU-AZ] [B] sorted [E] [F-K] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

233 Bucket (Quick) Sort for Humans
[AL-AO] [AA-AK] [AU-AZ] [B] sorted [E] [F-K] [U-Z] [AL-AO] [AP-AT] [AU-AZ] sorted sorted sorted

234 Bucket (Quick) Sort for Humans
[AA-AZ] [E] [F-K] sorted [U-Z]

235 Bucket (Quick) Sort for Humans
[A] [E] [F-K] sorted [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

236 Bucket (Quick) Sort for Humans
[E] [F-K] sorted [U-Z] [BA-BE] [BF-BK] [BL-BO] [BP-BT] [BU-BZ]

237 Bucket (Quick) Sort for Humans
[BA-BE] [A] [BU-BZ] [C] sorted [E] [F-K] [U-Z] A stack of piles each pile unsorted all in one pile before all in next A sorted pile before all the rest (upside down)

238 Bucket (Quick) Sort for Humans
[A-ZT] [ZU-ZZ] sorted [ZU-ZZ] sorted

239 Bucket (Quick) Sort for Humans
[A-Z] sorted Exit

240 RadixSort   Bucket Sort Machine: Select one digit Separates cards into k piles. wrt selected digit. Stable sort: If two cards are the same for that digit, their order does not change. Input: A of stack of N punch cards Each card contains d digits Each digit between 0&(k-1) Output: Sort the cards.

241 RadixSort   The most significant. The next most significant. Sort wrt which digit first? Sort wrt which digit Second? All meaning in first sort lost.

242 RadixSort   The least significant. The next least significant. Sort wrt which digit first? Sort wrt which digit Second?

243 RadixSort   The least significant. The next least significant. Sort wrt which digit first? Sort wrt which digit Second? Is sorted wrt first i digits.

244 RadixSort   Is sorted wrt first i digits. Is sorted wrt first i+1 digits. These are in the correct order because sorted wrt high order digit. Sort wrt i+1st digit. i+1

245 RadixSort   Is sorted wrt first i digits. Is sorted wrt first i+1 digits. These are in the correct order because was sorted & stable sort left sorted. Sort wrt i+1st digit. i+1

246 CountingSort Input: 1 3 2 Output: 1 2 3
3 2 Output: 1 2 3 Input: N records each labeled with a digit between 0&(k-1). Output: Stable sort the numbers. Algorithm: Count to determine where records go Go through the records in order putting them where they go.

247 CountingSort Input: Index: 1 3 2 Output: 1 2 3
11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 1 3 2 Output: 1 2 3 Stable sort: If two records are the same for that digit, their order does not change. Therefore, the 4th record in input with digit 1 must be the 4th record in output with digit 1. It belongs in output index 8, because 8 records go before it Count These ie 5 records with a smaller digit & records with the same digit

248 CountingSort Input: 1 3 2 Output: Index: Value v: 1 2 3
3 2 Output: Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 # of records with digit v: 2 3 9 5 N records, k different values. Time to count? (N× k) (N) We have counted # of each value in the first i values.

249 CountingSort Input: 1 3 2 Output: Index: Value v: 1 2 3
3 2 Output: Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 # of records with digit v: 2 3 9 5 # of records with digit < v: 5 14 17 N records, k different values. Time to count? (k2) Too much

250 CountingSort Input: 1 3 2 Output: Index: Value v: 1 2 3
3 2 Output: Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 # of records with digit v: 2 3 9 5 # of records with digit < v: 5 14 17 Computed N records, k different values. Time to count? (k)

251 CountingSort Input: 1 3 2 Location of first record with digit v. 1 2 3
3 2 Location of first record with digit v. 1 2 3 Output: Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 # of records with digit < v: 5 14 17

252 CountingSort Input: 1 3 2 Output: ? 1 Index: Value v: 1 2 3
3 2 Output: ? 1 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of first record with digit v. 5 14 17 Algorithm: Go through the records in order putting them where they go.

253 CountingSort Input: 1 3 2 Output: 1 Index: Value v: 1 2 3
3 2 Output: 1 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 5 14 17 Algorithm: Go through the records in order putting them where they go.

254 CountingSort Input: 1 3 2 Output: 1 Index: Value v: 1 2 3
3 2 Output: 1 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 6 14 17 Algorithm: Go through the records in order putting them where they go.

255 CountingSort Input: 1 3 2 Output: 1 Index: Value v: 1 2 3
3 2 Output: 1 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 1 6 14 17 Algorithm: Go through the records in order putting them where they go.

256 CountingSort Input: 1 3 2 Output: 1 1 Index: Value v: 1 2 3
3 2 Output: 1 1 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 6 14 17 Algorithm: Go through the records in order putting them where they go.

257 CountingSort Input: 1 3 2 Output: 1 1 3 Index: Value v: 1 2 3
3 2 Output: 1 1 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 7 14 17 Algorithm: Go through the records in order putting them where they go.

258 CountingSort Input: 1 3 2 Output: 1 1 1 3 Index: Value v: 1 2 3
3 2 Output: 1 1 1 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 7 14 18 Algorithm: Go through the records in order putting them where they go.

259 CountingSort Input: 1 3 2 Output: 1 1 1 1 3 Index: Value v: 1 2 3
3 2 Output: 1 1 1 1 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 8 14 18 Algorithm: Go through the records in order putting them where they go.

260 CountingSort Input: 1 3 2 Output: 1 1 1 1 3 3 Index: Value v: 1 2 3
3 2 Output: 1 1 1 1 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 9 14 18 Algorithm: Go through the records in order putting them where they go.

261 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 3 3 Index: Value v: 1 2 3
3 2 Output: 1 1 1 1 1 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 9 14 19 Algorithm: Go through the records in order putting them where they go.

262 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 3 3 Index: Value v: 1 2 3
3 2 Output: 1 1 1 1 1 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 2 10 14 19 Algorithm: Go through the records in order putting them where they go.

263 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 2 3 3 Index: Value v: 1 2
3 2 Output: 1 1 1 1 1 2 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 3 10 14 19 Algorithm: Go through the records in order putting them where they go.

264 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 1 2 3 3 Index: Value v: 1
3 2 Output: 1 1 1 1 1 1 2 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 3 10 15 19 Algorithm: Go through the records in order putting them where they go.

265 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 1 2 3 3 Index: Value v: 1
3 2 Output: 1 1 1 1 1 1 2 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 3 10 15 19 Algorithm: Go through the records in order putting them where they go.

266 CountingSort Input: 1 3 2 Output: 1 1 1 1 1 1 1 1 1 2 2 2 3 3 Index:
3 2 Output: 1 1 1 1 1 1 1 1 1 2 2 2 3 3 Index: 11 10 9 8 7 6 5 4 3 2 1 12 13 14 15 16 17 18 Value v: 1 2 3 Location of next record with digit v. 5 14 17 19 Algorithm: Go through the records in order putting them where they go. Time = (N) (Counting Ops) Total = (N+k) ×(log N + log k) (bit ops)

267 Radix/Counting Sort Input: N numbers, each L bits long. Output: Sort the numbers. N L d = L / log k N = log k (k will be set to N)

268 Radix/Counting Sort Input: N numbers, each L bits long Each card contains d digits Each digit between 0&(k-1) Output: Sort the numbers. Use Radix Sort: Sort wrt each digit using Counting Sort. d = L / log k N = log k digit between 0&(k-1).

269 Radix/Counting Sort

270 Radix/Counting Sort Time = (Time of Radix Sort)
= (# of digits) × (Time of Counting Sort) = L/log k × (N+k) × (log N + log k) bit ops Set k to minimize time. Wants k big. Really wants k small, but does not care as long as k  N. Set k=N

271 Radix/Counting Sort Time = (Time of Radix Sort)
= (# of digits) × (Time of Counting Sort) = L/log k × (N+k) × (log N + log k) bit ops = L/log N × N × log N bit ops = (L × N) bit ops = (n) bit ops Size = Size of N numbers, each L bits long. = (L × N) = n “Linear” Time But sorting should take (N log N) time!!!

272 Radix/Counting Sort Merge or Quick Sort Time = (L × N) bit ops
 (N log N) bit ops Size = Size of N numbers, each L bits long. = (L × N) L = > log N if you want N distinct numbers. Merge or Quick Sort Time = (N log N) comparisons = (n) Size = Size of N numbers, each arbitrarily long. = (N log N) bits.

273 End Iterative Algorithms
Math Review Recursive Algorithms

274 The Goal is to UNDERSTAND Algorithms
Fundamentally to their core

275 Rudich www.discretemath.com
Representation: Understand the relationship between different representations of the same information or idea 1 2 3 Rudich

276 Different Representations of Algorithms
Code Running example DFA and Turing Machines Higher level abstract view

277 Code Representation of an Algorithm
class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }} Pros and Cons?

278 Code Representation of an Algorithm
Pros: Cons: Runs on computers Precise and succinct Perception that being able to code is the only thing needed to get a job. (false) I am not a computer I need a higher level of intuition. Prone to bugs Language dependent

279 Running Example Representation of an Algorithm
Try out a problem or solution on small examples.

280 Running Example Representation of an Algorithm
88 88 52 98 14 14 31 98 25 30 62 23 79

281 Running Example Representation of an Algorithm
14,23,25,30,31,52,62,79,88,98 Pros and Cons?

282 Running Example Representation of an Algorithm
Pros: Cons: Concrete Dynamic Visual Relies on you to find the pattern. Does not explain why it works. Demonstrates for only one of many inputs.

283 DFA and Turing Machines Representation of an Algorithm
Pros and Cons?

284 DFA and Turing Machines Representation of an Algorithm
Pros: Cons: Good for theorems about algorithms Not good for designing algorithms

285 Higher Level Abstract View Representation of an Algorithm
T+1

286 Levels of Abstraction vs vs
It is hard to think of love in terms of the firing of neurons. vs Software developers view subsystems as entities with separate personalities, roles, and interactions, not details of code. vs

287 Higher Level Abstract View Representation of an Algorithm
Pros: Cons: Intuitive for humans Useful for thinking about designing describing algorithms View from which correctness is self evident. Mathematical mumbo jumbo Too abstract Students resist it Method of choice

288 Abstract away the inessential features of a problem
Value Simplicity Abstract away the inessential features of a problem = Goal: Understand and think about complex algorithm in simple ways. Don’t tune out. There are deep ideas within the simplicity.

289 Specifying a Computational Problem
Max( A ) “preCond: Input is array A[1,n] of n values.” i = 1 m = A[1] loop exit when (i=n) i = i m = max(m,A[i]) endloop return(m) “postCond: m is max in A[1,n]” Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns.

290 Definition of Correctness
<PreCond> & <code> Þ <PostCond> If the input meets the preconditions, then the output must meet the postconditions. If the input does not meet the preconditions, then nothing is required.

291 A Sequence of Assertions
<assertion0> if( <condition1> ) then code<1,true> else code<1,false> end if <assertion1> <assertionr-1> if( <conditionr> ) then code<r,true> code<r,false> <assertionr> Definition of Correctness <assertion0> any <conditions> code <assertionr> How is this proved?

292 A Sequence of Assertions
<assertion0> if( <condition1> ) then code<1,true> else code<1,false> end if <assertion1> <assertionr-1> if( <conditionr> ) then code<r,true> code<r,false> <assertionr> Definition of Correctness <assertion0> any <conditions> code <assertionr> Must check 2r different settings of <conditions> paths through the code. Is there a faster way?

293 A Sequence of Assertions
<assertion0> if( <condition1> ) then code<1,true> else code<1,false> end if <assertion1> <assertionr-1> if( <conditionr> ) then code<r,true> code<r,false> <assertionr> Step 1 <assertion0> <condition1> code<1,true> <assertion1> <assertion0> ¬<condition1> code<1,false> <assertion1>

294 A Sequence of Assertions
<assertion0> if( <condition1> ) then code<1,true> else code<1,false> end if <assertion1> <assertionr-1> if( <conditionr> ) then code<r,true> code<r,false> <assertionr> Step 2 <assertion1> <condition2> code<2,true> <assertion2> <assertion1> ¬<condition2> code<2,false> <assertion2>

295 A Sequence of Assertions
<assertion0> if( <condition1> ) then code<1,true> else code<1,false> end if <assertion1> <assertionr-1> if( <conditionr> ) then code<r,true> code<r,false> <assertionr> Step r <assertionr-1> <conditionr> code<r,true> <assertionr> <assertionr-1> ¬<conditionr> code<r,false> <assertionr>

296 A Sequence of Assertions
Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” Example <preCond> code <assertion0>

297 A Sequence of Assertions
Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” Example <assertion0> <condition1> code<1,true> <assertion1> <assertion0> ¬<condition1> code<1,false> <assertion1>

298 A Sequence of Assertions
Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” Example <assertion1> <condition2> code<2,true> <assertion2> <assertion1> ¬<condition2> code<2,false> <assertion2>

299 A Sequence of Assertions
Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = b endif “assert: m is max in {a,b}” if( c>m ) m = c “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}” Example <assertion2> code <postCond>


Download ppt "Iterative Algorithms & Loop Invariants"

Similar presentations


Ads by Google