Central Algorithmic Techniques Iterative Algorithms

COSC 3101, PROF. J. ELDER 2 Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }} Pros and Cons?

COSC 3101, PROF. J. ELDER 3 Code Representation of an Algorithm Runs on computers Precise and succinct I am not a computer I need a higher level of intuition. Prone to bugs Language dependent Pros:Cons:

COSC 3101, PROF. J. ELDER 4 Two Key Types of Algorithms Iterative Algorithms Recursive Algorithms

COSC 3101, PROF. J. ELDER 5 Iterative Algorithms Take one step at a time towards the final destination loop (done) take step end loop

COSC 3101, PROF. J. ELDER 6 Loop Invariants A good way to structure many programs: –Store the key information you currently know in some data representation. –In the main loop, take a step forward towards destination by making a simple change to this data.

COSC 3101, PROF. J. ELDER 7 The Getting to School Problem

COSC 3101, PROF. J. ELDER 8 Problem Specification Pre condition: location of home and school Post condition: Traveled from home to school

COSC 3101, PROF. J. ELDER 9 General Principle Do not worry about the entire computation. Take one step at a time!

COSC 3101, PROF. J. ELDER 10 A Measure of Progress 79 km to school 75 km to school

COSC 3101, PROF. J. ELDER 11 Algorithm specifies from which locations it knows how to step. Safe Locations

COSC 3101, PROF. J. ELDER 12 “The computation is presently in a safe location.” May or may not be true. Loop Invariant

COSC 3101, PROF. J. ELDER 13 Defining Algorithm From every safe location, define one step towards school.

COSC 3101, PROF. J. ELDER 14 Take a step What is required of this step?

COSC 3101, PROF. J. ELDER 15 Can we be assured that the computation will always be in a safe location? If the computation is in a safe location, it does not step into an unsafe one. Maintain Loop Invariant No. What if it is not initially true?

COSC 3101, PROF. J. ELDER 16 From the Pre-Conditions on the input instance we must establish the loop invariant. Establishing Loop Invariant

COSC 3101, PROF. J. ELDER 17 Maintain Loop Invariant Can we be assured that the computation will always be in a safe location? By what principle?

COSC 3101, PROF. J. ELDER 18 Maintain Loop Invariant By Induction the computation will always be in a safe location.

COSC 3101, PROF. J. ELDER 19 Ending The Algorithm Define Exit Condition Termination: With sufficient progress, the exit condition will be met. When we exit, we know –exit condition is true –loop invariant is true from these we must establish the post conditions. Exit 0 kmExit

COSC 3101, PROF. J. ELDER 20 Let’s Recap

COSC 3101, PROF. J. ELDER 21 Designing an Algorithm Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

Simple Example Insertion Sort Algorithm

COSC 3101, PROF. J. ELDER 23 Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }}

COSC 3101, PROF. J. ELDER 24 Higher Level Abstract View Representation of an Algorithm 9 km 5 km

COSC 3101, PROF. J. ELDER 25 Designing an Algorithm Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

COSC 3101, PROF. J. ELDER 26 Problem Specification Precondition: The input is a list of n values with the same value possibly repeated. Post condition: The output is a list consisting of the same n values in non-decreasing order ,23,25,30,31,52,62,79,88,98

COSC 3101, PROF. J. ELDER ,23,25,30,31,52,62,79,88, ,31,52,88 Some subset of the elements are sorted The remaining elements are off to the side. Define Loop Invariant

COSC 3101, PROF. J. ELDER 28 Defining Measure of Progress ,31,52,88 6 elements to school

COSC 3101, PROF. J. ELDER 29 Define Step Select arbitrary element from side. Insert it where it belongs ,31,52, ,31,52,62,

COSC 3101, PROF. J. ELDER 30 Making progress while Maintaining the loop invariant ,31,52, ,31,52,62, km75 km 5 elements to school 6 elements to school Exit Sorted sublist

COSC 3101, PROF. J. ELDER n elements to school 14,23,25,30,31,52,62,79,88,98 0 elements to school Beginning & Ending Exit 0 kmExit

COSC 3101, PROF. J. ELDER 32 Running Time Inserting an element into a list of size i takes  (i) time. Total = …+n =  (n 2 ) ,31,52, ,31,52,62,

COSC 3101, PROF. J. ELDER 33 Ok I know you knew Insertion Sort But hopefully you are beginning to appreciate Loop Invariants for describing algorithms

Assertions in Algorithms

COSC 3101, PROF. J. ELDER 35 Purpose of Assertions Useful for – thinking about algorithms – developing – describing – proving correctness

COSC 3101, PROF. J. ELDER 36 Definition of Assertions An assertion is a statement about the current state of the data structure that is either true or false. eg. the amount in your bank account is not negative.

COSC 3101, PROF. J. ELDER 37 Definition of Assertions It is made at some particular point during the execution of an algorithm. If it is false, then something has gone wrong in the logic of the algorithm.

COSC 3101, PROF. J. ELDER 38 Definition of Assertions An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader.

COSC 3101, PROF. J. ELDER 39 Specifying a Computational Problem Example of Assertions Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns..

COSC 3101, PROF. J. ELDER 40 Definition of Correctness &  If the input meets the preconditions, then the output must meet the postconditions. If the input does not meet the preconditions, then nothing is required.

COSC 3101, PROF. J. ELDER 41 An Example: A Sequence of Assertions if( ) then code else code end if if( ) then code else code end if … any code How is this proved? Definition of Correctness Must check 2 r different settings of paths through the code. Is there a faster way?

COSC 3101, PROF. J. ELDER 42 An Example: A Sequence of Assertions if( ) then code else code end if if( ) then code else code end if … Step 1 code ¬ code

COSC 3101, PROF. J. ELDER 43 An Example: A Sequence of Assertions if( ) then code else code end if if( ) then code else code end if … Step 2 code ¬ code

COSC 3101, PROF. J. ELDER 44 An Example: A Sequence of Assertions if( ) then code else code end if if( ) then code else code end if … Step r code ¬ code

COSC 3101, PROF. J. ELDER 45 A Sequence of Assertions if( ) then code else code end if if( ) then code else code end if Step r code ¬ code …

COSC 3101, PROF. J. ELDER 46 Another Example: A Loop codeA loop exit when codeB endloop codeC Type of Algorithm: Iterative Type of Assertion: Loop Invariants

COSC 3101, PROF. J. ELDER 47 codeA loop exit when codeB endloop codeC Definition of Correctness? Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 48 codeA loop exit when codeB endloop codeC Definition of Correctness any code How is this proved? Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 49 codeA loop exit when codeB endloop codeC The computation may go around the loop an arbitrary number of times. any code Is there a faster way? Iterative Algorithms Loop Invariants Definition of Correctness

COSC 3101, PROF. J. ELDER 50 codeA loop exit when codeB endloop codeC Step 0 codeA Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 51 codeA loop exit when codeB endloop codeC Step 1 ¬ codeB Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 52 codeA loop exit when codeB endloop codeC Step 2 ¬ codeB Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 53 codeA loop exit when codeB endloop codeC Step 3 ¬ codeB Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 54 codeA loop exit when codeB endloop codeC Step i ¬ codeB Iterative Algorithms Loop Invariants All these steps are the same and therefore only need be done once!

COSC 3101, PROF. J. ELDER 55 codeA loop exit when codeB endloop codeC Last Step codeC Iterative Algorithms Loop Invariants

COSC 3101, PROF. J. ELDER 56 Partial Correctness Proves that IF the program terminates then it works &  codeC Clean up loose ends ¬ codeB Maintaining Loop Invariant codeA Establishing Loop Invariant Exit

COSC 3101, PROF. J. ELDER 57 Algorithm Termination 79 km75 km Measure of progress 0 km Exit

COSC 3101, PROF. J. ELDER 58 Algorithm Correctness Partial Correctness + Termination Correctness

COSC 3101, PROF. J. ELDER 59 Designing Loop Invariants Coming up with the loop invariant is the hardest part of designing an algorithm. It requires practice, perseverance, and insight. Yet from it the rest of the algorithm follows easily

COSC 3101, PROF. J. ELDER 60 Don’t start coding You must design a working algorithm first.

COSC 3101, PROF. J. ELDER 61 Exemplification: Try solving the problem on small input examples.

COSC 3101, PROF. J. ELDER 62 Start with Small Steps What basic steps might you follow to make some kind of progress towards the answer? Describe or draw a picture of what the data structure might look like after a number of these steps.

COSC 3101, PROF. J. ELDER 63 Picture from the Middle Leap into the middle of the algorithm. What would you like your data structure to look like when you are half done?

COSC 3101, PROF. J. ELDER 64 Ask for 100% Pretend that a genie has granted your wish. –You are now in the middle of your computation and your dream loop invariant is true.

COSC 3101, PROF. J. ELDER 65 Ask for 100% Maintain the Loop Invariant: – From here, are you able to take some computational steps that will make progress while maintaining the loop invariant?

COSC 3101, PROF. J. ELDER 66 Ask for 100% If you can maintain the loop invariant, great. If not, –Too Weak: If your loop invariant is too weak, then the genie has not provided you with everything you need to move on. –Too Strong: If your loop invariant is too strong, then you will not be able to establish it initially or maintain it.

COSC 3101, PROF. J. ELDER 67 Differentiating between Iterations x=x+2 –Meaningful as code –False as a mathematical statement x’ = x i = value at the beginning of the iteration x’’ = x i+1 = new value after going around the loop one more time. x'' = x'+2 –Meaningful as a mathematical statement

Loop Invariants for Iterative Algorithms Three Search Examples

COSC 3101, PROF. J. ELDER 69 Define Problem: Binary Search PreConditions –Key 25 –Sorted List PostConditions –Find key in list (if there)

COSC 3101, PROF. J. ELDER 70 Define Loop Invariant Maintain a sublist. If the key is contained in the original list, then the key is contained in the sublist. key

COSC 3101, PROF. J. ELDER 71 Define Step Make Progress Maintain Loop Invariant key

COSC 3101, PROF. J. ELDER 72 Define Step Cut sublist in half. Determine which half the key would be in. Keep that half. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. mid

COSC 3101, PROF. J. ELDER 73 Define Step It is faster not to check if the middle element is the key. Simply continue. key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

COSC 3101, PROF. J. ELDER 74 Make Progress The size of the list becomes smaller km75 km 79 km75 km Exit

COSC 3101, PROF. J. ELDER 75 Initial Conditions key If the key is contained in the original list, then the key is contained in the sublist. The sublist is the entire original list. n km

COSC 3101, PROF. J. ELDER 76 Ending Algorithm If the key is contained in the original list, then the key is contained in the sublist. Sublist contains one element. Exit km If the key is contained in the original list, then the key is at this location. key 25

COSC 3101, PROF. J. ELDER 77 If key not in original list If the key is contained in the original list, then the key is contained in the sublist. Loop invariant true, even if the key is not in the list. If the key is contained in the original list, then the key is at this location key 24 Conclusion still solves the problem. Simply check this one location for the key.

COSC 3101, PROF. J. ELDER 78 Running Time The sublist is of size n, n / 2, n / 4, n / 8,…,1 Each step  (1) time. Total =  (log n) key If key ≤ mid, then key is in left half. If key > mid, then key is in right half.

COSC 3101, PROF. J. ELDER 79

COSC 3101, PROF. J. ELDER 80 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

COSC 3101, PROF. J. ELDER 81

COSC 3101, PROF. J. ELDER 82 Simple, right? Although the concept is simple, binary search is notoriously easy to get wrong. Why is this?

COSC 3101, PROF. J. ELDER 83 The Devil in the Details The basic idea behind binary search is easy to grasp. It is then easy to write pseudocode that works for a ‘typical’ case. Unfortunately, it is equally easy to write pseudocode that fails on the boundary conditions.

COSC 3101, PROF. J. ELDER 84 The Devil in the Details or What condition will break the loop invariant?

COSC 3101, PROF. J. ELDER 85 The Devil in the Details key mid Bug!!

COSC 3101, PROF. J. ELDER 86 The Devil in the Details OK Not OK!!

COSC 3101, PROF. J. ELDER 87 key The Devil in the Details or Shouldn’t matter, right?

COSC 3101, PROF. J. ELDER The Devil in the Details key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. mid

COSC 3101, PROF. J. ELDER The Devil in the Details key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. mid

COSC 3101, PROF. J. ELDER The Devil in the Details key If key ≤ mid, then key is in left half. If key > mid, then key is in right half. Another bug! No progress toward goal: Loops Forever! mid 79 km75 km Exit

COSC 3101, PROF. J. ELDER 91 The Devil in the Details OK Not OK!!

COSC 3101, PROF. J. ELDER 92 How Many Possible Algorithms?

COSC 3101, PROF. J. ELDER 93 Alternative Algorithm: Less Efficient but More Clear

COSC 3101, PROF. J. ELDER 94 Moral Use the loop invariant method to think about algorithms. Be careful with your definitions. Be sure that the loop invariant is always maintained. Be sure progress is always made. Having checked the ‘typical’ cases, pay particular attention to boundary conditions and the end game.

Loop Invariants for Iterative Algorithms A Second Search Example: The Binary Search Tree

COSC 3101, PROF. J. ELDER Define Problem: Binary Search Tree PreConditions –Key 25 –A binary search tree. –PostConditions –Find key in BST (if there).

COSC 3101, PROF. J. ELDER Binary Search Tree All nodes in left subtree ≤ Any node ≤ All nodes in right subtree ≤ ≤

COSC 3101, PROF. J. ELDER 98 Define Loop Invariant Maintain a sub-tree. If the key is contained in the original tree, then the key is contained in the sub-tree. key

COSC 3101, PROF. J. ELDER 99 Define Step Cut sub-tree in half. Determine which half the key would be in. Keep that half. key If key < root, then key is in left half. If key > root, then key is in right half. If key = root, then key is found

COSC 3101, PROF. J. ELDER 100 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

End of Lecture 4 Sept 20, 2007

COSC 3101, PROF. J. ELDER 102 A volunteer, please. Card Trick

Loop Invariants for Iterative Algorithms A Third Search Example: A Card Trick

COSC 3101, PROF. J. ELDER 104 Pick a Card Done

COSC 3101, PROF. J. ELDER 105 Loop Invariant: The selected card is one of these.

COSC 3101, PROF. J. ELDER 106 Which column? left

COSC 3101, PROF. J. ELDER 107 Loop Invariant: The selected card is one of these.

COSC 3101, PROF. J. ELDER 108 Selected column is placed in the middle

COSC 3101, PROF. J. ELDER 109 I will rearrange the cards

COSC 3101, PROF. J. ELDER 110 Relax Loop Invariant: I will remember the same about each column.

COSC 3101, PROF. J. ELDER 111 Which column? right

COSC 3101, PROF. J. ELDER 112 Loop Invariant: The selected card is one of these.

COSC 3101, PROF. J. ELDER 113 Selected column is placed in the middle

COSC 3101, PROF. J. ELDER 114 I will rearrange the cards

COSC 3101, PROF. J. ELDER 115 Which column? left

COSC 3101, PROF. J. ELDER 116 Loop Invariant: The selected card is one of these.

COSC 3101, PROF. J. ELDER 117 Selected column is placed in the middle

COSC 3101, PROF. J. ELDER 118 Here is your card. Wow!

COSC 3101, PROF. J. ELDER 119 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

COSC 3101, PROF. J. ELDER 120 Ternary Search How many iterations are required to guarantee success? Loop Invariant: selected card in central subset of cards

Loop Invariants for Iterative Algorithms A Fourth Example: Partitioning (Not a search problem: can be used for sorting, e.g., Quicksort)

COSC 3101, PROF. J. ELDER 122 The “Partitioning” Problem Input: ≤ 52 ≤ Output: x=52 Problem: Partition a list into a set of small values and a set of large values.

COSC 3101, PROF. J. ELDER 123 Precise Specification p r p r q

COSC 3101, PROF. J. ELDER subsets are maintained –One containing values less than or equal to the pivot –One containing values greater than the pivot –One containing values yet to be processed Loop Invariant

COSC 3101, PROF. J. ELDER 125 Maintaining Loop Invariant Consider element at location j –If greater than pivot, incorporate into ‘> set’ by incrementing j. –If less than or equal to pivot, incorporate into ‘≤ set’ by swapping with element at location i+1 and incrementing both i and j. –Measure of progress: size of unprocessed set.

COSC 3101, PROF. J. ELDER 126 Maintaining Loop Invariant

COSC 3101, PROF. J. ELDER 127 Establishing Loop Invariant

COSC 3101, PROF. J. ELDER 128 Establishing Postcondition Exhaustive on exit

COSC 3101, PROF. J. ELDER 129 Establishing Postcondition

COSC 3101, PROF. J. ELDER 130 An Example

COSC 3101, PROF. J. ELDER 131 Running Time Each iteration takes  (1) time  Total =  (n) or

COSC 3101, PROF. J. ELDER 132 Algorithm Definition Completed Define ProblemDefine Loop Invariants Define Measure of Progress Define StepDefine Exit ConditionMaintain Loop Inv Make ProgressInitial ConditionsEnding 79 km to school Exit 79 km75 km Exit 0 kmExit

More Examples of Iterative Algorithms Using Constraints on Input to Achieve Linear- Time Sorting

COSC 3101, PROF. J. ELDER 134 Recall: InsertionSort

COSC 3101, PROF. J. ELDER 135 Recall: MergeSort

COSC 3101, PROF. J. ELDER 136 Comparison Sorts InsertionSort and MergeSort are examples of (stable) Comparison Sort algorithms. QuickSort is another example we will study shortly. Comparison Sort algorithms sort the input by successive comparison of pairs of input elements. Comparison Sort algorithms are very general: they make no assumptions about the values of the input elements.

COSC 3101, PROF. J. ELDER 137 Comparison Sorts Can we do better?

COSC 3101, PROF. J. ELDER 138 Comparison Sort: Decision Trees Example: Sorting a 3-element array A[1..3]

COSC 3101, PROF. J. ELDER 139 Comparison Sort Worst-case time is equal to the height of the binary decision tree. The height of the tree is the log of the number of leaves. The leaves of the tree represent all possible permutations of the input. How many are there? Thus MergeSort is asymptotically optimal.

COSC 3101, PROF. J. ELDER 140 Linear Sorts?

COSC 3101, PROF. J. ELDER 141 Example 1. Counting Sort Counting Sort applies when the elements to be sorted come from a finite (and preferably small) set. For example, the elements to be sorted are integers in the range [0…k-1], for some fixed integer k. We can then create an array V[0…k-1] and use it to count the number of elements with each value [0…k-1]. Then each input element can be placed in exactly the right place in the output array in constant time.

COSC 3101, PROF. J. ELDER 142 Counting Sort Input: N records with integer keys between [0…k-1]. Output: Stable sorted keys. Algorithm: –Count frequency of each key value to determine transition locations –Go through the records in order putting them where they go. Input: Output:

COSC 3101, PROF. J. ELDER 143 CountingSort Stable sort: If two keys are the same, their order does not change. Input: Output: Thus the 4 th record in input with digit 1 must be the 4 th record in output with digit 1. It belongs at output index 8, because 8 records go before it ie, 5 records with a smaller digit & 3 records with the same digit Count These! Index:

COSC 3101, PROF. J. ELDER 144 CountingSort Input: Output: Index: Value v: # of records with digit v: N records. Time to count?  (N)

COSC 3101, PROF. J. ELDER 145 CountingSort Input: Output: Index: Value v: # of records with digit v: # of records with digit < v: N records, k different values. Time to count?  (k)

COSC 3101, PROF. J. ELDER 146 CountingSort Input: Output: Index: Value v: # of records with digit < v: = location of first record with digit v

COSC 3101, PROF. J. ELDER 147 CountingSort Input: Output: Index: Value v: Location of first record with digit v. Algorithm: Go through the records in order putting them where they go. 1 0 ?

COSC 3101, PROF. J. ELDER 148 Loop Invariant The first i-1 keys have been placed in the correct locations in the output array The auxiliary data structure v indicates the location at which to place the i th key for each possible key value from [1..k-1].

COSC 3101, PROF. J. ELDER 149 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 1 Algorithm: Go through the records in order putting them where they go.

COSC 3101, PROF. J. ELDER 150 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 0 Algorithm: Go through the records in order putting them where they go. 1

COSC 3101, PROF. J. ELDER 151 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 00 Algorithm: Go through the records in order putting them where they go. 1

COSC 3101, PROF. J. ELDER 152 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go. 1 0

COSC 3101, PROF. J. ELDER 153 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 154 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 155 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 156 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 157 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 158 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 159 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 160 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 161 CountingSort Input: Output: Index: Value v: Location of next record with digit v. 01 Algorithm: Go through the records in order putting them where they go

COSC 3101, PROF. J. ELDER 162 CountingSort Input: Output: Index: Value v: Location of next record with digit v  (N) Time =  (N+k) Total =

End of Lecture 5 Sept 25, 2007

COSC 3101, PROF. J. ELDER 164 Input: A of stack of N punch cards. Each card contains d digits. Each digit between [0…k-1] Output: Sorted cards. Example 2. RadixSort Digit Sort: Select one digit Separate cards into k piles based on selected digit (e.g., Counting Sort) Stable sort: If two cards are the same for that digit, their order does not change.

COSC 3101, PROF. J. ELDER 165 RadixSort Sort wrt which digit first? The most significant Sort wrt which digit Second? The next most significant All meaning in first sort lost.

COSC 3101, PROF. J. ELDER 166 RadixSort Sort wrt which digit first? Sort wrt which digit Second? The least significant The next least significant

COSC 3101, PROF. J. ELDER 167 RadixSort Sort wrt which digit first? Sort wrt which digit Second? The least significant The next least significant Is sorted wrt least sig. 2 digits.

COSC 3101, PROF. J. ELDER 168 Sort wrt i+1st digit Is sorted wrt first i digits Is sorted wrt first i+1 digits. i+1 These are in the correct order because sorted wrt high order digit RadixSort

COSC 3101, PROF. J. ELDER 169 Sort wrt i+1st digit Is sorted wrt first i digits i+1 These are in the correct order because was sorted & stable sort left sorted Is sorted wrt first i+1 digits. RadixSort

COSC 3101, PROF. J. ELDER 170 Loop Invariant The keys have been correctly stable-sorted with respect to the i-1 least-significant digits.

COSC 3101, PROF. J. ELDER 171 Running Time

COSC 3101, PROF. J. ELDER 172 Example 3. Bucket Sort Applicable if input is constrained to finite interval, e.g., [0…1). If input is random and uniformly distributed, expected run time is Θ(n).

COSC 3101, PROF. J. ELDER 173 Bucket Sort

COSC 3101, PROF. J. ELDER 174 Loop Invariants Loop 1 –The first i-1 keys have been correctly placed into buckets of width 1/n. Loop 2 –The keys within each of the first i-1 buckets have been correctly stable-sorted.

COSC 3101, PROF. J. ELDER 175 PseudoCode

COSC 3101, PROF. J. ELDER 176 Examples of Iterative Algorithms Binary Search Partitioning Insertion Sort Counting Sort Radix Sort Bucket Sort Which can be made stable? Which sort in place? How about MergeSort?

End of Iterative Algorithms