The Linear and Binary Search and more Lecture Notes 9

Linear Search Suppose we have a vector of ints How do we search for a target Write the code to do this Analyze your code Is it working under all conditions? Have you tested all possible inputs?

Linear Search Algorithm Public int linearSearch(int target){ for (int I=0;I<A.size();I++) if (A.elementAt(I)==target) return i; return -1; }

Linear Search -analysis Worst case is target item not in the list or at the end – N comparisons – Order(n) Best case is target is at the first position – Order(1) complexity Average case – (n+(n-1)+(n-2)+…+1)/n comparisons – Average complexity is order(n)

Binary Search Linear Search is highly time consuming for arrays(vectors) that are large So what if the array is sorted? Is there a better way to do the search Answer: yes Look at the middle element – If the target is smaller, look to the left sub-array – If the target is larger, look to the right sub-array

Binary Search ctd.. Consider The middle element is (n+1)/2 If the target is 15, then – Search left – Else search right Write the code Analyze the algorithm

Binary Search Algorithm public int binarySearch(Object target){ int first=0, last=A.size()-1; while (first<=last){ middle=(first+last)/2; if (A.elementAt(middle) > target) {last=middle-1;} else if (A.elementAt(middle) < target) {first=middle+1;} else return middle; } return -1 }

Analysis of the binary Search

Other Search Algorithms Breadth-first search is one of the two most common search algorithms The approach of breadth-first search is to start with a queue containing a list of nodes to visit A node is a state or a value; in programming it is usually represented as a structure containing particular information about the environment or domain of the problem.

Other Search Algorithms ctd.. The algorithm starts by placing the initial state of the problem into the head (beginning) of the queue The search then proceeds by visiting the first node and adding all nodes connected to that node to the queue The search then proceeds by visiting the first node and adding all nodes connected to that node to the queue. When viewed as a tree graph, it would move from left to right from the current node (usually represented as a circle on a graph) along links (represented as connecting lines in between the node circles) to connected nodes, adding the connected nodes to the queue. As the head node of the queue is visited it is removed. The search then moves to the next node on the queue and continues until the goal is reached.

Sorts Many programs will execute more efficiently if the data they process is sorted before processing begins. – We first looked at a linear search it doesn’t care whether the data is sorted or not the algorithm starts at the first element in the vector, and looks at every element in the vector, in sequence, until it finds what it is looking for, or comes to the end of the vector – With small data sets this algorithm performs acceptably – If the data sets are of significant size, than performance can become unacceptable We have since looked at Binary Search, which improves performance, IF the data is sorted

Sorts If we know that a data set is sorted in some order (lowest to greatest, largest to smallest, highest priority to lowest priority), then we can write searches to take advantage of this fact. – If you have misplaced your calculator in the library in the afternoon, you do not have to retrace your steps from when you arrived on campus in the morning to find it – You start your search in the library – When looking up a phone number in the phone book, you do not start at page 1 and scan each page in sequence, until you find the name you are looking for

Sorting Comparison There are all kinds of sorts For our purposes a sort is a rearrangement of data into either ascending or descending order Fast Sortsversus Slow Sorts O (N log 2 N)O (N 2 ) Slow sorts are easy to code and sufficient when the amount of data is small

Bubble Sort - N 2 Sort Strategy – ‘bubble’ the smallest item to the left (slot 0) – ‘bubble’ the next smallest item to slot 1 – ‘bubble’ the third smallest item to slot 2 algorithm (for one pass) walk through the Vector if this item (in spot j) is smaller than the item in spot 0 swap item in spot j with the item in spot 0 code for (int j = 0; j < count; j++) { if (nums[j] < nums[0] { int temp = nums[j]; nums[j] = nums[0]; nums[0] = temp; }

Bubble Sort (con’t) This code for one pass finds the smallest element in the vector and puts it into slot 0 Now wrap this code in a loop that will find succeeding smaller elements and put them into the proper position in the Vector // outer for loop controls the spot that gets the appropriate smallest // value for (int left = 0; left < count - 1; left++) { // code from previous slide, with starting point being left + 1 // instead of 0, and comparing nums[j] to nums[left] for (int j = left + 1; j < count; j++) if (nums[j] < nums[left] { int temp = nums[j]; nums[j] = nums[left]; nums[left] = nums[j]; }

Selection Sort We have noticed that the most “expensive” part of the bubble sort is swapping (three lines of code to execute) A selection sort reduces the number of swaps until the end of each pass, when we know what the smallest remaining value is, and placing it appropriately Strategy – find the smallest item, and swap it with slot 0 – find the next smallest item and swap it with slot 1 – find the third smallest item and swap it with slot 2 algorithm (for one pass) walk through the Vector the first item is labeled the smallest Every other element is compared to the smallest if it is smaller, than it is labeled the smallest At the end of the walkthrough, the first is swapped with the smallest

Selection Sort (con’t) The algorithm for one pass finds the smallest element in the vector and puts it into slot 0 Now wrap this code in a loop that will find succeeding smaller elements and put them into the proper position in the Vector // outer for loop controls the spot that gets the appropriate smallest // value (same as bubble sort) for (int left = 0; left < count - 1; left++) { // last one will be correct int smallest = left;// we will keep the index to the smallest for (int j = left + 1; j < count; j++) if (nums[j] < nums[smallest] { smallest = j; } if (smallest != left) // no sense swapping if left is smallest { int temp = nums[smallest]; nums[smallest] = nums[left]; nums[left] = temp; }

Use FindSmallest() routine with SelectionSort() We have spent a lot of time looking at FindSmallest() routines… you should know how to do that Incorporate that knowledge into selection sort // outer for loop controls the spot that gets the appropriate smallest // value (same as bubble sort) for (int left = 0; left < count - 1; left++) { int smallest = findSmallest(nums, count, left); // use what we already know // pass the starting point of the // remainder of the vector to look // at if (smallest != left ) { int temp = nums[smallest]; nums[smallest] = nums[left]; nums[left] = temp; }

findSmallest(const int nums[], int count, int left); int findSmallest(const int nums[], int count, int left) { int smallest = left; // start with first index as the smallest for (int j = left + 1; j < count; j++) if (nums[j] < nums[smallest] { smallest = j; } return smallest; }

Quick Sort Most widely used algorithm Invented in 1960 by C.A.R. Hoare Not difficult to implement Good general purpose sort (works well in most cases) Average performance is n log n Reecursive (drawback) Worst case performance is n 2 (why is that????) – file already sorted time = n 2 /2 and space = n precise mathematical analysis backed up by empirical results Divide and conquer algorithm

Quick Sort Algorithm – Partitioning Step Choose a pivot element say a = v[j] Determine its final position in the sorted array – a > v[I] for all I < j – a j – Recursive Step Perform above step on left array and right array An early look at quicksort code (incomplete) void quicksort( vector &A, int left, int right) { int I; if (right > left) { Pivot(A, left, right); I = partition(A, left, right); quicksort(A, left, I-1); quicksort(A, I+1, right); }

Quick Sort Code ctd.. More Detailed look at the partition code // Partition(): rearrange A into 3 sublists, a sublist // A[left] Ö A[j-1] of values at most A[j], a sublist A[j], // and a sublist A[j+1] Ö A[right] of values at least A[j] int Partition(vector &A, int left, int right) { char pivot = A[left]; int i = left; int j = right+1; do { do ++i; while (A[i] < pivot); do --j; while (A[j] > pivot); if (i < j) { Swap(A[i], A[j]); } } while (i < j); Swap(A[j], A[left]); return j; }

Quick Sort Code ctd.. More Detailed look at the pivot code // Pivot(): prepare A for partitioning void Pivot(vector &A, int left, int right) { if (A[left] > A[right]) Swap(A[left], A[right]); } eg: trace the quick sort code for

Quick Sort Analysis Divide the file by half Recurrence relation is – C(n) = 2C(n/2) + n – 2C(n/2) is the cost of doing two partitions of size n/2 – n is the cost of examining each element Using the above recurrence relation, prove that quick sort is of n log n Let us assume that n = 2 k for some k without the loss of generality C(n)= 2C(n/2) + n = 2 (2C(n/4) + n/2) + n = 2 (2(2C(n/8)+n/4) + n/2) + n = 2 3 C(n/2 3 ) + 3 n ….. After 3 iterations So after k iterations we have C(n) = 2 k C(n/2 k ) + k. n = 2 k C(1) + k. n since n = 2 k = 2 log n. 1 + k.n = n + n log n = f(n log n) In Lab 5 you must use the recursive quicksort