Review of Searching and Sorting Algorithms CSE1322 Review of Searching and Sorting Algorithms 4/4/2019 CSE 1321 Module 8

Motivation To learn search algorithms (linear and binary) and simple sort algorithms (Bubble, Insertion, and Selection), and to understand how each algorithm processes data and the performance of each algorithm. 4/4/2019 CSE 1321 Module 8

Topics Linear Search Binary Search Bubble Sort Selection Sort Insertion Sort Merge Sort …. Quick Sort 4/4/2019 CSE 1321 Module 8

Linear Search It is the process of examining all values in a list (or array) until a target value is found or the search reaches the end of the list and no target is found. Number of visits for a linear search of a list (array) of size n elements - The average search visits n/2 elements - The maximum visits is n Linear search is to the order of n (i.e., O(n)) algorithm. Thus, the complexity is O(n), Not bad! 4/4/2019 CSE 1321 Module 8 4

Linear Search Algorithm Method LinearSearch using list G, search for element target BEGIN FOR each element temp in list G IF (temp == target) RETURN true ENDIF ENDFOR RETURN false END LinearSearch 4/26/2018 CSE 1321 Module 8 5

Java Linear Search Algorithm public bool LinearSearch (int [] G) { foreach (int temp in G) if (temp == find) return true; } return false; 4/26/2018 CSE 1321 Module 8 6

C# Linear Search Algorithm public bool LinearSearch (int [] G) { foreach (int temp in G) if (temp == find) return true; } return false; 4/26/2018 CSE 1321 Module 8 7

Python Linear Search Algorithm def linear_search(obj, item, start=0): for i in range(start, len(obj)): if obj[i] == item: return i return -1 numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] results = linear_search(numbers, 7, start=0) if results > 0: print("Number 7 found at: ", results) else: print("Number 7 not found!") 4/26/2018 CSE 1321 Module 8 8

Binary Search Binary search locates a value in a sorted list (array) by determining whether the value occurs in the first or second half of the list, then repeating the search process in one of the halves that may contain the target value being searched for. Binary search visits one element at a time (that is, the middle element in the list) then searches (recursively) either the left or right sub-list. Overall performance of Binary search is to the order of log (n) (i.e., O( log(n) ) algorithm. Good! Time complexity is O( log(n) ). Good! 4/4/2019 CSE 1321 Module 8

Binary Search Algorithm 4/26/2018 CSE 1321 Module 8

Binary Search Algorithm 4/26/2018 CSE 1321 Module 8

Binary Search Algorithm Method BinarySearch using array G, search for element target BEGIN low ← 0 high ← the number of elements in G mid ← 0 WHILE (true) mid ← (low + high) / 2 IF (target = G[mid]) RETURN true ELSE IF (target < G[mid]) high ← mid ELSE low ← mid ENDIF IF (mid+1 >= high) RETURN false ENDWHILE END BinarySearch 4/26/2018 CSE 1321 Module 8

Binary Search Algorithm 4/26/2018 CSE 1321 Module 8

Java Binary Search Algorithm public static boolean BinarySearch(int[] G, int target) { boolean found = false; int low = 0; int high = G.length; int mid = 0; while (!found) mid = (low + high) / 2; if (target == G[mid]) return true; else if (target < G[mid]) high = mid; else low = mid; if (low >= high - 1) return false; } return found; 4/26/2018 CSE 1321 Module 8

C# Binary Search Algorithm public static bool BinarySearch(int[] G, int target) { bool found = false; int low = 0; int high = G.Length; int mid = 0; while (!found) mid = (low + high) / 2; if (target == G[mid]) return true; else if (target < G[mid]) high = mid; else low = mid; if (low >= high - 1) return false; } return found; 4/26/2018 CSE 1321 Module 8

Python Linear Search Algorithm def binarySearch(alist, item): first = 0 last = len(alist)-1 found = False while first<=last and not found: midpoint = (first + last)//2 if alist[midpoint] == item: found = True else: if item < alist[midpoint]: last = midpoint-1 first = midpoint+1 return found testlist = [0, 1, 2, 8, 13, 17, 19, 32, 42,] print(binarySearch(testlist, 3)) print(binarySearch(testlist, 13)) 4/26/2018 CSE 1321 Module 8 13

Sorting Sorting is the process of re-arranging element to be in a specific order (Ascending or Descending) There are many ways to order elements within a list. Those are known as sort algorithms Sort algorithms vary in their complexity and performance. Thus, there is a trade-off between the complexity and performance of the algorithm A simple approach to sorting is slower than other methods that are more complex to code yet more efficient (faster) 4/4/2019 CSE 1321 Module 8 17

Bubble Sort Bubble sort (also known as Exchange sort) repeatedly pair-wise examine elements and swap them if needed This process “bubbles” place the elements to their correct positions in the list, thus called “Bubble” sort It’s a slow algorithm due the high number of comparisons and swaps it makes to sort the list. It is referred to as simple sort algorithm 4/4/2019 CSE 1321 Module 8 18

Bubble Sort Working (Version-A) Given List A = {8, 4, 2, 1} Let’s sort it in Ascending order The bubble sort finds the smallest value and then the next smallest etc. It does it by pair-wise comparisons and swaps. 8 and 4, 4 is smaller so it gets swapped, A = {4, 8, 2, 1} 4 and 2, 2 is smaller so it gets swapped, A = {2, 8, 4, 1} 2 and 1, 1 is smaller so it gets swapped, A = {1, 8, 4, 2} The first pass has been completed so now we start the same process starting with the second element in the list. 8 and 4, 4 is smaller, swap and A= {1, 4, 8, 2} 4 and 2, 2 is smaller, swap and A = {1, 2, 8, 4} That completes that pass and now the first 2 values are in the correct position. Now the third pass looks at 8 and 4 4 is smaller, swap and A= {1, 2, 4, 8} We’re done, A= {1, 2, 4, 8} 4-1 = 3 swaps 4-2 = 2 swaps 4-3 = 1 swap 4/4/2019 CSE 1321 Module 8 19

Bubble Sort Working (Version-B) Given List A = {8, 4, 2, 1} Let’s sort it in Ascending order 4/4/2019 CSE 1321 Module 8 20

Bubble Sort Algorithm Using list A of size n elements: FOR each I from 1 to n FOR each J from I+1 to n IF (A[J] < A[I]) temp ← A[J] A[J] ← A[I] A[I] ← temp ENDIF ENDFOR Notice that the extra variable temp is used to make the swap process easy without accidently misassigning values. 4/26/2018 CSE 1321 Module 8 17

Java Bubble Sort Algorithm for (int i = 0; i < A.length; i++) { for (int j = i+1; j < A.length; j++) if (A[j] < A[i]) int temp = A[j]; A[j] = A[i]; A[i] = temp; } //Notice that the extra variable temp is used to make the swap process easy without accidently misassigning values. 4/26/2018 CSE 1321 Module 8 18

C# Bubble Sort Algorithm for (int i = 0; i < A.Length; i++) { for (int j = i+1; j < A.Length; j++) if (A[j] < A[i]) int temp = A[j]; A[j] = A[i]; A[i] = temp; } //Notice that the extra variable temp is used to make the swap process easy without accidently misassigning values. 4/26/2018 CSE 1321 Module 8 19

Python Bubble Sort Algorithm def bubbleSort(unsorted): for i in range(len(unsorted)): for j in range(0,len(unsorted)-i-1): if unsorted[j] > unsorted[j+1]: temp = unsorted[j] unsorted[j] = unsorted[j+1] unsorted[j+1] = temp alist = [54,26,93,17,77,31,44,55,20] bubbleSort(alist) print(alist) 4/26/2018 CSE 1321 Module 8 20

Selection Sort Selection sort works as follows: Find the smallest value in the list and move to it to the first position in the list. This results in the first element being sorted Consider the remaining unsorted part of the list and apply the same process in step 1 Repeat step 2 on each un-sorted sub-list until the entire list is sorted 4/4/2019 CSE 1321 Module 8 25

Selection Sort Working Original list: 11 9 17 5 12 Find the smallest in the list and swap it with the first element 5 9 17 11 12 Find the next smallest in the unsorted sub-list and swap with second element. It is already in the correct place Find the next smallest in the unsorted sub-list and swap it with the third element. 5 9 11 17 12 Repeat 5 9 11 12 17 Notice that when the unsorted sub-list is of length 1, we are done 4/4/2019 CSE 1321 Module 8 26

Selection Sort Working arr[] = 64 25 12 22 11 // Find the minimum element in arr[0...4] // and place it at beginning 11 25 12 22 64 // Find the minimum element in arr[1...4] // and place it at beginning of arr[1...4] 11 12 25 22 64 // Find the minimum element in arr[2...4] // and place it at beginning of arr[2...4] 11 12 22 25 64 // Find the minimum element in arr[3...4] // and place it at beginning of arr[3...4] 4/4/2019 CSE 1321 Module 8 27

Selection Sort Algorithm Using list B of length n elements. FOR each I from 0 to n minPos ← I FOR each J from I + 1 to n IF (B[j] < B[minPos]) minPos ← J ENDIF ENDFOR IF (I != minPos AND minPos < n) temp ← B[minPos] B[minPos] ← B[I] B[I] ← temp 4/26/2018 CSE 1321 Module 8 23

Java Selection Sort Algorithm for (int i = 0; i < B.length; i++) { int minPos = i; for (int j = i + 1; j < B.length; j++) if (B[j] < B[minPos]) minPos = j; } if (i != minPos && minPos < B.length) int temp = B[minPos]; B[minPos] = B[i]; B[i] = temp; 4/26/2018 CSE 1321 Module 8 24

C# Selection Sort Algorithm for (int i = 0; i < B.Length; i++) { int minPos = i; for (int j = i + 1; j < B.Length; j++) if (B[j] < B[minPos]) minPos = j; } if (i != minPos && minPos < B.Length) int temp = B[minPos]; B[minPos] = B[i]; B[i] = temp; 4/26/2018 CSE 1321 Module 8 25

Python Selection Sort Algorithm def selectionSort(alist): for fillslot in range(len(alist)-1,0,-1): positionOfMax=0 for location in range(1,fillslot+1): if alist[location]>alist[positionOfMax]: positionOfMax = location temp = alist[fillslot] alist[fillslot] = alist[positionOfMax] alist[positionOfMax] = temp alist = [54,26,93,17,77,31,44,55,20] selectionSort(alist) print(alist) 4/26/2018 CSE 1321 Module 8 26

Bubble Sort and Selection Sort This algorithms are slow when run on large data sets because of the large number of swaps that are done during the processing. Both of these algorithms (and the insertion sort) are considered to be to the order of n squared (i.e., O(n2)). To keep things simple, consider the n to be the growth factor, so if a data set doubles in size, n would be 2; while if it tripled in size, n would be 3. O(n2) means that the time required grows by the square of the factor so if n was 2, the time would grow by about a factor of 4. If n was 3, by a factor of about 9. 4/4/2019 CSE 1321 Module 8 32

Insertion Sort Consider what you would do to sort a set of cards. This approach is an insertion sort - taking a new item and place it correctly relative to the other items in the "finished" portion.  You continually maintain two sets – unsorted set and a sorted set.  For each card in the unsorted set, take it out of that set and place it correctly relative to the sorted set. 4/4/2019 CSE 1321 Module 8 33

Insertion Sort 4/4/2019 CSE 1321 Module 8 34

Insertion Sort Working List A= { 8, 4, 2, 1} //assume indexing start from 0 the key = 4, index = 1 and position =1 4 < 8 and position > 0 so the while loop shifts 8 into the 4’s position. A= 8,8,2,1 then position-- so now position =0. The while loop stops. key (4) gets assigned to A[position] so A = 4, 8, 2, 1 Now index goes to 2, key =2 and position =2 2 < 8 and position > 0 so the while loops shifts 8 into the 2’s position. A = 4, 8, 8, 1 then position-- so now position =1. The while loop continues. 2< 4 so the while loops shifts 4 into the first 8’s position. A = 4, 4, 8, 1, position -- so now position=0, the while loop ends key (2) gets assigned to A[position] so A = 2, 4, 8, 1 Then one more pass with 1, shifting 8, 4, 2 right 1 space each so then A = 1, 2, 4, 8 4/4/2019 CSE 1321 Module 8 35

Insertion Sort Algorithm Using list A of size n elements FOR each index from 2 to n key ← A[index] position ← index // Shift larger values to the right WHILE (position > 0 AND key < A[position-1]) A[position] = A[position - 1] position ← position -1 ENDWHILE list [position] = key ENDFOR 4/26/2018 CSE 1321 Module 8 30

Java Insertion Sort Algorithm public void insertionSort (int[] list) { for (int index=1; index < list.length; index++) int key = list [index]; int position = index; // Shift larger values to the right while (position > 0 && key < list[position-1]) list [position] = list [position - 1]; position--; } list [position] = key; 4/26/2018 CSE 1321 Module 8 31

C# Insertion Sort Algorithm public void insertionSort (int[] list) { for (int index = 1; index < list.Length; index++) int key = list [index]; int position = index; // Shift larger values to the right while (position > 0 && key < list[position-1]) list [position] = list [position - 1]; position--; } list [position] = key; 4/26/2018 CSE 1321 Module 8 32

Python Insertion Sort Algorithm def insertionSort(alist): for index in range(1,len(alist)): currentvalue = alist[index] position = index while position>0 and alist[position- 1]>currentvalue: alist[position]=alist[position-1] position = position-1 alist[position]=currentvalue alist = [54,26,93,17,77,31,44,55,20] insertionSort(alist) print(alist) 4/26/2018 CSE 1321 Module 8 33

Sorting in a Java Program The Arrays and ArrayList classes contains sort methods. To use them, the data type you are sorting must be able to be naturally ordered or you must specify a comparator The Arrays class belongs to java.util.Arrays. int [] list1 = new int[10]; ArrayList <Person> errorList= new ArrayList<Person>(); Arrays.sort(list1); errorList.sort(null); 4/26/2018 CSE 1321 Module 8

Sorting in a C# Program The Array and List classes contains sort methods. To use them, the data type you are sorting must implement the IComparble interface. List<string> myList= new List<string>(); . . . myList.Sort(); To sort an array of integers int [] nums= new int [50]; … Array.Sort(nums); 4/26/2018 CSE 1321 Module 8

Sorting in a Python Program sorted(iterable[, cmp[, key[, reverse]]]) Return a new sorted list from the items in iterable. The optional arguments cmp, key, and reverse have the same meaning as those for the list.sort() method (described in section Mutable Sequence Types). cmp specifies a custom comparison function of two arguments (iterable elements) which should return a negative, zero or positive number depending on whether the first argument is considered smaller than, equal to, or larger than the second argument: cmp=lambda x,y: cmp(x.lower(), y.lower()). The default value is None. key specifies a function of one argument that is used to extract a comparison key from each list element: key=str.lower. The default value is None (compare the elements directly). reverse is a boolean value. If set to True, then the list elements are sorted as if each comparison were reversed. In general, the key and reverse conversion processes are much faster than specifying an equivalent cmp function. This is because cmp is called multiple times for each list element while key and reverse touch each element only once. Use functools.cmp_to_key() to convert an old-style cmp function to a key function. The built-in sorted() function is guaranteed to be stable. A sort is stable if it guarantees not to change the relative order of elements that compare equal — this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). 4/26/2018 CSE 1321 Module 8 36

Conclusion Search is the process of looking for a value in a list of values Searching can be done either in linear or binary fashion Linear approach is slow and takes to the order of O(n) Binary search is much faster and take to the order of O(log(n)) Bubble, Selection, and Insertion sort algorithms are relatively slow compared to other advanced sort method. They perform to the order of O(n2). Meaning that it takes about n2 swaps to sort a list of size n. The larger n and slower it gets. 4/4/2019 CSE 1321 Module 8