1. CHAPTER 09 Compiled by: Dr. Mohammad Omar Alhawarat Sorting & Searching

2. Content  Sorting:  Bubble Sort.  Insertion Sort.  Selection Sort.  Merge Sort.  Quicksort  Searching:  Sequential Search.  Binary Search.  Hashing. 2

3. Sorting 3

4.  Definition: Rearranging the values into a specific order: (Ascending OR Descending).  Sorting is important and is required in many Applications, i.e., Searching.  one of the fundamental problems in computer science  can be solved in many ways: fast/slow use more/less memory depends on data utilize multiple computers / processors,... 4

5. Sorting  Comparison-based sorting: determining order by comparing pairs of elements.  An internal sort requires that the collection of data fit entirely in the computer’s main memory.  We can use an external sort when the collection of data cannot fit in the computer’s main memory all at once but must reside in secondary storage such as on a disk.  We will analyze only Comparison-based and internal sorting algorithms. 5

6. Bubble Sort  Idea:  Repeatedly pass through the array  Swaps adjacent elements that are out of order  Easy to implement, but slow O(N 2 ) 6

7. Example 7

8. 8

9. 9

10. 10

11. Bubble Sort – Analysis  Worst-case: O(n 2 )  Array is in reverse order:  Average-case: O(n 2 )  We have to look at all possible initial data organizations.  So, Bubble Sort is O(n 2 ) 11

12. Insertion Sort  Insertion sort is a simple sorting algorithm that is appropriate for small inputs.  The list is divided into two parts: sorted and unsorted.  In each pass, the first element of the unsorted part is picked up, transferred to the sorted sublist, and inserted at the appropriate place.  A list of n elements will take at most n-1 passes to sort the data. 12

13. Example 13

14. Insertion Sort – Analysis  Worst-case: O(n 2 )  Array is in reverse order:  Average-case: O(n 2 )  We have to look at all possible initial data organizations.  So, Insertion Sort is O(n 2 ) 14

15. Selection Sort  Idea:  Find the smallest element in the array  Exchange it with the element in the first position  Find the second smallest element and exchange it with the element in the second position  Continue until the array is sorted 15

16. Example 1329648 8329641 8349621 8649321 8964321 8694321 9864321 9864321 16

17. Selection Sort – Analysis  Worst-case: O(n 2 )  Array is in reverse order:  Average-case: O(n 2 )  We have to look at all possible initial data organizations.  So, Selection Sort is O(n 2 ) 17

18. Merge sort  Idea:  Is based on “Merging” idea where two sorted lists are combined in the right order.  The start point is to consider each element in the list as an ordered small list.  The result is a list of two-element sorted lists.  Repeatedly combine the ordered list until having one list 18

19. Merging Algorithm Merging two ordered lists: 1. Access the first item from both lists 2. While neither sequence is finished 1. Compare the current items of both 2. Copy smaller current item to the output 3. Access next item from that input sequence 3. Copy any remaining from first sequence to output 4. Copy any remaining from second to output 19

20. Example of Merging 20

21. Example: Merge sort 21

22. Merge sort – Analysis  Worst-case: O(N LogN)  Array is in reverse order:  Average-case: O(N LogN)  We have to look at all possible initial data organizations.  So, Merge sort Sort is O(N LogN)  But, merge sort requires an extra array whose size equals to the size of the original array. 22

23. Quicksort  Idea:  Repeatedly partition the data into two halves.  Only the element in the middle is sorted.  After (Log 2 N) repetitions then the data is sorted.  Advantage: One of the practically best sorting Algorithms [O(N Log 2 N)] in the average case.  Drawbacks: O(N 2 ) in the worst case. 23

24. Searching 24

25. Introduction to Search Algorithms  Search: locate an item in a list (array, vector, etc.) of information  Three algorithms:  Linear search (Also known as: Sequential Search)  Binary search  Hashing 25

26. Linear Search Example  Following Array contains:  Searching for the value 11, linear search examines 17, 23, 5, and 11  Searching for the value 7, linear search examines 17, 23, 5, 11, 2, 29, and 3 17235112293 26

27. Linear Search Tradeoffs  Benefits  Easy algorithm to understand  Array can be in any order  Disadvantage  Inefficient O(N) (slow): for array of N elements, examines N/2 elements on average for value that is found in the array, N elements for value that is not in the array 27

28. Binary Search Algorithm 1. Divide a sorted array into three sections: middle element elements on one side of the middle element elements on the other side of the middle element 2. If the middle element is the correct value, done. Otherwise, go to step 1, using only the half of the array that may contain the correct value. 3. Continue steps 1 and 2 until either the value is found or there are no more elements to examine. 28

29. Ignoring one-half of the data when the data is sorted. Binary Search Algorithm 29

30. Binary Search Example  If the following Array contains:  Searching for the value 11, binary search examines 11 and stops  Searching for the value 7, binary search examines 11, 3, 5, and stops 23511172329 30

31. Binary Search Example 31

32. Binary Search Example 32

33. Binary Search Tradeoffs  Benefit  Much more efficient than linear search (For array of N elements, performs at most log 2 N comparisons) O( log 2 N)  Disadvantage  Requires that array elements be sorted 33

34. Time Complexity Summary Worst CaseAverage Case O(N 2 ) Bubble Sort O(N 2 ) Insertion Sort O(N 2 ) Selection Sort O(N LogN) Merge Sort O(N LogN) Heap Sort O(N 2 )O(N LogN)Quick Sort O(N) Sequential Search O(LogN) Binary Search O(N)O(LogN)Binary search Tree 34

35. Sorting 35

36. Hashing  Hashing can be classified as one of the searching techniques that is usually used with external storage as Hard disk drive (HDD).  Hashing, is an information retrieval strategy for providing efficient access to information based on a key.  One usage is indexing databases. In such case, the location of a record in a database is linked to the key/index of that record.  Information can usually be accessed in constant time. 36

37. Concept of Hashing  The information to be retrieved is stored in a hash table which is best thought of as an array of m locations, called buckets  The mapping between a key and a bucket is called the hash function  The time to store and retrieve data is proportional to the time to compute the hash function (constant) 37

38. Hashing function  The ideal function, termed a perfect hash function, would distribute all elements across the buckets such that no collisions ever occurred  h(v) = f(v) mod m  Knuth (1973) suggests using as the value for m a prime number 38

39.  Determines position of key in the array  Assume table (array) size is N  Function f(x) maps any key x to an integer between 0 and N − 1  For example, assume that N=15, that key x is a non-negative integer between 0 and MAX_INT, and hash function f(x) = x Mod 15. 39 Hash Function

40. 40 Let f(x) = x Mod 15. Then, if x = 25 129 35 2501 47 36 f(x) = 10 9 5 11 2 6 Storing the keys in the array is straightforward: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 _ _ 47 _ _ 35 36 _ _ 129 25 2501 _ _ _ Thus, delete and find can be done in O(1), and also insert, except… Hash Function

41. 41 Hash Function What happens when you try to insert: x = 65 ? x = 65 f(x) = 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 _ _ 47 _ _ 35 36 _ _ 129 25 2501 _ _ _ 65(?) This is called a collision.

42. Handling Collisions A collision occurs when two different keys hash to the same value:  Ex.: For TableSize = 17, the keys 18 and 35 hash to the same value  18 mod 17 = 1 and 35 mod 17 = 1  Cannot store both data records in the same slot in array!  Resolution:  Separate Chaining (Closed Addressing): Use a dictionary data structure (such as a linked list) to store multiple items that hash to the same slot  Closed Hashing (Open Addressing): search for empty slots and store item in first empty slot that is found  Multi-Hash functions: use another hash function to resolve the collision. 42

43. Separate Chaining 43 Let each array element be the head of a chain. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14       47 65 36 129 25 2501  35 Where would you store: 29, 16, 14, 99, 127 ?

44. Separate Chaining 44 Let each array element be the head of a chain: Where would you store: 29, 16, 14, 99, 127 ? 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14          16 47 65 36 127 99 25 2501 14    35 129 29 New keys go at the front of the relevant chain.

45. Closed Hashing 45 The hash table should be large enough to include all possible keys: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 47 65 36 127 99 25 2501 14 Where would you store: 29, 60, 24, 97?

46. Closed Hashing 46 The hash table should be large enough to include all possible keys: Where would you store: 29, 60, 24, 97? 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 29 16 47 60 65 36 127 97 99 25 2501 24 14 New keys go at the next free bucket in the hash table.

47. Factors affecting efficiency  Choice of hash function  Collision resolution strategy  Hashing offers excellent performance for insertion and retrieval of data. 47

48. Questions ? 48