1. LEVEL II, TERM II CSE – 243 MD. MONJUR-UL-HASAN LECTURER DEPT OF CSE, CUET EMAIL: MAIL@MONJUR-UL-HASAN.INFO Sorting

2. Bubble Sort bubble1(data, n) { for j=1 to n for i=1 to n - j if data[i-1]>data[i] swap(data,i-1,i); } The internal loop runs n-1 times, then n-2 times, then n-3 times and so on … What is the complexity? 2 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

3. Analysis of Running Time Let ’ s assume the swapping procedure along with the if that surrounds it run in at most k time units. Total running time is then at most: What is the lower bound? 3 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

4. 4 One Pass Array after Completion of Each Pass

5. Bubble Sort bubble1(data, n) { for j=1 to n for i=1 to n - j if data[i-1]>data[i] swap(data,i-1,i); } Question: Is this a stable sort? Question: What happens if we start this on a sorted array? Question: What happens in the following case? a) 1 4 6 20 10 b) 9 8 6 5 4 3 Calculate the number of swap for each case. 5 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

6. Modified Bubble Sort bubble1(data, n) { for j=1 to n boolean movedSomething = false; //added this for i=1 to n - j if data[i-1]>data[i] movedSomething = true; //added this swap(data,i-1,i); if(!movedSomething) break; //added this } We detect when the array is sorted and stop in the middle. 6 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

7. Bubble Sort Again Answer the following Question: Is this a stable sort? Question: What happens if we start this on a sorted array? Question: What happens in the following case? a) 1 4 6 20 10 b) 9 8 6 5 4 3 Calculate the number of swap for each case. 7 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

8. Modified Bubble Sort Before the modification, running time was always nearly the same but now what? What is the best case? What is the worst case? What is the running time in each case? Which version of bubble sort is better? 8 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

9. Complexity of Modified Bubble Sort. What is the worst case running time? O(n 2 ) - The array is sorted in reverse – we don ’ t exit early. What is the best case running time? Ω(n) – in the best case we start with a sorted array. Go over it once and see that nothing was moved. 9 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

10. Selection Sort Pseudo Code: For i=0,1, …,n-2  Run over the array from i onwards.  Find j- the index of the minimal item.  Swap the values in indexes i,j. 10 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

11. 11

12. Running Time of selection sort What is the worst case? What is the best case? Can we say something about the average case? What is the running time in each one? 12 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

13. Running Time of selection sort Note first that the running time is always pretty much the same. Finding the minimal value always requires us to go over the entire sub-array. There is no shortcut, and we never stop early. First, we have an array of size n-1 to scan, then size n- 2, then size n-2 … We ’ ve already analyzed this pattern: Running time is O(n 2 ). (HOW ?) 13 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

14. Selection Sort: Analysis Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 14 Number of comparisons: (n-1) + (n-2) +... + 3 + 2 + 1 =n * (n-1)/2 =(n² - n )/2  O(n²) Number of exchanges (worst case): n – 1  O(n) Overall (worst case) O(n) + O(n²) = O(n²) (‘quadratic sort‘)

15. Finding the smallest difference The na ï ve way: minimalDif(numbers,length) { diff = MAX_VALUE; for i=0 to length for j=i+1 to length currentDiff = abs(numbers[i]-numbers[j]); if(currentDiff<diff) diff = currentDiff; return diff; } Try to find a better algorithm to do this.

16. Insertion Sorting insertSort(data, length) { for i=1 to length //assume before index i the array is sorted insert(data,i); //insert i into array 0...(i-1) } insert(data,index) { value = data[index]; I = index-1 while data[i]>value and i>=0 data[i+1]=data[i] i=i-1 data[i+1] = value; } 16 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

17. 17 Insertion Sort: 4 passes Pass 3

18. Complexity of Insertion Sort What is the worst case? What is the best case? What is the complexity in each one? 18 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

19. Insertion Sort: Analysis Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 19 Number of comparisons (worst case): (n-1) + (n-2) +... + 3 + 2 + 1  O(n²) Number of comparisons (best case): n –1  O(n) Number of exchanges (worst case): (n-1) + (n-2) +... + 3 + 2 + 1  O(n²) Number of exchanges (best case): 0  O(1) Overall worst case: O(n²) + O(n²) = O(n²)

20. Marge Sort Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 20 Divide & Conquer Algorithm Faster than Babble sort, Insertion sort and selection sort. Merge Sort Tree:  Take a binary tree T  Each node of T represents a recursive call of the merge sort algorithm.  We associate with each node v of T a the set of input passed to the invocation v represents.  The external nodes are associated with individual elements of S, upon which no recursion is called.

21. CIS 068 Merge Sort: Example 912191612543 912191612543 912191612543 912161913425 134912161925 912161912534 912191612543 split merge split

22. Marge Sort Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 22 Mergesort(a, from, to) { if (from == to) return; int mid = (from + to) / 2; merge_sort(a, from, mid); merge_sort(a, mid+1, to); merge(a, from, to); }

23. Marge sort (Cont) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 23 Merge(a,from,to) { mid = (from+to)/2 h = I = low; j = mid+1 while( j<= mid and j<=mid) if( a[h] <=a[j]) b[i] = a[h] // b is an auxiliary array h = h+1 else b[i] = a[j] j = j+ 1 i = i +1 if(h>j) for k = j to to b[i] = a[k] i = i+1; else for k = h to mid b[i] = a[k] i = i+1 for k= from to to a[k] = b[k] }

24. Complexity and Complexity Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 24 The complexity is O(n * Log(n)) The idea is:  We need Log(n) merging steps  Each merging step has complexity O(n) Problem: Merge Sort needs extra memory !

25. Heap Sort Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 25 Knowledge Required  Binary Tree Mathematics  Complete Binary Tree  Array Representation of Binary Tree Some Basic  A complete Binary tree have n node then it will have ceil(n/2) leaf and floor(n/2) non leaf  If we put all the node in a row level wise then each non leaf at position I will have left child at 2*i and right child at 2*i+1 position.  If a root have only left child then all subsequent child will be leaf.

26. Heap Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 26 Heap two types: Max Heap and Min Heap

27. Heap Sort: Pseudo Code Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 27 Max-Heapify(A,i,heapsize) { l = 2*i r = l+1 largest = i if l A[i] largest = l if r a[largest] largest = r if largest ≠ I exchange a[i]  a[largest] Max-Heapify(a,largest,heapsize) }

28. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 28

29. Building a Heap Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 29  Build Heap The above list is not a heap. Let’s see how to build it into a heap function Build-Max-Heap(a, start, heapsize) { for i = floor(heapsize/2) downto 1 Max-Heapify(A,i,heapsize) }

30. Building a Heap (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 30

31. Building a Heap (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 31

32. Building a Heap (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 32

33. Building a Heap (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 33

34. Building a Heap (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 34  Now, the list becomes a heap

35. Heap Sort: Pseudo Code (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 35 function heapSort(a,length) { heapsize = length Build-Max-Heap(A, heapsize) for i = length downto 2 exchange a[1]  a[i] heapsize = heapsize-1 Max-Heapify(A,1,heapsize) }

36. Heap Sort: Pseudo Code (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 36

37. Lower Bound for Sorting Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 37 Theorem : any sorting algorithm based on only comparisons takes  (N log N) comparisons in the worst case (worse-case input) to sort N elements. Prove: Suppose we want to sort N distinct elements How many possible orderings do we have for N elements?

38. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 38 We can have N! possible orderings e.g., the sorted output for a,b,c can be a b c b a c a c b c a b c b a b c a

39. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 39 Any comparison-based sorting process can be represented as a binary decision tree.  Each node represents a set of possible orderings, consistent with all the comparisons that have been made  The tree edges are results of the comparisons

40. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 40

41. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 41 A different algorithm would have a different decision tree Decision tree for Insertion Sort on 3 elements:

42. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 42 The worst-case number of comparisons used by the sorting algorithm is equal to the depth of the deepest leaf  The average number of comparisons used is equal to the average depth of the leaves A decision tree to sort N elements must have N! leaves  a binary tree of depth d has at most 2 d leaves  the tree must have depth at least  log 2 (N!)  Therefore, any sorting algorithm based on only comparisons between elements requires at least  log 2 (N!)  comparisons in the worst case.

43. Lower Bound for Sorting (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 43 Any sorting algorithm based on comparisons between elements requires  (N log N) comparisons.

44. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 44 Linear time sorting

45. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 45 Can we do better (linear time algorithm) if the input has special structure (e.g., uniformly distributed, every numbers can be represented by d digits)? Yes. Counting sort, radix sort

46. Counting Sort Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 46 for i  1 to k ⊳ k : highest number in A do C[i]  0 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 for i  2 to k do C[i]  C[i] + C[i–1] for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1

47. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 47 A:A: 41343 B:B: 12345 C:C: 1234

48. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 48 for i  1 to k do C[i]  0 A:A: 41343 B:B: 12345 C:C: 0000 1234

49. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 49 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 A:A: 41343 B:B: 12345 C:C: 0001 1234 j= 1 A[j] => a[1] => 4 C[A[j]] =>C[4]=> 0 C[A[j]] +1=> = > 0+1

50. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 50 A:A: 41343 B:B: 12345 C:C: 1001 1234 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 j= 2 A[j] => a[2] => 1 C[A[j]] =>C[1]=> 0 C[A[j]] +1=> = > 0+1

51. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 51 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 A:A: 41343 B:B: 12345 C:C: 1011 1234 j= 3 A[j] => a[3] => 3 C[A[j]] =>C[3]=> 0 C[A[j]] +1=> = > 0+1

52. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 52 A:A: 41343 B:B: 12345 C:C: 1012 1234 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 j= 4 A[j] => a[4] => 4 C[A[j]] =>C[4]=> 1 C[A[j]] +1=> = > 1+1

53. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 53 A:A: 41343 B:B: 12345 C:C: 1022 1234 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 j= 5 A[j] => a[5] => 3 C[A[j]] =>C[3]=> 1 C[A[j]] +1=> = > 1+1

54. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 54 A:A: 41343 B:B: 12345 C:C: 1022 1234 C':C': 1122 for i  2 to k do C[i]  C[i] + C[i–1]

55. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 55 A:A: 41343 B:B: 12345 C:C: 1022 1234 C':C': 1132 for i  2 to k do C[i]  C[i] + C[i–1]

56. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 56 A:A: 41343 B:B: 12345 C:C: 1022 1234 C':C': 1135 for i  2 to k do C[i]  C[i] + C[i–1]

57. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 57 A:A: 41343 B:B: 3 12345 C:C: 1135 1234 C':C': 1125 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 j= n =>j= 5 A[j] => a[5] => 3 C[A[j]] =>C[3]=> 3 B[C[A[j]] ] = > B[3]

58. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 58 A:A: 41343 B:B: 34 12345 C:C: 1125 1234 C':C': 1124 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 j= n =>j= 4 A[j] => a[4] => 4 C[A[j]] =>C[4]=> 5 B[C[A[j]] ] = > B[5]

59. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 59 A:A: 41343 B:B: 334 12345 C:C: 1124 1234 C':C': 1114 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 j= n =>j= 3 A[j] => a[3] => 3 C[A[j]] =>C[3]=> 3 B[C[A[j]] ] = > B[3]

60. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 60 A:A: 41343 B:B: 1334 12345 C:C: 1114 1234 C':C': 0114 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 j= n =>j= 2 A[j] => a[2] => 1 C[A[j]] =>C[1]=> 1 B[C[A[j]] ] = > B[1]

61. Counting Sort (Continue) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 61 A:A: 41343 B:B: 13344 12345 C:C: 0114 1234 C':C': 0113 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 j= n =>j= 1 A[j] => a[1] => 4 C[A[j]] =>C[4]=> 4 B[C[A[j]] ] = > B[4]

62. Complexity: Counting Sort Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 62 But it needs k Extra memory (n)(n) (k)(k) (n)(n) (k)(k) for j  1 to n do C[A[ j]]  C[A[ j]] + 1 for i  2 to k do C[i]  C[i] + C[i–1] for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1  (n + k)

63. Complexity: Counting Sort: Stable? Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET 63 A:A: 41343 B:B: 13344