1. IKI 10100I: Data Structures & Algorithms Ruli Manurung (acknowledgments to Denny & Ade Azurat) 1 Fasilkom UI Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Sorting algorithms & their analysis

2. 2 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Several sorting algorithms: Bubble Sort Selection Sort Insertion Sort Shell Sort Merge Sort Quick Sort For each algorithm: Basic Idea Example Implementation Algorithm Analysis Outline

3. 3 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Sorting Sorting = ordering. Sorted = ordered based on a particular way. Generally, collections of data are presented in a sorted manner. Examples of Sorting: Words in a dictionary are sorted (and case distinctions are ignored). Files in a directory are often listed in sorted order. The index of a book is sorted (and case distinctions are ignored). Many banks provide statements that list checks in increasing order (by check number). In a newspaper, the calendar of events in a schedule is generally sorted by date. Musical compact disks in a record store are generally sorted by recording artist. Why? Imagine finding the phone number of your friend in your mobile phone, but the phone book is not sorted.

4. 4 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Bubble Sort: Idea Idea: bubble in water. Bubble in water moves upward. Why? How? When a bubble moves upward, the water from above will move downward to fill in the space left by the bubble. As a sorting algo: During each iteration, compare adjacent items: the “bubble” floats, the “water” sinks! (i.e. swap items)

5. 5 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Bubble sort: an iteration 4021433650583424 2401433650583424 2401433650583424 2140433650583424 21404336505834242140433650583424 2140343650583424 21403436505834242140343650583424 2140343065583424 2140343065583424 2140343065583424 2140343065583424 2140343058653424 2140343058653424 2140343058365424 2140343058365424 2140343058342654 2140343058342654 2140343058342465

6. 6 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Bubble Sort: Example 4021433650583424 6521403430583424 6558 123400433424 1234006543583424 1 2 3 4 Notice that at least one element will be in the correct position each iteration.

7. 7 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 1032653435842404 Bubble Sort: Example 0126534358424043 0126534358424043 6 7 8 Stop here… why? 1203340654358424 5

8. 8 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Bubble Sort: Implementation void sort(int a[]){ for (int i = a.length; i>=0; i--) { boolean swapped = false; for (int j = 0; j<i; j++) {... if (a[j] > a[j+1]) { int T = a[j]; a[j] = a[j+1]; a[j+1] = T; swapped = true; }... } if (!swapped) return; }

9. 9 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Bubble Sort: Analysis Running time: Worst case: O(N 2 ) Best case: O(N) -- when? why? Variant: bi-directional bubble sort original bubble sort: only works in one direction bi-directional bubble sort: works back and forth.

10. 10 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Selection Sort: Idea 1. Initial condition: Unsorted list = data Sorted list = empty 2. Select the “best” (e.g. smallest) item from the unsorted group, then add the “best” item at the end of the sorted group. 3. Repeat the process until the unsorted group becomes empty.

11. 11 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4240213340655843 4021433404265583 4021433405836542 4021433650583424 Selection Sort: Example

12. 12 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4240213340655843 4221334065584340 4221334655843400 4221034655843403 Selection Sort: Example

13. 13 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 1 4221346558434030 420346558434032 1420346558434032 1420346558434032 1420346558434032 Selection Sort: Example

14. 14 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Selection Sort: Implementation void sort(int a[]) throws Exception { for (int i = 0; i < a.length; i++) { int min = i; for (int j = i + 1; j < a.length; j++) if (a[j] < a[min]) min = j; int T = a[min]; a[min] = a[i]; a[i] = T; } Select best item from unsorted list. Add it to the end of the sorted list.

15. 15 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Selection Sort: Analysis Running time: Worst case: O(N 2 ) Best case: O(N 2 ) Based on big-oh analysis, is selection sort better than bubble sort? Does the actual running time reflect the analysis?

16. 16 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Insertion Sort: Idea 1. Initial condition: Unsorted list = data Sorted list = empty 2. Take the first (any) item from the unsorted group, then insert it at its correct position in the sorted group. 3. Repeat the process until the unsorted group becomes empty.

17. 17 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Insertion Sort: Idea Imagine you are sorting a hand of cards. 8 | 5 9 2 6 3 5 8 | 9 2 6 3 5 8 9 | 2 6 3 2 5 8 9 | 6 3 2 5 6 8 9 | 3 2 3 5 6 8 9 |

18. 18 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4021433650583424 2401433650583424 1240433650583424 40 Insertion Sort: Example

19. 19 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 1234043650583424 1240433650583424 1234043650583424 Insertion Sort: Example

20. 20 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 1234043650583424 1234043650583424 12340436505834241234043650 Insertion Sort: Example

21. 21 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 12340436505834241234043650 12340436505842412334365058404365 123404365042412334365058404365 Insertion Sort: Example 12340436504212334365058443654258404365

22. 22 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Insertion Sort: Implementation public static void insertionSort (int[] a) { for (int ii = 1; ii < a.length; ii++) { int jj = ii; while (( jj > 0) && (a[jj] < a[jj - 1])) { int temp = a[jj]; a[jj] = a[jj - 1]; a[jj - 1] = temp; jj--; } Note that a[jj] is always the same  we can make things a bit more efficient! Get first item in unsorted list. Insert it in the sorted list.

23. 23 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Insertion Sort: Efficient Implementation A slightly more efficient Insertion sort public static void insertionSort2 (int[] a) { for (int ii = 1; ii < a.length; ii++) { int temp = a[ii]; int jj = ii; while (( jj > 0) && (temp < a[jj - 1])) { a[jj] = a[jj - 1]; jj--; } a[jj] = temp; }

24. 24 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Insertion Sort: Analysis Running time analysis: Worst case: O(N 2 ) Best case: O(N) Is insertion sort faster than selection sort? Notice the similarity and the difference between insertion sort and selection sort.

25. 25 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 A Lower Bound Bubble Sort, Selection Sort, Insertion Sort all have worst case of O(N 2 ). It turns out, for any algorithm that exchanges adjacent items, this is the best worst case: Ω (N 2 ) In other words, this is a lower bound! See proof in Section 8.3 of Weiss

26. 26 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4021433650583424 Original: 5-sort: Sort items with distance 5 elements: 4021433650583424 Shell Sort: Idea Donald Shell (1959): Exchange items that are far apart!

27. 27 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4021433650583424 Original: 4004334221583654 After 5-sort: 2031440342436558 After 3-sort: Shell Sort: Example After 1-sort: 12340436504212334365058443654258404365

28. 28 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Shell Sort: Gap Values Gap: the distance between items being sorted. As we progress, the gap decreases. Shell Sort is also called Diminishing Gap Sort. Shell proposed starting gap of N/2, halving at each step. There are many ways of choosing the next gap.

29. 29 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 O(N 3/2 )?O(N 5/4 )? O(N 7/6 )? Shell Sort: Analysis So we have 3 nested loops, but Shell Sort is still better than Insertion Sort! Why?

30. 30 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Mergesort: Basic Idea Divide and Conquer approach Idea: Merging two sorted array takes O(n) time Split an array into two takes O(1) time 12340436503442581023344042435865 Counter A Counter B Counter c

31. 31 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Mergesort: Merge Implementation Implement operation to merge two sorted arrays into one sorted array: public static void merge(int[] A, int[] B, int[] C) { } Assume A and B are sorted and |C| = |A| + |B|

32. 32 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Mergesort: Algorithm 1. Base case: if the number of items to sort is 0 or 1, return. 2. Recursive case: recursively sort the first and second half separately. 3. Merging step: merge the two sorted halves into a sorted group.

33. 33 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 4021433650583424402143365058342440214336505834244021433650583424 split Mergesort: Example

34. 34 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 402143365058342421365584244012433650583442 split merge 1240343650583442 Mergesort: Example

35. 35 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 merge 124034365058344212340436503442580123344042435862 Mergesort: Example

36. 36 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Mergesort: Implementation MergeSort implementation (and ‘driver’ method) void mergeSort(int[] array) {mergeSort(array, 0, a.length-1);} void mergeSort(int[] a, int left, int right) { if(left < right) { int centre = (left + right)/2; mergeSort(a, left, centre); mergeSort(a, center+1, right); merge(a, left, center+1, right); } How to merge the two subarrays of A without any “temporary” space?

37. 37 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Implement operation to merge two sorted subarrays: public static void merge(int[] A, int l, int c, int r) { } Mergesort: Merge Implementation

38. 38 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Mergesort: Analysis Running Time: O(n log n) Why?

39. 39 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Quicksort: Basic Idea Yet another divide and conquer approach quickSort(S) algorithm: If the number of items in S is 0 or 1, return. Pick any element v  S. This element is called the pivot. Partition S – { v } into two disjoint groups: L = { x  S – { v } | x  v } and R = { x  S – { v } | x  v } Return the result of quickSort(L), followed by v, followed by quickSort(R).

40. 40 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 40 2 1 3 43 65 0 58 3 42 4 Quicksort: Select Pivot

41. 41 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 2 1 3 43 65 0 58 3 42 4 40 Quicksort: Partition

42. 42 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 213 4365 0 58 3 42 4 40 213 4365 0 58 3 42 4 40 Quicksort: Recursive Sort & Merge

43. 43 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 402143365058342440214336505834244021433650583424 left 40 right Quicksort: Partition Algorithm 1 3214336505840424

44. 44 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 402143365058342440214336505834244021433650583424 leftright Quicksort: Partition Algorithm 1

45. 45 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 3210344042435865213342213034342652130344265584340214336505834244021433650583424332323 Quicksort: Partition Algorithm 1

46. 46 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Quicksort: Partition Algorithm 2 402143365058342440 original pivot = 40 while < pivot left++ while >= pivot right-- 4214336505834240 21433650583424 rightleft 4021336505843424 rightleft “move pivot out of the way”

47. 47 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Quicksort: Partition Algorithm 2 4021336505843424 rightleft 40213306558434246521330405843424 rightleft CROSSING: Quicksort recursively “move pivot back” Quicksort recursively

48. 48 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 static void QuickSort(int a[], int low, int high) { if(high <= low) return; // base case pivot = choosePivot(a); // select “ best ” pivot int i=low, j=high-1; swap(a,pivot,a[j]); // move pivot out of the way while(i <= j) { // find large element starting from left while(i<high && a[i]<pivot) i++; // find small element starting from right while(j>low && a[j]>=pivot) j--; // if the indexes have not crossed, swap if(i>j) swap(a, i, j); } swap(a,i,high-1); // restore pivot quickSort(a,low,i-1); // sort small elements quickSort(a,i+1,high); // sort large elements } Quicksort: Implementation

49. 49 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Quicksort: Analysis Partitioning takes O(n) Merging takes O(1) So, for each recursive call, the algorithm takes O(n) How many recursive calls does a quick sort need?

50. 50 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Quicksort: Choosing The Pivot Ideal pivot: Median element Common pivot First element Element at the middle Median of three

51. 51 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 Epilogue: Generic Sort So far we have methods to sort integers. What about Strings? Employees? Cookies? A new method for each class? No! In order to be sorted, objects should be comparable (less than, equal, greater than). Solution: use an interface that has a method to compare two objects. Remember: A class that implements an interface inherits the interface (method definitions) = interface inheritance, not implementation inheritance.

52. 52 Ruli Manurung (Fasilkom UI)IKI10100I: Data Structures & Algorithms Week 5 The Comparable Interface In Java, generic aspect of “comparable” is defined in an interface in package java.lang : Method compareTo returns: <0: the object ( this ) is smaller than the parameter ‘ ob ’ 0: the object ( this ) is equal to the parameter ‘ ob ’ >0: the object ( this ) is greater than the parameter ‘ ob ’ public interface Comparable { public int compareTo (Object ob); }