1. Merge sort, Insertion sort

2. Sorting I / Slide 2 Sorting * Selection sort (iterative, recursive?) * Bubble sort

3. Sorting I / Slide 3 Sorting * Selection sort and Bubble sort 1. Find the minimum value in the list 2. Swap it with the value in the first position 3. Repeat the steps above for remainder of the list (starting at the second position) * Insertion sort * Merge sort * Quicksort * Shellsort * Heapsort * Topological sort * …

4. Sorting I / Slide 4 * Worst-case analysis: N+N-1+ …+1= N(N+1)/2, so O(N^2) for (i=0; i<n-1; i++) { for (j=0; j<n-1-i; j++) { if (a[j+1] < a[j]) { // compare the two neighbors tmp = a[j]; // swap a[j] and a[j+1] a[j] = a[j+1]; a[j+1] = tmp; } Bubble sort and analysis

5. Sorting I / Slide 5 Two fundamental algorithmic principles * Insertion: n Incremental algorithm principle * Mergesort: n Divide and conquer principle

6. Sorting I / Slide 6 Insertion sort 1) Initially i = 1 2) Let the first i elements be sorted. 3) Insert the (i+1)th element properly in the list (go inversely from right to left) so that now i+1 elements are sorted. Proof by induction, recursion

7. Sorting I / Slide 7 Pseudo-code Input: an array a[left, right] Output: sorted array a[left,right] in increading order i  1 While (not end of the array) { pick up the next element at i+1 insert a[i+1] into the sorted array from 1 to i; } It can be recursive … but not efficient

8. Sorting I / Slide 8 Insertion Sort

9. Sorting I / Slide 9 Insertion Sort * Consists of N - 1 passes * For pass p = 1 through N - 1, ensures that the elements in positions 0 through p are in sorted order n elements in positions 0 through p - 1 are already sorted n move the element in position p left until its correct place is found among the first p + 1 elements http://www.cis.upenn.edu/~matuszek/cse121-2003/Applets/Chap03/Insertion/InsertSort.html

10. Sorting I / Slide 10 Extended Example To sort the following numbers in increasing order: 34 8 64 51 32 21 p = 1; tmp = 8; 34 > tmp, so second element a[1] is set to 34: {8, 34}… We have reached the front of the list. Thus, 1st position a[0] = tmp=8 After 1st pass: 8 34 64 51 32 21 (first 2 elements are sorted)

11. Sorting I / Slide 11 P = 2; tmp = 64; 34 < 64, so stop at 3 rd position and set 3 rd position = 64 After 2nd pass: 8 34 64 51 32 21 (first 3 elements are sorted) P = 3; tmp = 51; 51 < 64, so we have 8 34 64 64 32 21, 34 < 51, so stop at 2nd position, set 3 rd position = tmp, After 3rd pass: 8 34 51 64 32 21 (first 4 elements are sorted) P = 4; tmp = 32, 32 < 64, so 8 34 51 64 64 21, 32 < 51, so 8 34 51 51 64 21, next 32 < 34, so 8 34 34, 51 64 21, next 32 > 8, so stop at 1st position and set 2 nd position = 32, After 4th pass: 8 32 34 51 64 21 P = 5; tmp = 21,... After 5th pass: 8 21 32 34 51 64

12. Sorting I / Slide 12 Analysis: worst-case running time * Inner loop is executed p times, for each p=1..N  Overall: 1 + 2 + 3 +... + N = O(N 2 ) * Space requirement is O(N)

13. Sorting I / Slide 13 Analysis: best case * The input is already sorted in increasing order n When inserting A[p] into the sorted A[0..p-1], only need to compare A[p] with A[p-1] and there is no data movement n For each iteration of the outer for-loop, the inner for-loop terminates after checking the loop condition once => O(N) time * If input is nearly sorted, insertion sort runs fast

14. Sorting I / Slide 14 Summary on insertion sort * Simple to implement * Efficient on (quite) small data sets * Efficient on data sets which are already substantially sorted * More efficient in practice than most other simple O(n^2) algorithms such as selection sort or bubble sort: it is linear in the best caseOselection sortbubble sort * Stable (does not change the relative order of elements with equal keys) Stable * In-place (only requires a constant amount O(1) of extra memory space) In-place * It is an online algorithm, in that it can sort a list as it receives it.online algorithm

15. Sorting I / Slide 15 Mergesort Based on divide-and-conquer strategy * Divide the list into two smaller lists of about equal sizes * Sort each smaller list recursively * Merge the two sorted lists to get one sorted list

16. Sorting I / Slide 16 Pseudo-code Input: an array a[left, right] Output: sorted array a[left,right] in increasing order MergeSort (a, left, right) { if (left < right) { mid  divide (a, left, right) MergeSort (a, left, mid-1) MergeSort (a, mid+1, right) merge(a, left, mid+1, right) }

17. Sorting I / Slide 17 http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/MSort.html

18. Sorting I / Slide 18 How do we divide the list? How much time needed? How do we merge the two sorted lists? How much time needed?

19. Sorting I / Slide 19 How to divide? * If an array A[0..N-1]: dividing takes O(1) time we can represent a sublist by two integers left and right : to divide A[left..Right], we compute center=(left+right)/2 and obtain A[left..Center] and A[center+1..Right]

20. Sorting I / Slide 20 How to merge? * Input: two sorted array A and B * Output: an output sorted array C * Three counters: Actr, Bctr, and Cctr n initially set to the beginning of their respective arrays (1) The smaller of A[Actr] and B[Bctr] is copied to the next entry in C, and the appropriate counters are advanced (2) When either input list is exhausted, the remainder of the other list is copied to C

21. Sorting I / Slide 21 Example: Merge

22. Sorting I / Slide 22 Example: Merge... * Running time analysis: Clearly, merge takes O(m1 + m2) where m1 and m2 are the sizes of the two sublists. * Space requirement: n merging two sorted lists requires linear extra memory n additional work to copy to the temporary array and back

23. Sorting I / Slide 23

24. Sorting I / Slide 24 Analysis of mergesort Let T(N) denote the worst-case running time of mergesort to sort N numbers. Assume that N is a power of 2. * Divide step: O(1) time * Conquer step: 2 T(N/2) time * Combine step: O(N) time Recurrence equation: T(1) = 1 T(N) = 2T(N/2) + N

25. Sorting I / Slide 25 Analysis: solving recurrence Since N=2 k, we have k=log 2 n The ‘overhead’ is a linear algo, not a constant time in binary search

26. Sorting I / Slide 26 Don’t forget: We need an additional array for ‘merge’! So it’s not ‘in-place’!

27. Sorting I / Slide 27 An experiment * Code from textbook (using template)  Unix time utility

28. Sorting I / Slide 28