FASTER SORT using RECURSION : MERGE SORT COMP 103 FASTER SORT using RECURSION : MERGE SORT Marcus Frean School of Engineering and Computer Science, Victoria University of Wellington 2013-T1 Lecture 14

RECAP-TODAY RECAP TODAY Announcements 2 RECAP Recursion TODAY Fast Sorts: Divide and Conquer Merge Sort (and analysis) Announcements Mid-term test: it’s marked – available a.s.a.p., and we will go over it in detail, in the Thursday lecture. Assign#4 due 30th April (!) – that’s Tuesday when you come back. Use help-desk today and tomorrow to ask questions. Assign#5 (Sorting) out 30th April. Awkward timing, but unavoidable.

Slow Sorts Insertion sort, Selection Sort, Bubble Sort: Problem: All slow (except Insertion sort on almost sorted lists) O(n2 ) Problem: Insertion and Bubble only compare adjacent items only move items one step at a time Selection compares every pair of items ignores results of previous comparisons. Solution: Must compare and swap items at a distance Must not perform redundant comparisons

Divide and Conquer Sorts To Sort: Split Sort each part (recursive) Combine Where does the work happen? MergeSort: split is trivial combine does all the work QuickSort: split does all the work combine is trivial Array Split SubArray SubArray Split Combine SubArray Sort SortedSubArray Sort Split Combine SubArray Sort SortedSubArray SortedSubArray SortedSubArray Combine Sorted Array

MergeSort

MergeSort: recursion (...) ms(d, t, 0,16) ms(t, d, 0,8) ms(t, d, 8,16) mg(t, d, 0,8,16) ms(d,t,0,4) ms(d,t,4,8) mg(d,t,0,4,8) ms(d,t,8,12) ms(d,t,12,16) mg(d,t,0,8,16) ms(t,d,0,2) (...) ms(t,d,2,4) mg(t,d,0,2,4) ms(t,d,6,8) mg(t,d,4,6,8) ms(t,d,4,6) ms(d,t,0,1) ms(d,t,2,3) ms(d,t,1,2) ms(d,t,3,4) ms(d,t,4,5) ms(d,t,6,7) mg(d,t,0,1,2) mg(d,t,2,3,4) ms(d,t,5,6) ms(d,t,7,8) mg(d,t,6,7,8) mg(d,t,4,5,6)

MergeSort – the "wrapper" that starts it Needs a temporary array for copying create temporary array fill with a copy of the original data. public static <E> void mergeSort(E[] data, int size, Comparator<E> comp){ E[] other = (E[])new Object[size]; for (int i=0; i<size; i++)   other[i]=data[i]; mergeSort(data, other, 0, size, comp); }

MergeSort – the recursive method private static <E> void mergeSort(E[] data, E[] temp, int low, int high, Comparator<E> comp){ // sort items from low..high-1 using temp array if (high > low+1){ int mid = (low+high)/2; // mid = low of upper 1/2, = high of lower half. mergeSort(temp, data, low, mid, comp); mergeSort(temp, data, mid, high, comp); merge(temp, data, low, mid, high, comp); } there are multiple calls to the recursive method in here. this will make a "tree" structure we swap temp and data each recursive call

MergeSort – will the two arrays 'mess up'? data temp to sort 'working' to sort 'working' merge                 to sort 'working' merge                 merge

MergeSort: the ‘merge’ method temporary array 6 7 8 9 10 11 1 2 3 4 5 data array 1 2 3 4 5 6 7 8 9 10 11

Merge /** Merge from[low..mid-1] with from[mid..high-1] into to[low..high-1.*/ private static <E> void merge(E[] from, E[] to, int low, int mid, int high, Comparator<E> comp){ int index = low;        // where we will put the item into "to“ int indxLeft = low;    // index into the lower half of the "from" range int indxRight = mid;   // index into the upper half of the "from" range while (indxLeft<mid && indxRight < high){ if (comp.compare(from[indxLeft], from[indxRight]) <=0) to[index++] = from[indxLeft++]; else to[index++] = from[indxRight++]; } // copy over the remainder. Note only one loop will do anything. while (indxLeft<mid) while (indxRight<high)

data [p a1 r f e q2 w q1 t z2 x c v b z1 a2 ] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 msort(0..16) [p a1 r f e q2 w q1 t z2 x c v b z1 a2 ] msort(0..8) [p a1 r f e q2 w q1 ] msort(0..4 ) [p a1 r f ] msort(0..2 ) [p a1 ] msort(0..1 ) [p ] msort(1..2) [ a1 ] merge(0.1.2) [a1 p ] msort(2..4) [ r f ] msort(2..3) [ r ] msort(3..4) [ f ] merge(2.3.4) [ f r ] merge(0.2.4) [a1 f p r ] msort(4..8) [ e q2 w q1 ] msort(4..6) [ e q2 ] : : merge(4.5.6) [ e q2 ] msort(6..8) [ w q1 ] : : merge(6.7.8) [ q1 w ] merge(4.6.8) [ e q2 q1 w ] merge(0.4.8) [a1 e f p q2 q1 r w ] msort(8..16) [ t z2 x c v b z1 a2 ] : : merge(8.12.16) [ a2 b c t v x z1 z2 ] merge(0.8.16) [a1 a2 b c e f p q2 q1 r t v w x z1 z2 ]

Sorting Algorithm costs: Insertion sort, Selection Sort, Bubble Sort: All slow (except Insertion sort on almost-sorted lists) O(n 2) Merge Sort? There’s no inner loop! How do you analyse recursive algorithms?

MergeSort Cost of mergeSort: Three steps: first recursive call private static <E> void mergeSort (E[ ] data, E[ ] temp, int low, int high, Comparator<E> comp) {   if (high > low+1) {       int mid = (low+high)/2;          mergeSort(temp, data, low, mid, comp);       mergeSort(temp, data, mid, high, comp);     merge(temp, data, low, mid, high, comp); } } Cost of mergeSort: Three steps: first recursive call second recursive call merge: has to copy over (high-low) items

MergeSort Cost (the real order)

MergeSort Cost (analysis order)

MergeSort Cost Level 1: 2 * n/2 = n Level 2: 4 * n/4 = n so in general, at any level k, there are n comparisons How many levels? the number of times you can halve n is log n Total cost? = O( ) n = 1,000 n = 1,000,000

Analysing with Recurrence Relations private static <E> void mergeSort(E[] data, E[] temp, int low, int high, Comparator<E> comp){   if (high > low+1){       int mid = (low+high)/2;          mergeSort(temp, data, low, mid, comp);       mergeSort(temp, data, mid, high, comp);     merge(temp, data, low, mid, high, comp); } } Cost of mergeSort = C(n) C(n) = C(n/2) + C(n/2) + n = 2 C(n/2) + n Recurrence Relation: (we will) Solve by repeated substitution & find pattern (we could) Solve by general method

Solving Recurrence Relations C(n) = 2 C(n/2) + n = 2 [ 2 C(n/4) + n/2] + n = 4 C(n/4) + 2 n = 4 [ 2 (C(n/8) + n/4] + 2 n = 8 C(n/8) + 3 n = 16 C(n/16) + 4 n : = 2k C( n/2k ) + k * n when n = 2k, k = log(n) = n C (1) + log(n) * n and since C(1) = 0, C(n) = log(n) * n http://www.youtube.com/watch?v=XaqR3G_NVoo