1. CSC 213 – Large Scale Programming

2. Today’s Goals  Review past discussion of data sorting algorithms  Weaknesses of past approaches & when we use them  Can we find limit to how long sorting needs?  What does this mean for sorting & those past sorts  Get good idea of how merge sort is executed  What is algorithm and what will this require?  What are execution trees & how they show runtime?

3. Ghosts of Sorts Past  We have already seen & discussed 4 sorts  Bubble-sort -- O ( n 2 ) time sort; slowest sort  Selection-sort -- O ( n 2 ) time sort; PQ concept  Insertion-sort -- O ( n 2 ) time sort; PQ concept O ( n log n )  Heap-sort -- O ( n log n ) time sort; requires PQ  All of these sorts of limited usefulness

4. Ghosts of Sorts Past  We have already seen & discussed 4 sorts  Bubble-sort -- O ( n 2 ) time sort; slowest sort  Selection-sort -- O ( n 2 ) time sort; PQ concept  Insertion-sort -- O ( n 2 ) time sort; PQ concept O ( n log n )  Heap-sort -- O ( n log n ) time sort; requires PQ  All of these sorts of limited usefulness

5. Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting yesno Is x i > x j ?

6. Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting  Traveling from root to leaf sorts data  Tree’s height is lower-bound on sorting complexity

7. Decision Tree Height  Unique leaf for each ordering of data initially  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)

8. Decision Tree Height  Unique leaf for each ordering of data initially  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)  For sequence of n numbers, can arrange n! ways  Tree with n! leaves needed to sort n numbers  Given this many leaves, what is height of the tree?

9. Decision Tree Height  With n ! external nodes, binary tree’s height is:

10. The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

11. The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

12. The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(n log n)

13. Lower Bound on Sorting  Smallest number of comparisons is tree’s height  Decision tree sorting n elements has n! leaves  At least log( n !) height needed for this many leaves  As we saw, this simplifies to at most O(n log n) height  O(n log n) time needed to compare data!  Means that heap-sort is fastest possible (in big-Oh)  Pain-in-the- to code & requires external heap

14. Lower Bound on Sorting  Smallest number of comparisons is tree’s height  Decision tree sorting n elements has n! leaves  At least log( n !) height needed for this many leaves  As we saw, this simplifies to at most O(n log n) height  O(n log n) time needed to compare data!  Means that heap-sort is fastest possible (in big-Oh)  Pain-in-the- to code & requires external heap  Is there a simple sort using only Sequence ?

15. Julius, Seize Her!  Formula to Roman success  Divide peoples before an attack  Then conquer weakened armies  Common programming paradigm  Divide: split into 2 partitions  Recur: solve for partitions  Conquer: combine solutions

16. Divide-and-Conquer  Like all recursive algorithms, need base case  Has immediate solution to a simple problem  Work is not easy and sorting 2+ items takes work  Already sorted 1 item since it cannot be out of order  Sorting a list with 0 items even easer  Recursive step simplifies problem & combines it  Begins by splitting data into two equal Sequence s  Merges sub Sequence s after they have been sorted

17. Merge-Sort Algorithm mergeSort(Sequence S, Comparator C) if S.size() < 2 then // Base case return S else // Recursive case // Split S into two equal-sized partitions S 1 and S 2 mergeSort(S 1, C) mergeSort(S 2, C) S  merge(S 1, S 2, C) return S

18. Merging Sorted Sequences Algorithm merge(S 1, S 2, C) Sequence retVal = // Code instantiating a Sequence while  S 1.isEmpty() &&  S 2.isEmpty() if C.compare(S 1.get(0), S 2.get(0)) < 0 retVal.insertLast(S 1.removeFirst()) else retVal.insertLast(S 2.removeFirst()) endif end while  S 1.isEmpty() retVal.insertLast(S 1.removeFirst()) end while  S 2.isEmpty() retVal.insertLast(S 2.removeFirst()) end return retVal

19. Execution Tree  Depicts divide-and-conquer execution  Recursive call represented by each oval node  Original Sequence shown at start  At the end of the oval, sorted Sequence shown  Initial call at root of the (binary) tree  Bottom of the tree has leaves for base cases

20. Execution Example  Not in a base case 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

21. Execution Example  Not in a base case, so split into S 1 & S 2 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

22. Execution Example  Not in a base case, so split into S 1 & S 2 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

23. Execution Example  Recursively call merge-sort on S 1 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

24. Execution Example  Recursively call merge-sort on S 1 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

25. Execution Example  Not in a base case, so split into S 1 & S 2 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

26. Execution Example  Not in a base case, so split into S 1 & S 2 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

27. Execution Example  Recursively call merge-sort on S 1 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9

28. Execution Example  Recursively call merge-sort on S 1 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 7

29. Execution Example  Still no base case, so split again & recurse on S 1 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 7 7  7

30. Execution Example  Enjoy the base case – literally no work to do! 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 7 7  77  7

31. Execution Example  Recurse on S 2 and solve for this base case 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 7 7  77  72  22  2

32. Execution Example  Merge the two solutions to complete this call 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 7 7  77  72  22  2

33. Execution Example  Recurse on S 2 and sort this sub Sequence 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 7  77  72  22  2

34. Execution Example  Split into S 1 & S 2 and solve the base cases 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 9  94  44  47  77  72  22  2

35. Execution Example  Merge the 2 solutions to sort this Sequence 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 9  94  44  47  77  72  22  2

36. Execution Example  I feel an urge, an urge to merge 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 9  94  44  47  77  72  22  2

37. Execution Example  Let's do the merge sort again! (with S 2 ) 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 3 8 6 1  1 3 6 8 9  94  44  47  77  72  22  2

38. Execution Example  Let's do the merge sort again! (with S 2 ) 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 3 8 6 1  1 3 6 8 3 8  3 8 8  88  83  33  39  94  44  47  77  72  22  2

39. Execution Example  Let's do the merge sort again! (with S 2 ) 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 3 8 6 1  1 3 6 8 3 8  3 86 1  1 6 6  66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

40. Execution Example  Let's do the merge sort again! (with S 2 ) 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 3 8 6 1  1 3 6 8 3 8  3 86 1  1 6 6  66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

41. Execution Example  Merge the last call to get the final result 7 2 9 4  2 4 7 9 7 2 9 4 3 8 6 1  1 2 3 4 6 7 8 9 7 2  2 79 4  4 9 3 8 6 1  1 3 6 8 3 8  3 86 1  1 6 6  66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

42. For Next Lecture  New weekly assignment for week was posted  Discussing sorts which have few concepts to code  Will return soon enough; do not worry about it  Keep reviewing requirements for program #2  Preliminary deadlines arrive before final version  Time spent on requirements & design saves coding  Reading on quick sort for this Friday  Guess what? It can be really, really, quick  Severe drawbacks also exist; read to understand this