CSC 213 – Large Scale Programming

Today’s Goals  Review past discussion of data sorting algorithms  Weaknesses of past approaches & when we use them  Can we find limit to how long sorting needs?  What does this mean for sorting & those past sorts  Get good idea of how merge sort is executed  What is algorithm and what will this require?  What are execution trees & how they show runtime?

Ghosts of Sorts Past  We have already seen & discussed 4 sorts  Bubble-sort -- O ( n 2 ) time sort; slowest sort  Selection-sort -- O ( n 2 ) time sort; PQ concept  Insertion-sort -- O ( n 2 ) time sort; PQ concept O ( n log n )  Heap-sort -- O ( n log n ) time sort; requires PQ  All of these sorts of limited usefulness

Ghosts of Sorts Past  We have already seen & discussed 4 sorts  Bubble-sort -- O ( n 2 ) time sort; slowest sort  Selection-sort -- O ( n 2 ) time sort; PQ concept  Insertion-sort -- O ( n 2 ) time sort; PQ concept O ( n log n )  Heap-sort -- O ( n log n ) time sort; requires PQ  All of these sorts of limited usefulness

Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting yesno Is x i > x j ?

Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting  Traveling from root to leaf sorts data  Tree’s height is lower-bound on sorting complexity

Decision Tree Height  Unique leaf for each ordering of data initially  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)

Decision Tree Height  Unique leaf for each ordering of data initially  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)  For sequence of n numbers, can arrange n! ways  Tree with n! leaves needed to sort n numbers  Given this many leaves, what is height of the tree?

Decision Tree Height  With n ! external nodes, binary tree’s height is:

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(n log n)

Lower Bound on Sorting  Smallest number of comparisons is tree’s height  Decision tree sorting n elements has n! leaves  At least log( n !) height needed for this many leaves  As we saw, this simplifies to at most O(n log n) height  O(n log n) time needed to compare data!  Means that heap-sort is fastest possible (in big-Oh)  Pain-in-the- to code & requires external heap

Lower Bound on Sorting  Smallest number of comparisons is tree’s height  Decision tree sorting n elements has n! leaves  At least log( n !) height needed for this many leaves  As we saw, this simplifies to at most O(n log n) height  O(n log n) time needed to compare data!  Means that heap-sort is fastest possible (in big-Oh)  Pain-in-the- to code & requires external heap  Is there a simple sort using only Sequence ?

Julius, Seize Her!  Formula to Roman success  Divide peoples before an attack  Then conquer weakened armies  Common programming paradigm  Divide: split into 2 partitions  Recur: solve for partitions  Conquer: combine solutions

Divide-and-Conquer  Like all recursive algorithms, need base case  Has immediate solution to a simple problem  Work is not easy and sorting 2+ items takes work  Already sorted 1 item since it cannot be out of order  Sorting a list with 0 items even easer  Recursive step simplifies problem & combines it  Begins by splitting data into two equal Sequence s  Merges sub Sequence s after they have been sorted

Merge-Sort Algorithm mergeSort(Sequence S, Comparator C) if S.size() < 2 then // Base case return S else // Recursive case // Split S into two equal-sized partitions S 1 and S 2 mergeSort(S 1, C) mergeSort(S 2, C) S  merge(S 1, S 2, C) return S

Merging Sorted Sequences Algorithm merge(S 1, S 2, C) Sequence retVal = // Code instantiating a Sequence while  S 1.isEmpty() &&  S 2.isEmpty() if C.compare(S 1.get(0), S 2.get(0)) < 0 retVal.insertLast(S 1.removeFirst()) else retVal.insertLast(S 2.removeFirst()) endif end while  S 1.isEmpty() retVal.insertLast(S 1.removeFirst()) end while  S 2.isEmpty() retVal.insertLast(S 2.removeFirst()) end return retVal

Execution Tree  Depicts divide-and-conquer execution  Recursive call represented by each oval node  Original Sequence shown at start  At the end of the oval, sorted Sequence shown  Initial call at root of the (binary) tree  Bottom of the tree has leaves for base cases

Execution Example  Not in a base case 

Execution Example  Not in a base case, so split into S 1 & S 

Execution Example  Not in a base case, so split into S 1 & S 

Execution Example  Recursively call merge-sort on S 

Execution Example  Recursively call merge-sort on S  

Execution Example  Not in a base case, so split into S 1 & S  

Execution Example  Not in a base case, so split into S 1 & S  

Execution Example  Recursively call merge-sort on S  

Execution Example  Recursively call merge-sort on S    2 7

Execution Example  Still no base case, so split again & recurse on S     7

Execution Example  Enjoy the base case – literally no work to do!     77  7

Execution Example  Recurse on S 2 and solve for this base case     77  72  22  2

Execution Example  Merge the two solutions to complete this call     77  72  22  2

Execution Example  Recurse on S 2 and sort this sub Sequence      77  72  22  2

Execution Example  Split into S 1 & S 2 and solve the base cases      94  44  47  77  72  22  2

Execution Example  Merge the 2 solutions to sort this Sequence      94  44  47  77  72  22  2

Execution Example  I feel an urge, an urge to merge      94  44  47  77  72  22  2

Execution Example  Let's do the merge sort again! (with S 2 )       94  44  47  77  72  22  2

Execution Example  Let's do the merge sort again! (with S 2 )        88  83  33  39  94  44  47  77  72  22  2

Execution Example  Let's do the merge sort again! (with S 2 )         66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

Execution Example  Let's do the merge sort again! (with S 2 )         66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

Execution Example  Merge the last call to get the final result         66  61  11  18  88  83  33  39  94  44  47  77  72  22  2

For Next Lecture  New weekly assignment for week was posted  Discussing sorts which have few concepts to code  Will return soon enough; do not worry about it  Keep reviewing requirements for program #2  Preliminary deadlines arrive before final version  Time spent on requirements & design saves coding  Reading on quick sort for this Friday  Guess what? It can be really, really, quick  Severe drawbacks also exist; read to understand this