Sorting With Priority Queue In-place Extra O(N) space Small amount O(1) extra space not extra O(N) space Typical algorithms you have seen before © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Priority Queue Sorting Insert the elements one by one with a series of insert operations Remove the elements in sorted order with a series of removeMin operations Algorithm PQ-Sort(S, C) Input list S, comparator C for the elements of S Output list S sorted in increasing order according to C P  priority queue with comparator C while S.isEmpty () e  S.remove(S.first ()) P.insert (e, ) while P.isEmpty() e  P.removeMin().getKey() S.addLast(e) © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Selection-Sort with PQ [skip] Selection-sort is the variation of PQ-sort where the priority queue is implemented with an unsorted sequence Running time of Selection-sort: Inserting the elements into the priority queue with n insert operations takes O(n) time Removing the elements in sorted order from the priority queue with n removeMin operations takes time proportional to 1 + 2 + …+ n Selection-sort runs in O(n2) time © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Selection-Sort Example [skip] Sequence S Priority Queue P Input: (7,4,8,2,5,3,9) () Phase 1 (a) (4,8,2,5,3,9) (7) (b) (8,2,5,3,9) (7,4) .. .. .. (g) () (7,4,8,2,5,3,9) Phase 2 (a) (2) (7,4,8,5,3,9) (b) (2,3) (7,4,8,5,9) (c) (2,3,4) (7,8,5,9) (d) (2,3,4,5) (7,8,9) (e) (2,3,4,5,7) (8,9) (f) (2,3,4,5,7,8) (9) (g) (2,3,4,5,7,8,9) () © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Insertion-Sort with PQ [skip] Insertion-sort is the variation of PQ-sort where the priority queue is implemented with a sorted sequence Running time of Insertion-sort: Inserting the elements into the priority queue with n insert operations takes time proportional to 1 + 2 + …+ n Removing the elements in sorted order from the priority queue with a series of n removeMin operations takes O(n) time Insertion-sort runs in O(n2) time © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Insertion-Sort Example [skip] Sequence S Priority queue P Input: (7,4,8,2,5,3,9) () Phase 1 (a) (4,8,2,5,3,9) (7) (b) (8,2,5,3,9) (4,7) (c) (2,5,3,9) (4,7,8) (d) (5,3,9) (2,4,7,8) (e) (3,9) (2,4,5,7,8) (f) (9) (2,3,4,5,7,8) (g) () (2,3,4,5,7,8,9) Phase 2 (a) (2) (3,4,5,7,8,9) (b) (2,3) (4,5,7,8,9) .. .. .. (g) (2,3,4,5,7,8,9) () © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

(In-Place) Sorting Small amount of O(1) extra space Selection Sort not extra O(N) space Selection Sort Insertion Sort Bubble Sort ... more later © 2014 Goodrich, Tamassia, Goldwasser Priority Queues

Selection Sort To sort an array on integers in ascending order: Find the smallest number and record its index swap (interchange) the smallest number with the first element of the array the sorted part of the array is now the first element the unsorted part of the array is the remaining elements repeat Steps 2 and 3 until all elements have been placed each iteration increases the length of the sorted part by one

Selection Sort Example Key: smallest remaining value sorted elements Problem: sort this 10-element array of integers in ascending order: 1st iteration: smallest value is 3, its index is 4, swap a[0] with a[4] before: after: 2nd iteration: smallest value in remaining list is 5, its index is 6, swap a[1] with a[6] How many iterations are needed? Chapter 10 Java: an Introduction to Computer Science & Programming - Walter Savitch 9

Insertion Sort Basic Idea: Keeping expanding the sorted portion by one Insert the next element into the right position in the sorted portion Algorithm: Start with one element [is it sorted?] – sorted portion While the sorted portion is not the entire array Find the right position in the sorted portion for the next element Insert the element If necessary, move the other elements down Expand the sorted portion by one

Insertion Sort: An example First iteration Before: [5], 3, 4, 9, 2 After: [3, 5], 4, 9, 2 Second iteration Before: [3, 5], 4, 9, 2 After: [3, 4, 5], 9, 2 Third iteration Before: [3, 4, 5], 9, 2 After: [3, 4, 5, 9], 2 Fourth iteration Before: [3, 4, 5, 9], 2 After: [2, 3, 4, 5, 9]

Bubble Sort Basic Idea: Expand the sorted portion one by one “Sink” the largest element to the bottom after comparing adjacent elements The smaller items “bubble” up Algorithm: While the unsorted portion has more than one element Compare adjacent elements Swap elements if out of order Largest element at the bottom, reduce the unsorted portion by one

Bubble Sort: An example First Iteration: [5, 3], 4, 9, 2  [3, 5], 4, 9, 2 3, [5, 4], 9, 2  3, [4, 5], 9, 2 3, 4, [5, 9], 2  3, 4, [5, 9], 2 3, 4, 5, [9, 2]  3, 4, 5, [2, 9] Second Iteration: [3, 4], 5, 2, 9  [3, 4], 5, 2, 9 3, [4, 5], 2, 9  3, [4, 5], 2, 9 3, 4, [5, 2], 9  3, 4, [2, 5], 9 Third Iteration: [3, 4], 2, 5, 9  [3, 4], 2, 5, 9 3, [4, 2], 5, 9  3, [2, 4], 5, 9 Fourth Iteration: [3, 2], 4, 5, 9  [2, 3], 4, 5, 9

Summary of Worst-case Analysis Comparisons (more important) Moves Selection Insertion Bubble

Summary of Worst-case Analysis Comparisons (more important) Moves Selection O(N2) O(N) Insertion Bubble

Selection Sort Comparisons N – 1 iterations First iteration: how many comparisons? Second iteration: how many comparisons? (N – 1) + (N – 2) + … + 2 + 1 = N(N-1)/2 = (N2 – N)/2 Moves (worst case: every element is in the wrong location) First iteration: how many swaps/moves? Second iteration: how many swaps/moves? (N – 1) x 3 = 3N - 3

Insertion Sort Comparisons (worst case: correct order) N – 1 iterations First iteration: how many comparisons? Second iteration: how many comparisons? 1 + 2 + … + (N – 2) + (N – 1) = N(N-1)/2 = (N2 – N)/2 Moves (worst case: reverse order) First iteration: how many moves? Second iteration: how many moves? 3 + 4 + … + N + (N + 1) = (N + 4)(N - 1)/2 = (N2 + 3N - 4)/2

Bubble Sort Comparisons N – 1 iterations First iteration: how many comparisons? Second iteration: how many comparisons? (N – 1) + (N – 2) + … + 2 + 1 = N(N-1)/2 = (N2 – N)/2 Moves (worst case: reverse order) First iteration: how many swaps/moves? Second iteration: how many swaps/moves? [(N – 1) + (N – 2) + … + 2 + 1] x 3 = 3N(N-1)/2 = (3N2 – 3N)/2

Summary of Worst-case Analysis Comparisons (more important) Moves Selection (N2 – N)/2 3N - 3 Insertion (N2 + 3N - 4)/2 Bubble (3N2 – 3N)/2

Recall PQ Sorting We use a priority queue Algorithm PQ-Sort(S, C) Insert the elements with a series of insert operations Remove the elements in sorted order with a series of removeMin operations The running time depends on the priority queue implementation: Unsorted sequence gives selection-sort: O(n2) time Sorted sequence gives insertion-sort: O(n2) time Can we do better? Algorithm PQ-Sort(S, C) Input sequence S, comparator C for the elements of S Output sequence S sorted in increasing order according to C P  priority queue with comparator C while S.isEmpty () e  S.remove (S. first ()) P.insert (e, e) while P.isEmpty() e  P.removeMin().getKey() S.addLast(e) © 2014 Goodrich, Tamassia, Goldwasser Heaps

Sorting with a heap Consider a priority queue with n items implemented by means of a heap the space used is O(n) methods insert and removeMin take O(log n) time methods size, isEmpty, and min take time O(1) time Using a heap-based priority queue © 2014 Goodrich, Tamassia, Goldwasser Heaps

Time Complexity How many insertions? How many removeMin operations? Overall time complexity © 2014 Goodrich, Tamassia, Goldwasser Heaps

Time Complexity N insertions Each is O(log N) N removeMin operations Overall is O(N log N) © 2014 Goodrich, Tamassia, Goldwasser Heaps

Building the heap—first phase Inserting N entries one at a time More efficient algorithm Bottom-up heap construction © 2014 Goodrich, Tamassia, Goldwasser Heaps

Merging Two Heaps We are given two heaps and a key k 3 2 We are given two heaps and a key k We create a new heap with the root node storing k and with the two heaps as subtrees We perform downheap to restore the heap-order property 8 5 4 6 7 3 2 8 5 4 6 2 3 4 8 5 7 6 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Bottom-up Heap Construction We can construct a heap storing n given keys in using a bottom-up construction with log n phases In phase i, pairs of heaps with 2i -1 keys are merged into heaps with 2i+1-1 keys 2i -1 2i+1-1 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Example 16 15 4 12 6 7 23 20 25 5 11 27 16 15 4 12 6 7 23 20 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Example (contd.) 25 5 11 27 16 15 4 12 6 9 23 20 15 4 6 20 16 25 5 12 11 9 23 27 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Example (contd.) 7 8 15 4 6 23 16 25 5 12 11 9 27 20 4 6 15 5 8 20 16 25 7 12 11 9 23 27 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Example (end) 10 4 6 15 5 8 23 16 25 7 12 11 9 27 20 4 5 6 15 7 8 20 16 25 10 12 11 9 23 27 © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis Consider a complete binary tree, with each level filled Downheap occurs at each internal node and the root Worst-case is swapping down to a leaf © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path Each edge is involved in downheap, except those on the far left Total number of swaps < total number of edges How many edges total? © 2014 Goodrich, Tamassia, Goldwasser Heaps

Worst-case Analysis a proxy path that goes first right and then repeatedly goes left until the bottom of the heap this path may differ from the actual downheap path We care about the length of proxy path (not the proxy path itself) length of worst-case actual path Each edge is involved in downheap, except those on the far left Total number of swaps < total number of edges bottom-up heap construction runs in O(n) time © 2014 Goodrich, Tamassia, Goldwasser Heaps

Time Complexity of Sorting with a Heap Constructing the heap O(N) N RemoveMin operations O(N log N) Overall is O(N log N) © 2014 Goodrich, Tamassia, Goldwasser Heaps

Heap Sort Uses a heap in-place O(1) extra space © 2014 Goodrich, Tamassia, Goldwasser Heaps

Heap Sort Assuming ascending order is desirable Instead of parent <= child Child <= parent Root has the largest item removeMax instead of removeMin Then Swap max with the “last node” © 2014 Goodrich, Tamassia, Goldwasser Heaps

© 2014 Goodrich, Tamassia, Goldwasser Heaps

Time Complexity of Heap Sort Constructing the heap O(N) N RemoveMax operations O(N log N) Overall is O(N log N) © 2014 Goodrich, Tamassia, Goldwasser Heaps

Summary of In-place Sorting Algorithms (so far) Time Complexity Heap Sort O(N log N) Selection Sort O(N2) Insertion Sort Bubble Sort © 2014 Goodrich, Tamassia, Goldwasser Heaps