1. Sorting With Priority Queue In-place Extra O(N) space
Small amount O(1) extra space
not extra O(N) space
Typical algorithms you have seen before
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

2. Priority Queue Sorting
Insert the elements one by one with a series of insert operations
Remove the elements in sorted order with a series of removeMin operations
Algorithm PQ-Sort(S, C)
Input list S, comparator C for the elements of S
Output list S sorted in increasing order according to C
P  priority queue with comparator C
while S.isEmpty ()
e  S.remove(S.first ())
P.insert (e, )
while P.isEmpty()
e  P.removeMin().getKey()
S.addLast(e)
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

3. Selection-Sort with PQ [skip]
Selection-sort is the variation of PQ-sort where the priority queue is implemented with an unsorted sequence
Running time of Selection-sort:
Inserting the elements into the priority queue with n insert operations takes O(n) time
Removing the elements in sorted order from the priority queue with n removeMin operations takes time proportional to …+ n
Selection-sort runs in O(n2) time
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

4. Selection-Sort Example [skip]
Sequence S Priority Queue P
Input: (7,4,8,2,5,3,9) ()
Phase 1
(a) (4,8,2,5,3,9) (7)
(b) (8,2,5,3,9) (7,4)
(g) () (7,4,8,2,5,3,9)
Phase 2
(a) (2) (7,4,8,5,3,9)
(b) (2,3) (7,4,8,5,9)
(c) (2,3,4) (7,8,5,9)
(d) (2,3,4,5) (7,8,9)
(e) (2,3,4,5,7) (8,9)
(f) (2,3,4,5,7,8) (9)
(g) (2,3,4,5,7,8,9) ()
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

5. Insertion-Sort with PQ [skip]
Insertion-sort is the variation of PQ-sort where the priority queue is implemented with a sorted sequence
Running time of Insertion-sort:
Inserting the elements into the priority queue with n insert operations takes time proportional to
…+ n
Removing the elements in sorted order from the priority queue with a series of n removeMin operations takes O(n) time
Insertion-sort runs in O(n2) time
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

6. Insertion-Sort Example [skip]
Sequence S Priority queue P
Input: (7,4,8,2,5,3,9) ()
Phase 1
(a) (4,8,2,5,3,9) (7)
(b) (8,2,5,3,9) (4,7)
(c) (2,5,3,9) (4,7,8)
(d) (5,3,9) (2,4,7,8)
(e) (3,9) (2,4,5,7,8)
(f) (9) (2,3,4,5,7,8)
(g) () (2,3,4,5,7,8,9)
Phase 2
(a) (2) (3,4,5,7,8,9)
(b) (2,3) (4,5,7,8,9)
(g) (2,3,4,5,7,8,9) ()
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

7. (In-Place) Sorting Small amount of O(1) extra space Selection Sort
not extra O(N) space
Selection Sort
Insertion Sort
Bubble Sort
... more later
© 2014 Goodrich, Tamassia, Goldwasser
Priority Queues

8. Selection Sort To sort an array on integers in ascending order:
Find the smallest number and record its index
swap (interchange) the smallest number with the first element of the array
the sorted part of the array is now the first element
the unsorted part of the array is the remaining elements
repeat Steps 2 and 3 until all elements have been placed
each iteration increases the length of the sorted part by one

9. Selection Sort Example
Key:
smallest remaining value
sorted elements
Problem: sort this 10-element array of integers in ascending order:
1st iteration: smallest value is 3, its index is 4, swap a[0] with a[4]
before:
after:
2nd iteration: smallest value in remaining list is 5, its index is 6, swap a[1] with a[6]
How many iterations are needed?
Chapter 10
Java: an Introduction to Computer Science & Programming - Walter Savitch 9

10. Insertion Sort Basic Idea: Keeping expanding the sorted portion by one
Insert the next element into the right position in the sorted portion
Algorithm:
Start with one element [is it sorted?] – sorted portion
While the sorted portion is not the entire array
Find the right position in the sorted portion for the next element
Insert the element
If necessary, move the other elements down
Expand the sorted portion by one

11. Insertion Sort: An example
First iteration
Before: [5], 3, 4, 9, 2
After: [3, 5], 4, 9, 2
Second iteration
Before: [3, 5], 4, 9, 2
After: [3, 4, 5], 9, 2
Third iteration
Before: [3, 4, 5], 9, 2
After: [3, 4, 5, 9], 2
Fourth iteration
Before: [3, 4, 5, 9], 2
After: [2, 3, 4, 5, 9]

12. Bubble Sort Basic Idea: Expand the sorted portion one by one
“Sink” the largest element to the bottom after comparing adjacent elements
The smaller items “bubble” up
Algorithm:
While the unsorted portion has more than one element
Compare adjacent elements
Swap elements if out of order
Largest element at the bottom, reduce the unsorted portion by one

13. Bubble Sort: An example
First Iteration:
[5, 3], 4, 9, 2  [3, 5], 4, 9, 2
3, [5, 4], 9, 2  3, [4, 5], 9, 2
3, 4, [5, 9], 2  3, 4, [5, 9], 2
3, 4, 5, [9, 2]  3, 4, 5, [2, 9]
Second Iteration:
[3, 4], 5, 2, 9  [3, 4], 5, 2, 9
3, [4, 5], 2, 9  3, [4, 5], 2, 9
3, 4, [5, 2], 9  3, 4, [2, 5], 9
Third Iteration:
[3, 4], 2, 5, 9  [3, 4], 2, 5, 9
3, [4, 2], 5, 9  3, [2, 4], 5, 9
Fourth Iteration:
[3, 2], 4, 5, 9  [2, 3], 4, 5, 9

14. Summary of Worst-case Analysis
Comparisons
(more important)
Moves
Selection
Insertion
Bubble

15. Summary of Worst-case Analysis
Comparisons
(more important)
Moves
Selection
O(N2)
O(N)
Insertion
Bubble

16. Selection Sort Comparisons N – 1 iterations
First iteration: how many comparisons?
Second iteration: how many comparisons?
(N – 1) + (N – 2) + … = N(N-1)/2 = (N2 – N)/2
Moves (worst case: every element is in the wrong location)
First iteration: how many swaps/moves?
Second iteration: how many swaps/moves?
(N – 1) x 3 = 3N - 3

17. Insertion Sort Comparisons (worst case: correct order)
N – 1 iterations
First iteration: how many comparisons?
Second iteration: how many comparisons?
… + (N – 2) + (N – 1) = N(N-1)/2 = (N2 – N)/2
Moves (worst case: reverse order)
First iteration: how many moves?
Second iteration: how many moves?
… + N + (N + 1) = (N + 4)(N - 1)/2 = (N2 + 3N - 4)/2

18. Bubble Sort Comparisons N – 1 iterations
First iteration: how many comparisons?
Second iteration: how many comparisons?
(N – 1) + (N – 2) + … = N(N-1)/2 = (N2 – N)/2
Moves (worst case: reverse order)
First iteration: how many swaps/moves?
Second iteration: how many swaps/moves?
[(N – 1) + (N – 2) + … ] x 3 = 3N(N-1)/2 = (3N2 – 3N)/2

19. Summary of Worst-case Analysis
Comparisons
(more important)
Moves
Selection
(N2 – N)/2
3N - 3
Insertion
(N2 + 3N - 4)/2
Bubble
(3N2 – 3N)/2

20. Recall PQ Sorting We use a priority queue Algorithm PQ-Sort(S, C)
Insert the elements with a series of insert operations
Remove the elements in sorted order with a series of removeMin operations
The running time depends on the priority queue implementation:
Unsorted sequence gives selection-sort: O(n2) time
Sorted sequence gives insertion-sort: O(n2) time
Can we do better?
Algorithm PQ-Sort(S, C)
Input sequence S, comparator C for the elements of S
Output sequence S sorted in increasing order according to C
P  priority queue with comparator C
while S.isEmpty ()
e  S.remove (S. first ())
P.insert (e, e)
while P.isEmpty()
e  P.removeMin().getKey()
S.addLast(e)
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

21. Sorting with a heap
Consider a priority queue with n items implemented by means of a heap
the space used is O(n)
methods insert and removeMin take O(log n) time
methods size, isEmpty, and min take time O(1) time
Using a heap-based priority queue
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

22. Time Complexity How many insertions? How many removeMin operations?
Overall time complexity
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

23. Time Complexity N insertions Each is O(log N) N removeMin operations
Overall is O(N log N)
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

24. Building the heap—first phase
Inserting N entries one at a time
More efficient algorithm
Bottom-up heap construction
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

25. Merging Two Heaps We are given two heaps and a key k3 2
We are given two heaps and a key k
We create a new heap with the root node storing k and with the two heaps as subtrees
We perform downheap to restore the heap-order property 8 5 4 6 7 3 2 8 5 4 6 2 3 4 8 5 7 6
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

26. Bottom-up Heap Construction
We can construct a heap storing n given keys in using a bottom-up construction with log n phases
In phase i, pairs of heaps with 2i -1 keys are merged into heaps with 2i+1-1 keys
2i -1
2i+1-1
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

27. Example 16 15 4 12 6 7 23 20 25 5 11 27 16 15 4 12 6 7 23 20
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

28. Example (contd.) 25 5 11 27 16 15 4 12 6 9 23 20 15 4 6 20 16 25 5 12 11 9 23 27
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

29. Example (contd.) 7 8 15 4 6 23 16 25 5 12 11 9 27 20 4 6 15 5 8 20 16 25 7 12 11 9 23 27
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

30. Example (end) 10 4 6 15 5 8 23 16 25 7 12 11 9 27 20 4 5 6 15 7 8 20 16 25 10 12 11 9 23 27
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

31. Worst-case Analysis
Consider a complete binary tree, with each level filled
Downheap occurs at each internal node and the root
Worst-case is swapping down to a leaf
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

32. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

33. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

34. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

35. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

36. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
Each edge is involved in downheap, except those on the far left
Total number of swaps < total number of edges
How many edges total?
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

37. Worst-case Analysis
a proxy path that goes first right and then repeatedly goes left until the bottom of the heap
this path may differ from the actual downheap path
We care about the length of proxy path (not the proxy path itself)
length of worst-case actual path
Each edge is involved in downheap, except those on the far left
Total number of swaps < total number of edges
bottom-up heap construction runs in O(n) time
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

38. Time Complexity of Sorting with a Heap
Constructing the heap
O(N)
N RemoveMin operations
O(N log N)
Overall is O(N log N)
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

39. Heap Sort Uses a heap in-place O(1) extra space
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

40. Heap Sort Assuming ascending order is desirable
Instead of parent <= child
Child <= parent
Root has the largest item
removeMax instead of removeMin
Then Swap max with the “last node”
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

41. © 2014 Goodrich, Tamassia, Goldwasser
Heaps

42. Time Complexity of Heap Sort
Constructing the heap
O(N)
N RemoveMax operations
O(N log N)
Overall is O(N log N)
© 2014 Goodrich, Tamassia, Goldwasser
Heaps

43. Summary of In-place Sorting Algorithms (so far)
Time Complexity
Heap Sort
O(N log N)
Selection Sort
O(N2)
Insertion Sort
Bubble Sort
© 2014 Goodrich, Tamassia, Goldwasser
Heaps