1. Chapter 11 Sorting Algorithms and their Efficiency
CS Data Structures
Mehmet H Gunes
Modified from authors’ slides

2. Contents Basic Sorting Algorithms Faster Sorting Algorithms
A Comparison of Sorting Algorithms
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

3. Sorting means . . .
Sorting rearranges the elements into either ascending or descending order within the array.
We’ll use ascending order.
The values stored in an array have keys of a type for which the relational operators are defined.
We also assume unique keys. 36 24 10 6 12 6 10 12 24 36

4. Basic Sorting Algorithms
Sorting is:
A process
It organizes a collection of data
Organized into ascending/descending order
Internal: data fits in memory
External: data must reside on secondary storage
Sort key: data item which determines order
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

5. Straight Selection Sort
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
Divides the array into two parts:
already sorted, and
not yet sorted.
On each pass,
finds the smallest of the unsorted elements, and
swaps it into its correct place,
thereby increasing the number of sorted elements by one. 36 24 10 6 12

6. Selection Sort: Pass One
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ] 36 24 10 6 12 U N S O R T E D

7. Selection Sort: End Pass One
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
SORTED 6 24 10 36 12 U N S O R T E D

8. Selection Sort: Pass Two
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
SORTED 6 24 10 36 12 U N S O R T E D

9. Selection Sort: End Pass Two
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
SORTED 6 10 24 36 12 U N S O R T E D

10. Selection Sort: Pass Three
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
SORTED 6 10 24 36 12 U N S O R T E D

11. Selection Sort: End Pass Three
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ] 6 10 12 36 24 S O R T E D
UNSORTED

12. Selection Sort: Pass Four
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ] 6 10 12 36 24 S O R T E D
UNSORTED

13. Selection Sort: End Pass Four
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ] 6 10 12 24 36 S O R T E D

14. template <class ItemType >
int MinIndex(ItemType values [ ], int start, int end)
// Post: Function value = index of the smallest value
// in values [start] . . values [end]. {
int indexOfMin = start ;
for(int index = start + 1 ; index <= end ; index++)
if (values[ index] < values [indexOfMin])
indexOfMin = index ;
return indexOfMin; }

15. template <class ItemType >
void SelectionSort (ItemType values[ ],
int numValues )
// Post: Sorts array values[0 . . numValues-1 ]
// into ascending order by key {
int endIndex = numValues - 1;
for (int current=0; current<endIndex; current++)
Swap (values[current],
values[MinIndex(values, current, endIndex)]); }

16. Selection Sort: How many comparisons?
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ] 6 10 12 24 36
4 compares for values[0]
3 compares for values[1]
2 compares for values[2]
1 compare for values[3] =

17. For selection sort in general
The number of comparisons when the array contains N elements is
Sum = (N-1) + (N-2)
Arithmetic series:
O(N2)

18. The Selection Sort A selection sort of an array of five integers
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

19. The Selection Sort
View implementation of the selection sort, Listing 11-1
Analysis
This is an O (n2 ) algorithm
If sorting a very large array, selection sort algorithm probably too inefficient to use
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

20. An insertion sort partitions the array into two regions
The Insertion Sort
Take each item from unsorted region, insert into its correct order in sorted region
An insertion sort partitions the array into two regions
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

21. Insertion Sort 36 24 10 6 12 values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
One by one, each as yet unsorted array element is inserted into its proper place with respect to the already sorted elements.
On each pass, this causes the number of already sorted elements to increase by one. 36 24 10 6 12

22. Insertion Sort
Works like someone who “inserts” one more card at a time into a hand of cards that are already sorted.
To insert 12,
we need to make room for it by moving first 36 and
then 24. 6 10 24 36 12

23. Insertion Sort
Works like someone who “inserts” one more card at a time into a hand of cards that are already sorted.
To insert 12,
we need to make room for it by moving first 36 and
then 24. 6 10 24 36 12

24. Insertion Sort
Works like someone who “inserts” one more card at a time into a hand of cards that are already sorted.
To insert 12,
we need to make room for it by moving first 36 and
then 24. 24 36 10 6 12

25. Insertion Sort
Works like someone who “inserts” one more card at a time into a hand of cards that are already sorted.
To insert 12,
we need to make room for it by moving first 36 and
then 24. 12 24 36 10 6

26. template <class ItemType >
void InsertionSort ( ItemType values[], int numValues)
// Post: Sorts array values[0 . . numValues-1 ] into
// ascending order by key {
for (int count = 0 ; count < numValues; count++)
InsertItem ( values , 0 , count ); }

27. template <class ItemType >
void InsertItem (ItemType values[], int start, int end)
// Post: Elements between values[start] and values
// [end] have been sorted into ascending order by key. {
bool finished = false ;
int current = end ;
bool moreToSearch = (current != start);
while (moreToSearch && !finished )
if (values[current] < values[current - 1])
Swap(values[current], values[current - 1);
current--;
moreToSearch = ( current != start ); }
else
finished = true;

28. A Snapshot of the Insertion Sort Algorithm

29. The Insertion Sort An insertion sort of an array of five integers
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

30. The Insertion Sort View implantation of insertion sort, Listing 11-3
Analysis
An algorithm of order O(n2)
Best case O(n)
Appropriate for few data items
Good for sorted or almost sorted arrays!
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

31. Bubble Sort
values [ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
Compares neighboring pairs of array elements,
- starting with the last array element, and
- swaps neighbors whenever they are not in correct order.
On each pass, this causes the smallest element to “bubble up” to its correct place in the array. 36 24 10 6 12

32. Snapshot of BubbleSort

33. Code for BubbleSort template<class ItemType>
void BubbleSort(ItemType values[], int numValues) {
int current = 0;
while (current < numValues-1)
BubbleUp(values, current, numValues-1);
current++; }

34. Code for BubbleUp template<class ItemType>
void BubbleUp(ItemType values[],
int startIndex,
int endIndex)
// Post: Adjacent pairs that are out of
// order have been switched between
// values[startIndex]..values[endIndex]
// beginning at values[endIndex]. {
for (int index=endIndex; index>startIndex; index--)
if (values[index] < values[index-1])
Swap(values[index], values[index-1]); }

35. Observations on BubbleSort
This algorithm is always O(N2).
There can be a large number of intermediate swaps.
Can this algorithm be improved?
Add a “flag” to exit iterations
if nothing changes in a single iteration

36. Code for BubbleSort2 template<class ItemType>
void BubbleSort2(ItemType values[], int numValues) {
int current = 0;
bool sorted = false;
while (current < numValues-1 && ! sorted)
BubbleUp2(values, current, numValues-1, sorted);
current++; }

37. Code for BubbleUp2 template<class ItemType>
void BubbleUp(ItemType values[],
int startIndex,
int endIndex,
bool& sorted )
// Post: Adjacent pairs that are out of
// order have been switched between
// values[startIndex]..values[endIndex]
// beginning at values[endIndex]. {
sorted = true;
for (int index=endIndex; index>startIndex; index--)
if (values[index] < values[index-1]){
Swap(values[index], values[index-1]);
sorted = false; }

38. BubbleSort2 Complexity
(N - 1) + (N - 2) + (N - 3) (N - K) = KN - (sum of 1 through K) = KN – K(K+1)/2 = (2KN - K2 - K) / 2
1st call nd call rd call Kth call
O(N2)
Good for almost in order items!

39. The Bubble Sort
The first two passes of a bubble sort of an array of five integers
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

40. The Bubble Sort
View an implementation of the bubble sort, Listing 11-2
Analysis
Best case, O(n) algorithm
Worst case, O(n2) algorithm
Again, a poor choice for large amounts of data
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

41. Divide and Conquer Sorts

42. Merge Sort Algorithm 74 36 . . . 95 75 29 . . . 52 [last]
Cut the array in half.
Sort the left half.
Sort the right half.
Merge the two sorted halves into one sorted array.
[last]
[first] [middle] [middle + 1]

43. Using Merge Sort Algorithm with N = 16

44. Example 1 2 3 4 5 6 7 8
middle = 4 1 2 3 4 7 5 6 8 1 2 5 3 4 7 6 8 1 5 2 3 4 7 6 8

45. Example (cont.) 1 2 3 4 5 6 7 8 1 2 3 4 7 5 6 8 1 2 5 3 4 7 6 8 1 5 2 3 4 7 6 8

46. Merge Step Merge two “sorted” lists into a new “sorted” list
Can be done in O(N) time N 1 2 3 4 5 6 7 8
N/2
N/2

47. // Recursive merge sort algorithm
template <class ItemType >
void MergeSort ( ItemType values[ ], int first, int last )
// Pre: first <= last
// Post: Array values[first..last] sorted into
// ascending order. {
if ( first < last ) // general case
int middle = ( first + last ) / 2;
MergeSort ( values, first, middle );
MergeSort ( values, middle + 1, last );
// now merge two subarrays
// values [ first middle ] with
// values [ middle + 1, last ].
Merge(values, first, middle, middle + 1, last); }

48. Merge Sort of N elements: How many comparisons?
The entire array can be subdivided into halves only log2N times.
Each time it is subdivided, function Merge is called to re-combine the halves.
Function Merge uses a temporary array to store the merged elements.
Merging is O(N) because it compares each element in the subarrays.
Copying elements back from the temporary array to the values array is also O(N).
MERGE SORT IS O(N*log2N).

49. The Merge Sort A merge sort with an auxiliary temporary array
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

50. The Merge Sort A merge sort of an array of six integers
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

51. The Merge Sort View implementation of the merge sort, Listing 11-4
Analysis
Merge sort is of order O(n  log n)
This is significantly faster than O(n2)
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

52. The Merge Sort A worst-case instance of the merge step in a merge sort
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

53. The Merge Sort
Levels of recursive calls to mergeSort , given an array of eight items
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

54. Using quick sort algorithm
A . . Z A . . L M . . Z A . . F G . . L M . . R S . . Z

55. Split
splitVal = 9
smaller values larger values
in left part in right part

56. Before call to function Split
splitVal = 9
GOAL: place splitVal in its proper position with
all values less than or equal to splitVal on its left
and all larger values on its right
values[first] [last]

57. After call to function Split
splitVal = 9
smaller values larger values
in left part in right part
values[first] [last]
splitVal in correct position

58. Example 1 3 2 4 6 7 6 7 1 3 2 4 3 2 3 4 2 3

59. // Recursive quick sort algorithm template <class ItemType >
void QuickSort ( ItemType values[ ], int first, int last )
// Pre: first <= last
// Post: Sorts array values[ first . . last ] into ascending order {
if ( first < last ) // general case
int splitPoint = first;
Split ( values, first, last, splitPoint ) ;
// values [first]..values[splitPoint - 1] <= splitVal
// values [splitPoint] = splitVal
// values [splitPoint + 1]..values[last] > splitVal
QuickSort(values, first, splitPoint - 1);
QuickSort(values, splitPoint + 1, last); }
} ;

60. Quick Sort of N elements: How many comparisons?
N For first call, when each of N elements is
compared to the split value
2 * N/2 For the next pair of calls, when N/2 elements
in each “half” of the original array are compared
to their own split values.
4 * N/4 For the four calls when N/4 elements in each
“quarter” of original array are compared to
their own split values. .
. HOW MANY SPLITS CAN OCCUR?

61. Quick Sort of N elements: How many splits can occur?
It depends on the order of the original array elements!
If each split divides the subarray approximately in half, there will be only log2N splits, and QuickSort is O(N*log2N).
But, if the original array was sorted to begin with, the recursive calls will split up the array into parts of unequal length, with one part empty, and the other part containing all the rest of the array except for split value itself.
In this case, there can be as many as N-1 splits, and QuickSort is O(N2).

62. Before call to function Split
splitVal = 9
GOAL: place splitVal in its proper position with
all values less than or equal to splitVal on its left
and all larger values on its right
values[first] [last]

63. After call to function Split
splitVal = 9
no smaller values larger values
empty left part in right part with N-1 elements
values[first] [last]
splitVal in correct position

64. Best case: balanced splits
O(nlogn)

65. Worst case: unbalanced splits2 1
O(n2)
O(n)

66. But … is every unbalanced split a bad split?
O(nlogn)
Need to look at split ratio!

67. Split ratio split ratio: (n/2) / (n/2) =const
O(nlogn)
O(nlogn)
split ratio: (n/2) / (n/2) =const
split ratio: (n/10) / (9n/10) =const

68. Split ratio (cont’d) n n - 1 n - 2 n - 3 2 1 O(n2)
split ratio: n / 1 = n not const

69. Randomized Quicksort
Randomly permute the elements of the input array before sorting.
Or, choose splitPoint randomly.

70. Randomized Quicksort
At each step of the algorithm we exchange element A[p] with an element chosen at random from A[p…r]
The pivot element x = A[p] is equally likely to be any one of the r – p + 1 elements of the subarray

71. Randomized Quicksort Worst case becomes less likely
Worst case occurs only if we get “unlucky” numbers from the random number generator.
Randomization can NOT eliminate the worst-case but it can make it less likely!

72. The Quick Sort A partition about a pivot
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

73. The Quick Sort Partitioning of array during quick sort
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

74. The Quick Sort Partitioning of array during quick sort
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

75. The Quick Sort
Median-of-three pivot selection: (a) The original array; (b) the array with its first, middle, and last entries sorted
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

76. The Quick Sort
(a) The array with its first, middle, and last entries sorted; (b) the array after positioning the pivot and just before partitioning
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

77. The Quick Sort Note function that performs a quick sort, Listing 11-5
Analysis
Worst case O(n2)
Average case O(n  log n)
Does not require extra memory like merge sort
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

78. The Quick Sort kSmall versus quickSort
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

79. Comparison of Sorting Algorithms

80. How Fast Can We Sort?
Selection Sort, Bubble Sort, Insertion Sort:
Heap Sort, Merge sort:
Quicksort:
What is common to all these algorithms?
Make comparisons between input elements
ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj
O(n2)
O(nlgn)
O(nlgn) - average

81. Lower-Bound for Sorting
Theorem: To sort n elements, comparison sorts must make (nlgn) comparisons in the worst case.
(see CS477 for a proof)

82. Can we do better? Linear sorting algorithms
Radix Sort
Counting Sort
Bucket sort
Make certain assumptions about the data
Linear sorts are NOT “comparison sorts”

83. Counting Sort Assumptions: Idea:
n integers which are in the range [0 ... r]
r is in the order of n, that is, r=O(n)
Idea:
For each element x, find the number of elements x
Place x into its correct position in the output array
output array

84. Step 1
(i.e., frequencies)
(r=6)

85. Step 2
Cnew C
(frequencies)
(cumulative sums)

86. Algorithm Start from the last element of A
Place A[i] at its correct place in the output array
Decrease C[A[i]] by one 3 2 5 1 4 6 7 8 A 7 4 2 1 3 5
Cnew 8

87. Example 7 4 2 Cnew 8 A 2 5 3 2 3 3 3 B Cnew 3 B Cnew 3 B Cnew 3 2 B1 2 3 4 5 6 7 8 7 4 2 1 3 5
Cnew 8 A 2 5 3 2 3 3 3 1 2 4 5 6 7 8 B
Cnew 3 1 2 4 5 6 7 8 B
Cnew 3 1 2 4 5 6 7 8 B
Cnew 3 2 1 4 5 6 7 8 B
Cnew

88. Example (cont.) A 2 5 3 2 3 3 3 2 B C 5 3 2 B C 3 2 B C 5 3 2 B 1 2 34 5 6 7 8 A 2 5 3 2 3 3 3 2 1 4 5 6 7 8 B C 5 3 2 1 4 6 7 8 B C 3 2 1 4 5 6 7 8 B C 5 3 2 1 4 6 7 8 B

89. COUNTING-SORT Alg.: COUNTING-SORT(A, B, n, k) for i ← 0 to r1 j n A
Alg.: COUNTING-SORT(A, B, n, k)
for i ← 0 to r
do C[ i ] ← 0
for j ← 1 to n
do C[A[ j ]] ← C[A[ j ]] + 1
C[i] contains the number of elements equal to i
for i ← 1 to r
do C[ i ] ← C[ i ] + C[i -1]
C[i] contains the number of elements ≤ i
for j ← n downto 1
do B[C[A[ j ]]] ← A[ j ]
C[A[ j ]] ← C[A[ j ]] - 1 k C 1 n B

90. Analysis of Counting Sort
Alg.: COUNTING-SORT(A, B, n, k)
for i ← 0 to r
do C[ i ] ← 0
for j ← 1 to n
do C[A[ j ]] ← C[A[ j ]] + 1
C[i] contains the number of elements equal to i
for i ← 1 to r
do C[ i ] ← C[ i ] + C[i -1]
C[i] contains the number of elements ≤ i
for j ← n downto 1
do B[C[A[ j ]]] ← A[ j ]
C[A[ j ]] ← C[A[ j ]] - 1
O(r)
O(n)
O(r)
O(n)
Overall time: O(n + r)

91. Analysis of Counting Sort
Overall time: O(n + r)
In practice we use COUNTING sort when r = O(n)
 running time is O(n)

92. Bucket Sort Assumption: Idea:
the input is generated by a random process that distributes elements uniformly over [0, 1)
Idea:
Divide [0, 1) into k equal-sized buckets ( k=Θ(n) )
Distribute the n input values into the buckets
Sort each bucket (e.g., using mergesort)
Go through the buckets in order, listing elements in each one
Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i
Output: elements A[i] sorted

93. Example - Bucket Sort Distribute Into buckets A .78 .17 .39 .26 .72
.94
.21
.12
.23
.68 B / 2 1
.17 / 3 2
.26
.21 / 4 3 / 5 4
Distribute
Into buckets 6 5 7 / 6 8 7
.78 / 9 8 10 9 /

94. Example - Bucket Sort Sort within each bucket / .12 .17 / .21 .23/ 1
.12 / 2
.21
.23 / 3 /
Sort within each
bucket 4 5 6 / 7
.72 / 8 9 /

95. Example - Bucket Sort Concatenate the lists from
.17
.12
.23
.26
.21
.39
.68
.78
.72 / / 1
.12 / 2
.21
.23 / 3 / 4
Concatenate the lists from
0 to n – 1 together, in order 5 / 6 7
.72 / 8 9 /

96. Analysis of Bucket Sort
Alg.: BUCKET-SORT(A, n)
for i ← 1 to n
do insert A[i] into list B[nA[i]]
for i ← 0 to k - 1
do sort list B[i] with mergesort sort
concatenate lists B[0], B[1], , B[n -1] together in order
return the concatenated lists
O(n)
k O(n/k log(n/k))
=O(nlog(n/k)
O(k)
O(n) (if k=Θ(n))

97. Radix Sort Represents keys as d-digit numbers in some base-k
key = x1x2...xd where 0≤xi≤k-1
Example: key=15
key10 = 15, d=2, k=10 where 0≤xi≤9
key2 = 1111, d=4, k= where 0≤xi≤1

98. Radix Sort Assumptions Sorting looks at one column at a time
d=O(1) and k =O(n)
Sorting looks at one column at a time
For a d digit number, sort the least significant digit first
Continue sorting on the next least significant digit,
until all digits have been sorted
Requires only d passes through the list

99. RADIX-SORT Alg.: RADIX-SORT(A, d) for i ← 1 to d
do use a stable sort to sort array A on digit i
(stable sort: preserves order of identical elements)

100. The Radix Sort
Uses the idea of forming groups, then combining them to sort a collection of data.
Consider collection of three letter groups
ABC, XYZ, BWZ, AAC, RLT, JBX, RDT, KLT, AEO, TLJ
Group strings by rightmost letter
(ABC, AAC) (TLJ) (AEO) (RLT, RDT, KLT) (JBX) (XYZ, BWZ)
Combine groups
ABC, AAC, TLJ, AEO, RLT, RDT, KLT, JBX, XYZ, BWZ
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

101. The Radix Sort Group strings by middle letter Combine groups
(AAC) (A B C, J B X) (R D T) (A E O) (T L J, R L T, K L T) (B W Z) (X Y Z)
Combine groups
AAC, ABC, JBX, RDT, AEO, TLJ, RLT, KLT, BWZ, XYZ
Group by first letter, combine again
( A AC, A BC, A EO) ( B WZ) ( J BX) ( K LT) ( R DT, R LT) ( T LJ) ( X YZ)
Sorted strings
AAC, ABC, AEO, BWZ, JBX, KLT, RDT, RLT, TLJ, XYZ
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

102. The Radix Sort A radix sort of eight integers
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

103. The Radix Sort Analysis Has order O(n)
Large n requires significant amount of memory, especially if arrays are used
Memory can be saved by using chain of linked nodes
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

104. Analysis of Radix Sort
Given n numbers of d digits each, where each digit may take up to k possible values, RADIX-SORT correctly sorts the numbers in O(d(n+k))
One pass of sorting per digit takes O(n+k) assuming that we use counting sort
There are d passes (for each digit)
Assuming d=O(1) and k=O(n), running time is O(n)

105. Stable Sorting Algorithms
O(N2)
Selection Sort, Insertion Sort, Bubble Sort
O(N logN)
Merge Sort
O(N)
Counting Sort, Bucket Sort, Radix Sort

106. Comparison of Sorting Algorithms
Approximate growth rates of time required for eight sorting algorithms
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

108. Heap Sort: Recall that . . .
A heap is a binary tree that satisfies these
special SHAPE and ORDER properties:
Its shape must be a complete binary tree.
For each node in the heap, the value stored in that node is greater than or equal to the value in each of its children.

109. The largest element in a heap
is always found in the root node
root 70 12 60 30 8 10 40

110. The heap can be stored in an array
values
root
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ] 70 60 12 40 30 8 10 70 60 1 12 2 30 4 10 6 40 3 8 5

111. Heap Sort Approach
First, make the unsorted array into a heap by satisfying the order property.
Then repeat the steps below until there are no more unsorted elements.
Take the root (maximum) element off the heap by swapping it into its correct place in the array at the end of the unsorted elements.
Reheap the remaining unsorted elements.
This puts the next-largest element into the root position.

112. After creating the original heap
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 70 60 12 40 30 8 10 70 60 1 12 2 30 4 10 6 40 3 8 5

113. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 70 60 12 40 30 8 10 70 60 1 12 2 30 4 10 6 40 3 8 5

114. After swapping root element into it place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 10 60 12 40 30 8 70 10 60 1 12 2 30 4 40 3 8 5
NO NEED TO CONSIDER AGAIN

115. After reheaping remaining unsorted elements
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 60 40 12 10 30 8 70 60 40 1 12 2 30 4 10 3 8 5

116. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 60 40 12 10 30 8 70 60 40 1 12 2 30 4 10 3 8 5

117. After swapping root element into its place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 8 40 12 10 30 60 70 8 40 1 12 2 30 4 10 3
NO NEED TO CONSIDER AGAIN

118. After reheaping remaining unsorted elements
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 40 30 12 10 6 60 70 40 30 1 12 2 6 4 10 3

119. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 40 30 12 10 6 60 70 40 30 1 12 2 6 4 10 3

120. After swapping root element into its place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 6 30 12 10 40 60 70 6 30 1 12 2 10 3
NO NEED TO CONSIDER AGAIN

121. After reheaping remaining unsorted elements
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 30 10 12 6 40 60 70 30 10 1 12 2 6 3

122. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 30 10 12 6 40 60 70 30 10 1 12 2 6 3

123. After swapping root element into its place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 6 10 12 30 40 60 70 6 10 1 12 2
NO NEED TO CONSIDER AGAIN

124. After reheaping remaining unsorted elements
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 12 10 6 30 40 60 70 12 10 1 6 2

125. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 12 10 6 30 40 60 70 12 10 1 6 2

126. After swapping root element into its place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 6 10 12 30 40 60 70 6 10 1
NO NEED TO CONSIDER AGAIN

127. After reheaping remaining unsorted elements
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 10 6 12 30 40 60 70 10 6 1

128. Swap root element into last place in unsorted array
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 10 6 12 30 40 60 70 10 6 1

129. After swapping root element into its place
values
[ 0 ]
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
root 6 10 12 30 40 60 70 6
ALL ELEMENTS ARE SORTED

130. template <class ItemType >
void HeapSort ( ItemType values[], int numValues )
// Post: Sorts array values[ numValues-1 ] into
// ascending order by key {
int index ;
// Convert array values[0..numValues-1] into a heap
for (index = numValues/2 - 1; index >= 0; index--)
ReheapDown ( values , index , numValues-1 ) ;
// Sort the array.
for (index = numValues-1; index >= 1; index--)
Swap (values[0], values[index]);
ReheapDown (values , 0 , index - 1); }

131. ReheapDown template< class ItemType >
void ReheapDown ( ItemType values[], int root, int bottom )
// Pre: root is the index of a node that may violate the
// heap order property
// Post: Heap order property is restored between root and
// bottom {
int maxChild ;
int rightChild ;
int leftChild ;
leftChild = root * ;
rightChild = root * ;

132. if (leftChild <= bottom) // ReheapDown continued{
if (leftChild == bottom)
maxChild = leftChild;
else
if (values[leftChild] <= values [rightChild])
maxChild = rightChild ;
maxChild = leftChild ; }
if (values[ root ] < values[maxChild])
Swap (values[root], values[maxChild]);
ReheapDown ( maxChild, bottom ) ;

133. Heap Sort: How many comparisons?
In reheap down, an element
is compared with its 2 children
(and swapped with the larger).
But only one element at each
level makes this comparison,
and a complete binary tree
with N nodes has only O(log2N) levels.
root 24 60 1 12 2 30 3 40 4 8 5 10 6 15 7 6 8 18 9 70 10

134. Heap Sort of N elements: How many comparisons?
(N/2) * O(log N) compares to create original heap
(N-1) * O(log N) compares for the sorting loop
= O ( N * log N) compares total
(reheap takes O(log N) )