1. Heap Sort Example
Qamar Abbas

2. Why study Heapsort?
It is a well-known, traditional sorting algorithm you will be expected to know
Heapsort is always O(n log n)
Quicksort is usually O(n log n) but in the worst case slows to O(n2)
Quicksort is generally faster, but Heapsort is better in time-critical applications
Heapsort is a really cool algorithm!

3. What is a “heap”? Definitions of heap:
A large area of memory from which the programmer can allocate blocks as needed, and deallocate them (or allow them to be garbage collected) when no longer needed
A balanced, left-justified binary tree in which no node has a value greater than the value in its parent
These two definitions have little in common
Heapsort uses the second definition

4. The Heap Data Structure
Def: A heap is a nearly complete binary tree with the following two properties:
Structural property: all levels are full, except possibly the last one, which is filled from left to right
Order (heap) property: for any node x
Parent(x) ≥ x 8
It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 7 4 5 2
Heap

5. Representation of Heaps
A heap can be stored as an array A.
Root of tree is A[1]
Left child of A[i] = A[2i]
Right child of A[i] = A[2i + 1]
Parent of A[i] = A[ i/2 ]
Heapsize[A] ≤ length[A]
The elements in the subarray A[(n/2+1) .. n] are leaves
The root is the maximum element of the heap
floor(x) =x
A heap is a binary tree that is filled in order

6. The heap property
A node has the heap property if the value in the node is as large as or larger than the values in its children 12 8 3
Blue node has heap property 12 8
Blue node has heap property 12 8 14
Blue node does not have heap property
All leaf nodes automatically have the heap property
A binary tree is a heap if all nodes in it have the heap property

7. siftUp
Given a node that does not have the heap property, you can give it the heap property by exchanging its value with the value of the larger child 12 8 14
Blue node does not have heap property 14 8 12
Blue node has heap property
This is sometimes called sifting up
Notice that the child may have lost the heap property

8. Operations on Heaps Maintain the max-heap property
MAX-HEAPIFY
Create a max-heap from an unordered array
BUILD-MAX-HEAP
Sort an array in place
HEAPSORT
Priority queue operations

9. Constructing a heap
A tree consisting of a single node is automatically a heap
We construct a heap by adding nodes one at a time:
Add the node just to the right of the rightmost node in the deepest level
If the deepest level is full, start a new level
Examples:
Add a new node here
Add a new node here

10. Constructing a heap
Each time we add a node, we may destroy the heap property of its parent node
To fix this, we sift up
But each time we sift up, the value of the topmost node in the sift may increase, and this may destroy the heap property of its parent node
We repeat the sifting up process, moving up in the tree, until either
We reach nodes whose values don’t need to be swapped (because the parent is still larger than both children), or
We reach the root

11. Constructing a heap 8 8 10 10 8 10 8 5 1 2 3 10 8 5 12 10 12 5 8 12 10 5 8 4

12. Other children are not affected12 10 5 8 14 12 14 5 8 10 14 12 5 8 10
The node containing 8 is not affected because its parent gets larger, not smaller
The node containing 5 is not affected because its parent gets larger, not smaller
The node containing 8 is still not affected because, although its parent got smaller, its parent is still greater than it was originally

13. Heap Sort Example 25 22 17 19 11 14 15
Built Heap 17 22 25 19 11 14 15

14. Pass-1 Heap Sort Example17 22 25 19 11 14 15

15. Pass-1 Heap Sort Example17 22 25 19 11 14 15
Note: when right most or left node is replaced with root node then check the new root with both left and right and replace root with greatest from current left/right node

16. Pass-1 Heap Sort Example15 22 17 19 11 14

17. Pass-1 Heap Sort Example22 15 17 19 11 14

18. Pass-1 Heap Sort Example22 19 17 15 11 14 22 19 17 15 11 14 25

19. Pass-II Heap Sort Example19 15 17 14 11 19 15 17 14 11 22 25

20. Pass-III Heap Sort Example17 15 11 14 17 15 11 14 19 22 25

21. Pass-IV Heap Sort Example15 14 11 15 14 11 17 19 22 25

22. Pass-V Heap Sort Example14 11 14 11 15 17 19 22 25

23. Pass-V Heap Sort Example11 11 14 15 17 19 22 25

24. Pass-VI Sorted Heap 11 14 15 17 19 22 25

25. Alg: HEAPSORT(A) BUILD-MAX-HEAP(A) for i ← length[A] downto 2
do exchange A[1] ↔ A[i]
MAX-HEAPIFY(A, 1, i - 1)
Running time: O(nlgn)
O(n)
O(lgn)
n-1 times
In MAX-HEAPIFY(A, 1, i - 1) , i=i-1 in each iteration

26. Example: A=[7, 4, 3, 1, 2] MAX-HEAPIFY(A, 1, 2) MAX-HEAPIFY(A, 1, 3)
Deepest right most first and if no deepest right most then deepest lest – in both cases it will be the last element of the array

27. HEAP-EXTRACT-MAX Alg: HEAP-EXTRACT-MAX(A, n) if n < 1
then error “heap underflow”
max ← A[1]
A[1] ← A[n]
MAX-HEAPIFY(A, 1, n-1) remakes heap
return max
Running time: O(lgn)

28. Example: HEAP-EXTRACT-MAX8 2 4 14 7 1 16 10 9 3 8 2 4 14 7 1 10 9 3
max = 16
Heap size decreased with 1
If parent is smaller then replace this parent with the maximum of (left, right) children i.e if left is greater then replace with left else replace with right 4 2 1 8 7 14 10 9 3
Call MAX-HEAPIFY(A, 1, n-1)

29. Example: HEAP-EXTRACT-MAX8 2 4 14 7 1 16 10 9 3 8 2 4 14 7 1 10 9 3
max = 16
Heap size decreased with 1
If parent is smaller then replace this parent with the maximum of (left, right) children i.e if left is greater then replace with left else replace with right 4 2 1 8 7 14 10 9 3
Call MAX-HEAPIFY(A, 1, n-1)

30. A sample heap Here’s a sample binary tree after it has been heapified19 14 18 22 3 21 11 9 15 25 17
Notice that heapified does not mean sorted
Heapifying does not change the shape of the binary tree; this binary tree is balanced and left-justified because it started out that way

31. Removing the root Notice that the largest number is now in the root
Suppose we discard the root: 11 19 14 18 22 3 21 11 9 15 17
Note: if parent is smaller than its child then swap the parent with greater child.
How can we fix the binary tree so it is once again balanced and left- justified?
Solution: remove the rightmost leaf at the deepest level and use it for the new root

32. The reHeap method We can siftUp() the root
Our tree is balanced and left-justified, but no longer a heap
However, only the root lacks the heap property 19 14 18 22 3 21 9 15 17 11
We can siftUp() the root
After doing this, one and only one of its children may have lost the heap property

33. The reHeap method
Now the left child of the root (still the number 11) lacks the heap property 19 14 18 22 3 21 9 15 17 11
We can siftUp() this node
After doing this, one and only one of its children may have lost the heap property

34. The reHeap method
Now the right child of the left child of the root (still the number 11) lacks the heap property: 19 14 18 11 3 21 9 15 17 22
We can siftUp() this node
After doing this, one and only one of its children may have lost the heap property —but it doesn’t, because it’s a leaf

35. The reHeap method
Our tree is once again a heap, because every node in it has the heap property 19 14 18 21 3 11 9 15 17 22
Once again, the largest (or a largest) value is in the root
We can repeat this process until the tree becomes empty
This produces a sequence of values in order largest to smallest

36. Heap insertion and Deletion
Both the insert and remove operations modify the heap to conform to the shape property first, by adding or removing from the end of the heap. Then the heap property is restored by traversing up or down the heap. Both operations take O(log n) time.

37. Heap Insert Add the element to the bottom level of the heap.
Compare the added element with its parent; if they are in the correct order, stop.
If not, swap the element with its parent and return to the previous step.
The number of operations required is dependent on the amount of levels the new element must rise to satisfy the heap property, thus the insertion operation has a time complexity of O(log n).
As an example, say we have a max-heap
O(log n) means only one child (either left/right) will be used in the sift up operation that maximum takes O(log n)

38. Heap Insert

39. Heap Insert
and we want to add the number 15 to the heap. We first place the 15 in the position marked by the X. However, the heap property is violated since 15 is greater than 8, so we need to swap the 15 and the 8. So, we have the heap looking as follows after the first swap:

40. Heap Insert
However the heap property is still violated since 15 is greater than 11, so we need to swap again:

41. Heap Delete
Replace the root of the heap with the last element on the last level.
Compare the new root with its children; if they are in the correct order, stop.
If not, swap the element with one of its children and return to the previous step. (Swap with its smaller child in a min- heap and its larger child in a max-heap.)

42. Heap Delete
 if we have the same max-heap as before, we remove the 11 and replace it with the 4

43. Heap Delete
Now the heap property is violated since 8 is greater than 4. In this case, swapping the two elements, 4 and 8, is enough to restore the heap property and we need not swap elements further:

44. Max Heapify Overview of the algorithm:
Inputs: Array A (or binary tree)
index i in array
Precondition:
The binary trees rooted at LEFT(i) and RIGHT(i) are heaps
Note: A[i] may be smaller than its children
Postcondition:
The subtree rooted at index i is a heap

45. Max-Heapify Max-Heapify (A, i): left ← 2i right ← 2i + 1 largest ← i
 if left ≤ heap_length[A] and A[left] > A[largest] then:  largest ← left
 if right ≤ heap_length[A] and A[right] > A[largest] then:  largest ← right  
if largest ≠ i then:   swap A[i] ↔ A[largest]  Max-Heapify(A, largest)
if left ≤ heap_length[A] of if right ≤ heap_length[A] is false it means that “the node is a leaf node (having no child)”-----in case of greater means leaf
In comparison, leaf is not used, its parent is used
i is index, and Largest is also an index