1. Topic 5: Heap data structure heap sort Priority queue
CSE 2331/5331
Topic 5:
Heap
data structure
heap sort
Priority queue
CSE 2331/5331

2. Heapsort combines best of two
Sorting Revisited
Insertion sort
quadratic, but in-place sorting
Merge sort
nlg n, but not in-place
Heapsort combines best of two
CSE 2331/5331

3. Heap Introduction
An array A that can be viewed as a nearly complete binary tree
An array A representing a heap:
length(A): size of the array
heap-size(A) (≤ length(A)):
Only A[1, …, heap-size(A)] contain elements of the heap.
CSE 2331/5331

4. Array? Tree?
Index: 1 2 3 4 5 6 7 8 9 10
Left( i )
Return (2 i )
Right( i )
Return (2 i + 1 )
Parent( i )
Return (  i / 2  ) 16 14 10 7 8 3 9 4 2 1
Only nodes on bottom right of tree are missing
CSE 2331/5331

5. Array? Tree?
Index: 1 2 3 4 5 6 7 8 9 10
Let n = heap-size(A)
Depth of i’th element:
 lg i 
Height of the tree:
 lg n 
Leaves:
 n/2  +1 -> n 16 14 10 7 8 3 9
A[1] is the root, A[heap-size] is the last leaf. 4 2 1
CSE 2331/5331

6. Max-heap (Max-)heap property: A [ Parent( i ) ] ≥ A[ i ] 16 14 10 7 83 9 4 2 1 16 14 10 7 8 3 9 4 2 1
CSE 2331/5331

7. Max-heap (Max-)heap property: A [ Parent( i ) ] ≥ A[ i ] 16 14 10 7 83 9 4 2 1 16 14 10 7 8 3 9 4 2 1
For any node i, A[i] is the
maximum among all the
values in the subtree rooted at i.
CSE 2331/5331

8. Max-heap (Max-)heap property: A [ Parent( i ) ] ≥ A[ i ] 16 14 10 7 83 9 4 2 1
CSE 2331/5331

9. Heap Operations
Max-Heapify
Build-MaxHeap
HeapSort
CSE 2331/5331

10. Crucial in maintain heap property
Max-Heapify
Input:
For a node i, left and right subtrees of i are max-heaps
A[i] may be smaller than its children (violates max-heap property)
Output:
The subtree rooted at i is a max-heap.
Crucial in maintain heap property
CSE 2331/5331

11. Max-heapify Operation
Max-heapify( A, i, n ) :
Ex:
Max-heapify(A, 2) 16 4 10 7 14 3 9 8 2 1 4 14 8 14 4 4
in-place operation
CSE 2331/5331

12. Pseudo-code Writing recurrence is not always the best approach.
Running time:
O(height of i) = O(lg n)
CSE 2331/5331

13. An Non-recursive Algorithm
CSE 2331/5331

14. Build Max-heap Operation
leaves from
n/2 to n
Build-Max-Heap( A, n )
n = length ( A );
for i =  n / 2  downto 1 do
Max-Heapify ( A, i, n )
Still in-place !
Time complexity:
Easy bound: 𝑂(𝑛 lg 𝑛)
Tight bound: Θ 𝑛
CSE 2331/5331

15. Heapsort Input array: A[1 .. n] Wish to be in-place sorting
First: build heap 14 16 1 4 8
A[ 1 .. n-1 ] 16 14 1 4 8
A[ 1 .. n-2 ]
A[ 1 .. n-1 ] 16 14 10 7 8 3 9 4 2 1 1 16 16 1 16
A[ 1 .. n ]
CSE 2331/5331

16. Heapsort (A, n) Time complexity: Built-Max-Heap (A, n)
for i=n downto 2
Exchange A[1] with A[i]
Max-Heapify (A, 1, i-1)
Time complexity:
O(n lg n)
CSE 2331/5331

17. Priority Queue Operations: Init( S ) Insert ( S, key ) Maximum ( S )
Extract-Max ( S )
Increase-key ( S, i, key )
A priority queue can be implemented using various
data structures, heap is one of them.
CSE 2331/5331

18. Using Linked List for Priority Queue
Operations:
Init( S ) O(1)
Insert ( S, key ) O(1)
Maximum ( S ) O( sizeof(S) )
Extract-Max ( S ) O( sizeof(S) )
Increase-key ( S, i, key ) O(1)
CSE 2331/5331

19. Time complexity: O( lg n )
Heap Implementations
Maximum ( S ) -- trivially O(1)
Extract-Max:
Time complexity: O( lg n ) 1 16 16 1
CSE 2331/5331

20. Heap Implementation -- cont.
Increase-key ( S, i, key ) 16 14 10 7 8 3 9 4 2 1 19 14 16 14 19 19
CSE 2331/5331

21. Increase-Key
Time complexity: O( lg n)
CSE 2331/5331

22. Increase-Key How about Decrease-key ( S, i, key )
Time complexity: O( lg n)
CSE 2331/5331

23. Heap Implementation -- cont.
Time complexity:
O ( lg n )
CSE 2331/5331

24. Priority Queue – Heap Implementation
Operations:
Insert ( S, key ) O(lg n)
Maximum ( S ) O(1)
Extract-Max ( S ) O(lg n)
Increase-key ( S, i, key ) O(lg n)
A different way to build heap:
Insertion-Build-Heap
CSE 2331/5331

25. Another Heap-sort
Time complexity: O(n lg n)
CSE 2331/5331

26. Example of Alg. Analysis
Analyze the running time of the following algorithm, where the priority P is implemented by a Max-heap.
CSE 2331/5331

27. Another Example
Analyze the running time of the following algorithm, where the priority P is implemented by a Max-heap
CSE 2331/5331

28. Summary Heap Priority queue Data structure Heapsort
Implemented by heap
CSE 2331/5331

29. Lower Bound for Sorting
Model
What types of operations are allowed
E.g: partial sum
Both addition and subtraction
Addition-only model
For sorting:
Comparison-based model
CSE 2331/5331

30. Decision Tree
> a i j no
> a s t
yes
> a k m
CSE 2331/5331

31. Example
For insertion sort with 3 elements
CSE 2331/5331

32. Decision Tree Not necessary same height Worst case complexity:
Longest root-leaf path
Each leaf:
A possible outcome
I.e., a permutation of input
Every possible outcome should be some leaf
#leaves ≥ n!
CSE 2331/5331

33. Lower Bound Any binary tree of height h has at most 2 ℎ leaves
A binary tree with m leaves is of height at least lg m
Worst case complexity for any algorithm sorting n elements is :
(lg (n!) ) = (n lg n)
(by Stirling approximation)
CSE 2331/5331

34. Non-comparison Based Sorting
Assume inputs are integers  [1, … k]
k = O (n)
Count-Sort (A, n) (simplified version)
Initialize array C[ 1,…k ] with C[ i ] = 0
for i = 1 to n do
C[ A[ i ] ] ++
output based on C
Time and space complexity
𝑂 𝑘 =𝑂 𝑛
CSE 2331/5331