1. 2IL50 Data Structures
Spring 2015
Lecture 3: Heaps

2. Solving recurrences
one more time …

3. Solving recurrences
Easiest: Master theorem caveat: not always applicable
Alternatively: Guess the solution and use the substitution method to prove that your guess it is correct.
How to guess:
expand the recursion
draw a recursion tree

4. Example Example(A) ► A is an array of length n n = A.length if n==1
then return A[1]
else begin
Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]
Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]
b = Example(B); c = Example(C)
for i = 1 to n
do for j = 1 to i
do A[i] = A[j]
return 43
end

5. Example Example(A) ► A is an array of length n n = A.length if n==1
Let T(n) be the worst case running time of Example on an array of length n.
Lines 1,2,3,4,11, and 12 take Θ(1) time.
Lines 5 and 6 take Θ(n) time.
Line 7 takes Θ(1) + 2 T( n/2 ) time.
Lines 8 until 10 take
time.
If n=1 lines 1,2,3 are executed,
else lines 1,2, and 4 until 12 are executed.
➨ T(n):
➨ use master theorem …
Example(A)
► A is an array of length n
n = A.length
if n==1
then return A[1]
else begin
Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]
Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]
b = Example(B); c = Example(C)
for i = 1 to n
do for j = 1 to i
do A[i] = A[j]
return 43
end
Θ(1) if n=1
2T(n/2) + Θ(n2) if n>1

6. The master theorem
Let a and b be constants, let f(n) be a function, and let T(n)
be defined on the nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n)
Then we have:
If f(n) = O(nlog a – ε) for some constant ε > 0, then T(n) = Θ(nlog a).
If f(n) = Θ(nlog a), then T(n) = Θ(nlog a log n)
If f(n) = Ω(nlog a + ε) for some constant ε > 0, and if af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently large n, then T(n) = Θ(f(n)) b b b b b

7. Quiz Recurrence T(n) = 4 T(n/2) + Θ(n3) T(n) = 4 T(n/2) + Θ(n)
T(n) = T(n/3) + T(2n/3) + n
T(n) = 9 T(n/3) + Θ(n2)
T(n) = √n T(√n) + n
Master theorem?
yes no
T(n) = Θ(n3)
T(n) = Θ(n2)
T(n) = Θ(log n)
T(n) = Θ(n log n)
T(n) = Θ(n2 log n)
T(n) = Θ(n log log n)

8. Tips
Analysis of recursive algorithms: find the recursion and solve with master theorem if possible
Analysis of loops: summations
Some standard recurrences and sums:
T(n) = 2T(n/2) + Θ(n) ➨
½ n(n+1) = Θ(n2)
Θ(n3)
T(n) = Θ(n log n)

9. Heaps

10. Event-driven simulation
Stores a set of events, processes first event (highest priority)
Supporting data structure:
insert event
find (and extract) event with highest priority
change the priority of an event

11. Priority queue
Max-priority queue abstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).
Operations Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x} Maximum(S): returns the element of S with the largest key Extract-Max(S): removes and returns the element of S with the largest key Increase-Key(S, x, k): give key[x] the value k
condition: k is larger than the current value of key[x]
Min-priority queue …

12. Implementing a priority queue
Insert
Maximum
Extract-Max
Increase-Key
sorted list
sorted array
Θ(n)
Θ(1)
Θ(1)
Θ(n)
Θ(n)
Θ(1)
Θ(n)
Θ(n)
Today
Insert
Maximum
Extract-Max
Increase-Key
heap
Θ(log n)
Θ(1)
Θ(log n)
Θ(log n)

13. Max-heap 35 19 30 21 8 35 19 30 11 12 8 5 17 2 35 19 30 12 8
Heap nearly complete binary tree, filled on all levels except possibly the lowest. (lowest level is filled from left to right)
Max-heap property: for every node i other than the root key[Parent(i)] ≥ key[i]

14. Properties of a max-heap
Lemma The largest element in a max-heap is stored at the root.
Proof:

15. Properties of a max-heap
Lemma The largest element in a max-heap is stored at the root.
Proof: x root
y arbitrary node
z1, z2, …, zk nodes on path between x and y
max-heap property ➨ key[x] ≥ key[z1] ≥ … ≥ key[zk] ≥ key[y]
➨ the largest element is stored at x ■ x z1 z2 y

16. Implementing a heap with an array35 30 19 30 8 12 11 17 2 5
array A[1 … A.length] 35 30 19 30 8 12 11 17 2 5 1 2 3 4
heap-size[A]
A.length = length of array A
heap-size[A] =number of elements in the heap

17. Implementing a heap with an array
level 0
level 1
level 2, position 3
array A[1 … A.length] 1 2 3 4
heap-size[A]
kth node on level j is stored at position
left child of node at position i = Left(i) =
right child of node at position i = Right(i) =
parent of node at position i = Parent(i) =
A[2j+k-1] 2i
2i + 1
i / 2

18. Priority queue
Max-priority queue abstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).
Operations Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x} Maximum(S): returns the element of S with the largest key Extract-Max(S): removes and returns the element of S with the largest key Increase-Key(S, x, k): give key[x] the value k
condition: k is larger than the current value of key[x]

19. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key. 35 30 19 8 12 11 17 2 5 1 3
heap-size[A] 4

20. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Maximum(A)
if heap-size[A] < 1
then error
else return A[1]
running time: O(1) 35 30 19 8 12 11 17 2 5 1 3
heap-size[A] 4

21. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key. 35 30 19 8 12 11 17 2 5 1 3
heap-size[A] 4

22. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key. 30 35 30 19 30 17 8 12 11 17 2 5

23. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Heap-Extract-Max(A)
if heap-size[A] < 1
then error
max = A[1]
A[1] = A [heap-size[A]]
heap-size[A] = heap-size[A]–1
Max-Heapify(A,1)
return max 35 5 30 19 30 8 12 11 17 2 5
restore heap property

24. Max-Heapify Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has the max-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps

25. Max-Heapify Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has the max-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps
Max-Heapify(A,1)
exchange A[1] with largest child
Max-Heapify(A,2)
exchange A[2] with largest child
Max-Heapify(A,5)
A[5] larger than its children ➨ done. 10 40 10 40 35 19 30 10 35 12 11 17 2 5

26. Max-Heapify Max-Heapify(A, i)
► ensures that the heap whose root is stored at position i has the max-heap property
► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps
if Left(i) ≤ heap-size[A] and A[Left(i)] > A[i]
then largest = Left(i)
else largest = i
if Right(i) ≤ heap-size[A] and A[Right(i)] > A[largest]
then largest = Right(i)
if largest ≠ i
then exchange A[i ] and A[largest]
Max-Heapify(A, largest)
running time?
O(height of the subtree rooted at i) = O(log n)

27. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Insert (A, key)
heap-size[A] = heap-size[A] +1
A[heap-size[A]] = - infinity
Increase-Key(A, heap-size[A], key)

28. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.

29. Implementing a max-priority queue
Set S is stored as a heap in an array A.
Operations: Maximum, Extract-Max, Insert, Increase-Key.
Increase-Key(A, i, key)
if key < A[i]
then error
A[i] = key
while i>1 and A[parent(i)] < A[i]
do exchange A[parent(i)] and A[i]
i = parent(i)
running time: O(log n) 35 30 19 30 8 12 42 11 17 2 5

30. Building a heap Build-Max-Heap(A) ► Input: array A[1..n] of numbers
► Output: array A[1..n] with the same numbers, but rearranged, such that the max-heap property holds
heap-size = A.length
for i = A.length downto 1
do Max-Heapify(A,i)
P(k): nodes k, k+1, …, n are each the root of a max-heap
Loop Invariant P(k+1) holds before line 3 is executed with i=k, P(k) holds afterwards
starting at A.length / 2 is sufficient

31. Building a heap Build-Max-Heap(A) heap-size = A.length
for i = A.length downto 1
do Max-Heapify(A,i)
∑i O(1 + height of i)
= ∑0 ≤ h ≤ log n (# nodes of height h) ∙ O(1+h)
≤ ∑0 ≤ h ≤ log n (n / 2h+1) ∙ O(1+h)
= O(n) ∙ ∑0 ≤ h ≤ log n h / 2h+1
= O(n)
O(height of node i)
height of node i # edges longest simple downward path from i to a leaf.
h=0
h=1
h=2

32. The sorting problem Input: a sequence of n numbers ‹a1, a2, …, an›
Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›
Important properties of sorting algorithms:
running time: how fast is the algorithm in the worst case
in place: only a constant number of input elements are ever stored outside the input array

33. Sorting with a heap: Heapsort
HeapSort(A)
Build-Max-Heap(A)
for i = A.length downto 2
do exchange A[1] and A[i ]
heap-size[A] = heap-size[A] – 1
Max-Heapify(A,1)
P(k): A[k+1 .. n] is sorted and contains the n-k largest elements, A[1..k] is a max-heap on the remaining elements
Loop Invariant P(k) holds before lines 3-5 are executed with i=k, P(k-1) holds afterwards

34. worst case running time
Sorting algorithms
worst case running time
in place
InsertionSort
MergeSort
HeapSort
Θ(n2)
yes
Θ(n log n) no
Θ(n log n)
yes

35. Announcements
Wednesday 3+4
Reading and writing mathematical proofs