Heaps Chapter 6 in CLRS

Motivation Dijkstra’s algorithm for single source shortest path Prim’s algorithm for minimum spanning trees

Motivation Want to find the shortest route from New York to San Francisco Model the road-map with a graph

A Graph G=(V,E) V is a set of vertices E is a set of edges (pairs of vertices)

Model driving distances by weights on the edges V is a set of vertices E is a set of edges (pairs of vertices)

Source and destination V is a set of vertices E is a set of edges (pairs of vertices)

Dijkstra’s algorithm Assume all weights are non-negative Finds the shortest path from some fixed vertex s to every other vertex

s s1s s3s3 s2s2 s4s Example

Maintain an upper bound d(v) on the shortest path to v 0 ∞ ∞ ∞ ∞ s s1s s3s3 s2s2 s4s Example

s s1s s3s3 s2s2 s4s Maintain an upper bound d(v) on the shortest path to v 0 ∞ ∞ ∞ ∞ A node is either scanned ( in S) or labeled ( in Q) Initially S =  and Q = V

s s1s s3s3 s2s2 s4s Pick a vertex v in Q with minimum d(v) and add it to S 0 ∞ ∞ ∞ ∞ Initially S =  and Q = V

s s1s s3s3 s2s2 s4s Pick a vertex v in Q with minimum d(v) and add it to S 0 ∞ ∞ ∞ ∞ Initially S =  and Q = V v w 15 For every edge (v,w) where w in Q relax(v,w)

s s1s s3s3 s2s2 s4s Relax(v,w) If d(v) + w(v,w) < d(w) then d(w) ← d(v) + w(v,w) π(w) ← v 0 ∞ ∞ ∞ ∞ v w For every edge (v,w) where w in Q relax(v,w) 15

s s1s s3s3 s2s2 s4s ∞ ∞ ∞ ∞ S = {s} Relax(s,s 4 )

s s1s s3s3 s2s2 s4s ∞ ∞ ∞ Relax(s,s 4 ) S = {s}

s s1s s3s3 s2s2 s4s ∞ ∞ ∞ Relax(s,s 4 ) Relax(s,s 3 ) S = {s}

s s1s s3s3 s2s2 s4s ∞ ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) S = {s}

s s1s s3s3 s2s2 s4s ∞ ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) Relax(s,s 1 ) S = {s}

s s1s s3s3 s2s2 s4s ∞ 10 Relax(s,s 4 ) Relax(s,s 3 ) Relax(s,s 1 ) S = {s}

s s1s s3s3 s2s2 s4s ∞ 10 S = {s} Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s ∞ 10 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s ∞ 10 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 )

s s1s s3s3 s2s2 s4s ∞ 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 )

s s1s s3s3 s2s2 s4s ∞ 9 S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 ) Relax(s 1,s 2 )

s s1s s3s3 s2s2 s4s S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 1,s 3 ) Relax(s 1,s 2 )

s s1s s3s3 s2s2 s4s S = {s,s 1 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 2,s 3 )

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S Relax(s 2,s 3 )

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2,s 3 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2,s 3,s 4 } Pick a vertex v in Q with minimum d(v) and add it to S

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2,s 3,s 4 } When Q =  then the d() values are the distances from s The π function gives the shortest path tree

s s1s s3s3 s2s2 s4s S = {s,s 1,s 2,s 3,s 4 } When Q =  then the d() values are the distances from s The π function gives the shortest path tree

Implementation of Dijkstra’s algorithm We need to find efficiently the vertex with minimum d() in Q We need to update d() values of vertices in Q

Required ADT Maintain items with keys subject to Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ)

Required ADT Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q)

Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q) How many times we do these operations ? n = |V| n n m = |E|

Do we know an algorithm for this ADT ? Insert(x,Q) min(Q) Deletemin(Q) Decrease-key(x,Q,Δ): Can simulate by Delete(x,Q), insert(x-Δ,Q) Yes! Use balanced binary search trees (RB-trees)

Heap Heap is  A complete binary tree  The items at the children of v are greater than the one at v Heap-ordered tree

Representing a heap with an array  Get the elements from top to bottom, from left to right Array Representation Q

Array Representation Left(i): 2i+1 Right(i): 2i+2 Parent(i):  (i-1)/2  Q

Find the minimum Return Q[0] Q

Delete the minimum Q

Delete the minimum Q

Delete the minimum Q

Delete the minimum Q

Delete the minimum Q

Delete the minimum Q Q[0] ← Q[size(Q)-1] Heapify-down(Q,0) size(Q) ← size(Q)-1

Heapify-down(Q,i) l ← left(i) r ← right(i) smallest ← i if l < size(Q) and Q[l] < Q[smallest] then smallest ← l if r < size(Q) and Q[r] < Q[smallest] then smallest ← r if smallest > i then Q[i] ↔ Q[smallest] Heapify-down(Q, smallest)

Inserting an item Q Insert(15,Q)

Inserting an item Q Insert(15,Q) Q[size(Q)] ← Heapify-up(Q,size(Q)) size(Q) ← size(Q) + 1

Inserting an item Q 29

Inserting an item Q 29

Inserting an item Q 29

Q Heapify-up Heapify-up(Q, i) while i > 0 and Q[i] < Q[parent(i)] do Q[i] ↔ Q[parent(i)] i ← parent(i)

Other operations Decrease-key(x,Q,Δ) Delete(x,Q) Can implement them easily using heapify

Heapsort (Williams, Floyd, 1964) Put the elements in an array Make the array into a heap Do a deletemin and put the deleted element at the last position of the array

Put the elements in the heap Q

Q Make the elements into a heap

Q Heapify-down(Q,4)

Q Heapify-down(Q,4)

Q Heapify-down(Q,3)

Q Heapify-down(Q,3)

Q Heapify-down(Q,2)

Q Heapify-down(Q,2)

Q Heapify-down(Q,1)

Q Heapify-down(Q,1)

Q Heapify-down(Q,1)

Q Heapify-down(Q,0)

Q Heapify-down(Q,0)

Q Heapify-down(Q,0)

Q Heapify-down(Q,0)

We have at most n/2 nodes heapified at height 1 How much time does it take to build the heap this way ? We have at most n/4 nodes heapified at height 2 We have at most n/8 nodes heapified at height 3 Total time =

We have at most n/2 nodes heapified at height 1 How much time does it take to build the heap this way ? We have at most n/4 nodes heapified at height 2 We have at most n/8 nodes heapified at height 3 Total time =

Summary We can build the heap in linear time We still have to deletemin the elements one by one in order to sort that will take O(nlog(n))

An example Given an array of length n, find the k smallest numbers. 77