HEAPS Amihood Amir Bar Ilan University 2014

Sorting Bubblesort: Until no exchanges: For i=1 to n-1 if A[i]>A[i+1] then exchange their values end Time : O(n 2 )

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 4277

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 3577

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 1277

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping No need to swap

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 5101

"Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping Largest value correctly placed

Sorting Selection sort: For i=1 to n do For j=i+1 to n do If A[i]>A[j] then exchange them end Time: = O(n 2 )

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping Swap 4277

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping Swap 3542

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping Swap

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping No need to swap

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping Swap 512

Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping Smallest value correctly placed

Sorting Insertion sort: For i=1 to n do Insert A[i+1] into appropriate (sorted) position in A[1],…,A[i] end Time: = O(n 2 )

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout

Complexity The Time of all algorithms we saw: O(n 2 ) Can we do better?

Merge Sort Based on the Merging operation. Given two sorted arrays: A[1],…,A[n] and B[1],…,B[m]. merge them into one sorted array: C[1],…,C[n+m]

Merging Pa, Pb, Pc <- 1 While (Pa < n+1 and Pb < m+1) If A[Pa] ≤ B[Pb] thenC[Pc] <- A[Pa] Pa <- Pa+1 Pc <- Pc+1 elseC[Pc] <- B[Pb] Pb <- Pb+1 Pc <- Pc+1 end

Merging If Pa=n+1 and Pb < m+1 then for i=Pb to m do C[Pc] <- B[i] Pc <- Pc +1 end If Pb=n+1 and Pa < m+1 then for i=Pa to n do C[Pc] <- A[i] Pc <- Pc +1 end end Algorithm

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C:

Merging A:B: C: Time: O(n+m) Linear !!!

Merge Sort A recursive sorting algorithm: Mergesort(A) If n=1 then Return(A) else Split A[1],…,A[n] to two length n/2 arrays: A[1],…,A[n/2] and A[n/2+1],…,A[n] Mergesort(A[1],…,A[n/2]) Mergesort(A[n/2+1],…,A[n]) Merge(A[1],…,A[n/2], A[n/2+1],…,A[n], B) A <- B Return(A)

Merge Sort Example

Merge Sort Example

Merge Sort Example

Merge Sort Example

Merge Sort Example

Merge Sort Example Merge

Merge Sort Example Merge

Merge Sort Example Merge

Merge Sort Example Merge

Merge Sort Example

Merge Sort Time The recurrence: T M (n)=2T M (n/2)+n T M (1)=1 Closed Form of T M (n) : O(n log n)

A Data Structures Algorithm Sort array A using AVL Trees: For i=1 to n do: Insert A[i] into AVL tree For i=1 to n do: Extract smallest element from AVL tree and put in A[i]

Sorting Algorithm using AVL Can we extract the smallest element from AVL tree? -- Yes. Go all the way to the left. Time: O(log n) Therefore: Sorting Algorithm Time: O(n log n)

Priority Queue What property of the AVL Tree did we use? Only that we can find the minimum fast. A Priority Queue is a data structure that supports the following operations: INSERT (Q,k) EXTRACTMIN(Q,m)

Priority Queue Implementation Heap: A binary tree where every vertex has a key. For every vertex v, key(v) ≤ key(w), where w is a child of v. Difference between heap and binary search tree: HeapBinary search tree

Heap Additional Heap Property: All levels of the heap are full, except possibly the last level, which is left-filled. Example:

Heap Properties The keys on every path from the root to a leaf are nondecreasing. Example:

Heap Properties The height of a heap with n elements is O(log n). Example: h < log n ≤ h+1

Heap Insertion Insert

Heap Insertion Insert

Heap Insertion Insert OK

Heap Deletion Delete Min Now fix heap

Heap Deletion Delete Min

Heap Deletion Delete Min OK

Heap Operations Time O(log n) Conclude: Heap of n elements can be constructed in time: O(n log n) But: We can do a lot better!!!

Build Heap Put all elements in a tree where all levels are full and the last level is left-filled. Construct heaps from leaves up

Time Analysis For trees in level h : 1 For trees in level h-1 : 2 For trees in level h-i :i For trees in level 0 :log n

Time Analysis Number of trees in level h:n/2 Number of trees in level h-1:n/2 2 … Number of trees in level h-i:n/2 i+1 … Number of trees in level 0 :1

Total Time Calculate it: Note that n/2 trees do constant time work, and another quarter need to go to two levels. Only one tree needs to fix log n levels. Formally: Need to calculate this

Total Time We know: for x<1. Differentiate both sides: Multiply both sides by x:

Total Time In our case: x = 1/2. So total time: O(n).

In our case: Look Ma No Pointers Our tree is almost complete. All levels are full and the last one is left-filled. So write down all keys sequentially from top to bottom and left to right.

Getting rid of Pointers Example: 3,4,6,21,10,7,8,22,28,23,35,25,

Calculate Location node i in level j = location 2 j -1+i in array. Location i in array = node in level

Node j in row i is the 2 i -1+j’s element. Its left son is the 2 i (j-1)+1’s element. But 2 i (j-1)+1 = 2 i+1 +2(j-1) = 2 i+1 +2j-2 = 2(2 i -1+j). Location of children row i j

Conclude: Location of left son of i is 2i. Location of right son of i is 2i+1. Location of father of i is: i/2,if i is even (i-1)/2,if i is odd Location of Children

How? Given array A of n elements: Construct a heap H A in-place. For i=n downto 1 do EXTRACTMIN (H A,v) Put v in location A[i] Endfor Time: O(n) for heap construction and O(nlog n) for loop. Heap Sort

Note: Array A is now sorted in non-increasing order. What do we do if we want to sort in non- decreasing order? Options: Reverse order of A in linear time in- place. Use MAX rather than MIN priority Queue. Heap Sort