Algorithms Sorting-part2

Heap Sort

Heaps and Priority Queues 2 6 5 7 9

Queues

Queue First-in, First-out (FIFO) structure Operations Sample use enqueue: insert element at rear dequeue: remove & return front element front: return front element isEmpty: check if the queue has no elements size: return number of elements in the queue Sample use handling requests and reservations

wrapped-around configuration Array-based Queue Use an array of size N in a circular fashion Two variables keep track of the front and rear f index of the front element r index immediately past the rear element Array location r is kept empty normal configuration Q 1 2 r f wrapped-around configuration Q 1 2 f r

Queue Operations We use the modulo operator (remainder of division) Algorithm size() return (N - f + r) mod N Algorithm isEmpty() return (f = r) We use the modulo operator (remainder of division) Q 1 2 r f Q 1 2 f r

Queue Operations (cont.) Algorithm enqueue(o) if size() = N  1 then throw FullQueueException else Q[r]  o r  (r + 1) mod N Operation enqueue throws an exception if the array is full This exception is implementation-dependent Q 1 2 r f Q 1 2 f r

Queue Operations (cont.) Algorithm dequeue() if isEmpty() then throw EmptyQueueException else o  Q[f] f  (f + 1) mod N return o Operation dequeue throws an exception if the queue is empty This exception is specified in the queue ADT Q 1 2 r f Q 1 2 f r

Linked List Implementation public class NodeQueue implements Queue { private Node front; private Node rear; private int size; … } front rear null

Enqueue front rear null null

Dequeue front rear null return this object

Time Complexity Analysis enqueue() : O(1) dequeue() : O(1) isEmpty() : O(1) size() : O(1) front(): O(1)

Trees Make Money Fast! Stock Fraud Ponzi Scheme Bank Robbery

Definitions A tree is an abstract data type one entry point, the root root node internal nodes A tree is an abstract data type one entry point, the root Each node is either a leaf or an internal node An internal node has 1 or more children, nodes that can be reached directly from that internal node. The internal node is said to be the parent of its child nodes leaf nodes

Tree Terminology subtree Root: node without parent (A) Internal node: node with at least one child (A, B, C, F) External node (a.k.a. leaf ): node without children (E, I, J, K, G, H, D) Ancestors of a node: parent, grandparent, grand-grandparent, etc. Depth of a node: number of ancestors Height of a tree: maximum depth of any node (3) Descendant of a node: child, grandchild, grand-grandchild, etc. Subtree: tree consisting of a node and its descendants A B D C G H E F I J K subtree

Binary Tree A binary tree is a tree with the following properties: Each internal node has two children left child and right child The children of a node are an ordered pair Alternative recursive definition: a binary tree is either a tree consisting of a single node, or a tree whose root has an ordered pair of children, each of which is a binary tree Applications: arithmetic expressions decision processes searching A B C D E F G H I

Arithmetic Expression Tree Binary tree associated with an arithmetic expression internal nodes: operators external nodes: operands Example: arithmetic expression tree for the expression (2  (a - 1) + (3  b)) +  - 2 a 1 3 b

Decision Tree Binary tree associated with a decision process internal nodes: questions with yes/no answer external nodes: decisions Example: dining decision Want a fast meal? Yes No How about coffee? On expense account? Yes No Yes No Starbucks Spike’s Al Forno Café Paragon

Inorder Traversal In an inorder traversal a node is visited after its left subtree and before its right subtree Application: draw a binary tree x(v) = inorder rank of v y(v) = depth of v Algorithm inOrder(v) if isInternal (v) inOrder (leftChild (v)) visit(v) inOrder (rightChild (v)) 6 2 8 1 4 7 9 3 5

Data Structure for Binary Trees A node is represented by an object storing Element Parent node Left child node Right child node  B   A D B A D     C E C E

Priority Queues Sell 100 IBM $122 300 $120 Buy 500 $119 400 $118

Priority queue A stack is first in, last out A queue is first in, first out A priority queue is least-first-out The “smallest” element is the first one removed (You could also define a largest-first-out priority queue) If there are several “smallest” elements, the implementer must decide which to remove first Remove any “smallest” element (don’t care which) Remove the first one added

The Priority Queue ADT A prioriy queue P supports the following methods: -size(): Return the number of elements in P -isEmpty(): Test whether P is empty -insertItem(k,e): Insert a new element e with key k into P -minElement(): Return (but don’t remove) an element of P with smallest key; an error occurs if P is empty. -minKey(): Return the smallest key in P; an error occurs if P is empty -removeMin(): Remove from P and return an element with the smallest key; an error condidtion occurs if P is empty.

What is a heap A heap is a binary tree storing keys at its internal nodes and satisfying the following properties: Heap-Order: for every internal node v other than the root, key(v)  key(parent(v)) Complete Binary Tree: let h be the height of the heap for i = 0, … , h - 1, there are 2i nodes of depth i at depth h - 1, the internal nodes are to the left of the external nodes The last node of a heap is the rightmost internal node of depth h - 1 2 5 6 9 7 last node

Binary Heap: Definition Almost complete binary tree. filled on all levels, except last, where filled from left to right Min-heap ordered. every child greater than (or equal to) parent 06 14 45 78 18 47 53 83 91 81 77 84 99 64

Binary Heap: Properties Min element is in root. Heap with N elements has height = log2 N. 06 N = 14 Height = 3 14 45 78 18 47 53 83 91 81 77 84 99 64

Heaps and Priority Queues We can use a heap to implement a priority queue We store a (key, element) item at each internal node We keep track of the position of the last node For simplicity, we show only the keys in the pictures (2, Ahmed) (5, Hassan) (6, Mona) (9, Sara) (7, Peter)

Insertion into a Heap Method insertItem of the priority queue corresponds to the insertion of a key k to the heap The insertion algorithm consists of three steps Find the insertion node z (the new last node) Store k at z and expand z into an internal node Restore the heap-order property (discussed next) 2 6 5 7 9 z insertion node 2 5 6 z 9 7 1

Upheap After the insertion of a new key k, the heap-order property may be violated Algorithm upheap restores the heap-order property by swapping k along an upward path from the insertion node Upheap terminates when the key k reaches the root or a node whose parent has a key smaller than or equal to k Since a heap has height O(log n), upheap runs in O(log n) time 2 1 5 1 5 2 z z 9 7 6 9 7 6

Binary Heap: Properties Min element is in root. Heap with N elements has height = log2 N. 06 N = 14 Height = 3 14 45 78 18 47 53 83 91 81 77 84 99 64

Binary Heap: Insertion Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered. Peter principle: nodes rise to level of incompetence 06 14 45 78 18 47 53 42 next free slot 83 91 81 77 84 99 64

Binary Heap: Insertion Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered. Peter principle: nodes rise to level of incompetence swap with parent 06 14 45 78 18 47 53 42 42 83 91 81 77 84 99 64

Binary Heap: Insertion Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered. Peter principle: nodes rise to level of incompetence swap with parent 06 14 45 78 18 47 42 42 83 91 81 77 84 99 64 53

Binary Heap: Insertion Insert element x into heap. Insert into next available slot. Bubble up until it's heap ordered. Peter principle: nodes rise to level of incompetence O(log N) operations. stop: heap ordered 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

Binary Heap: Decrease Key Decrease key of element x to k. Bubble up until it's heap ordered. O(log N) operations. 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

Removal from a Heap Method removeMin of the priority queue corresponds to the removal of the root key from the heap The removal algorithm consists of three steps Replace the root key with the key of the last node w Compress w and its children into a leaf Restore the heap-order property (discussed next) 2 6 5 7 9 w last node 7 5 6 w 9

Binary Heap: Delete Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. power struggle principle: better subordinate is promoted 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

Binary Heap: Delete Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. power struggle principle: better subordinate is promoted 53 14 42 78 18 47 45 83 91 81 77 84 99 64 06

Binary Heap: Delete Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. power struggle principle: better subordinate is promoted exchange with left child 53 14 42 78 18 47 45 83 91 81 77 84 99 64

Binary Heap: Delete Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. power struggle principle: better subordinate is promoted exchange with right child 14 53 42 78 18 47 45 83 91 81 77 84 99 64

Binary Heap: Delete Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. power struggle principle: better subordinate is promoted O(log N) operations. stop: heap ordered 14 18 42 78 53 47 45 83 91 81 77 84 99 64

Updating the Last Node The insertion node can be found by traversing a path of O(log n) nodes Go up until a left child or the root is reached If a left child is reached, go to the right child Go down left until a leaf is reached Similar algorithm for updating the last node after a removal

Using a heap to sort Using a heap-based priority queue, sorting can be done in O( n log n ) time Insert all n elements in the priority queue Repeatedly remove the elements from the priority queue (they will come out sorted) 2n operations on the priority queue, each taking O( log n ) time => O( n log n ) Sorting can be done within the array by treating the array as a heap

Heap sort example 6 10 5 12 3 9 20 2 15 8 18 6 10 5 9 20 12 3 2 15 8 18

Phase 1: build the heap for i  1 to n-1 do insert element s[i] into the heap consisting of the elements s[0]…s[i-1] - Once the heap is built, s[0] will contain the maximum element

Phase 1: build heap 6 10 5 9 20 12 3 2 15 8 18

Phase 1: build heap 10 6 5 9 20 12 3 2 15 8 18

Phase 1: build heap 10 6 5 9 20 12 3 2 15 8 18

Phase 1: build heap 12 10 5 9 20 6 3 2 15 8 18

Phase 1: build heap 12 10 5 9 20 6 3 2 15 8 18

Phase 1: build heap 12 10 9 5 20 6 3 2 15 8 18

Phase 1: build heap 20 10 12 5 9 6 3 2 15 8 18

Phase 1: build heap 20 10 12 5 9 6 3 2 15 8 18

Phase 1: build heap 20 15 12 5 9 10 3 2 6 8 18

Phase 1: build heap 20 15 12 5 9 10 8 2 6 3 18

Phase 1: build heap 20 Maximum element 18 12 5 9 10 15 2 6 3 8

Phase 2: repeatedly select max for i  n-1 down to 1 do swap s[0] and s[i] “demote” s[0] to its proper place in the heap consisting of the elements s[0]...s[i-1]

Phase 2: select max 20 18 12 5 9 10 15 2 6 3 8

Phase 2: select max 8 18 12 5 9 10 15 2 6 3 20

Phase 2: select max 18 15 12 5 9 10 8 2 6 3 20

Phase 2: select max 18 15 12 5 9 10 8 2 6 3 20

Phase 2: select max 3 15 12 5 9 10 8 2 6 18 20

Phase 2: select max 15 10 12 5 9 6 8 2 3 18 20

Phase 2: select max 15 10 12 5 9 6 8 2 3 18 20

Phase 2: select max 3 10 12 5 9 6 8 2 15 18 20

Phase 2: select max 12 10 9 5 3 6 8 2 15 18 20

Phase 2: select max 12 10 9 5 3 6 8 2 15 18 20

Phase 2: select max 2 10 9 5 3 6 8 12 15 18 20

Phase 2: select max 10 8 9 5 3 6 2 12 15 18 20

Phase 2: select max 3 8 9 5 10 6 2 12 15 18 20

Phase 2: select max 9 8 5 3 10 6 2 12 15 18 20

Phase 2: select max 3 8 5 9 10 6 2 12 15 18 20

Phase 2: select max 8 6 5 9 10 3 2 12 15 18 20

Phase 2: select max 2 6 5 9 10 3 8 12 15 18 20

Phase 2: select max 6 3 5 9 10 2 8 12 15 18 20

Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

Phase 2: select max 5 3 2 9 10 6 8 12 15 18 20

Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

Phase 2: select max 3 2 5 9 10 6 8 12 15 18 20

Phase 2: select max 2 3 5 9 10 6 8 12 15 18 20

Heap sort completed 2 3 5 6 8 9 10 12 15 18 20 2 3 5 9 10 6 8 12 15 18 20

In-place Heapification 8.4.2 Now, consider this unsorted array: This array represents the following complete tree: This is neither a min-heap, max-heap, or binary search tree

In-place Heapification 8.4.3 Let’s work bottom-up: each leaf node is a max heap on its own

In-place Heapification 8.4.3 Starting at the back, we note that all leaf nodes are trivial heaps Also, the subtree with 87 as the root is a max-heap

In-place Heapification 8.4.3 The subtree with 23 is not a max-heap, but swapping it with 55 creates a max-heap This process is termed percolating down

In-place Heapification 8.4.3 The subtree with 3 as the root is not max-heap, but we can swap 3 and the maximum of its children: 86

In-place Heapification 8.4.3 Starting with the next higher level, the subtree with root 48 can be turned into a max-heap by swapping 48 and 99

In-place Heapification 8.4.3 Similarly, swapping 61 and 95 creates a max-heap of the next subtree

In-place Heapification 8.4.3 As does swapping 35 and 92

In-place Heapification 8.4.3 The subtree with root 24 may be converted into a max-heap by first swapping 24 and 86 and then swapping 24 and 28

In-place Heapification 8.4.3 The right-most subtree of the next higher level may be turned into a max-heap by swapping 77 and 99

In-place Heapification 8.4.3 However, to turn the next subtree into a max-heap requires that 13 be percolated down to a leaf node

In-place Heapification 8.4.3 The root need only be percolated down by two levels

In-place Heapification 8.4.3 The final product is a max-heap

Black Board Example Sort the following 12 entries using heap sort 8.4.5 Sort the following 12 entries using heap sort 34, 15, 65, 59, 79, 42, 40, 80, 50, 61, 23, 46

Heap sort time complexity for i  1 to n-1 do insert element s[i] into the heap consisting of the elements s[0]…s[i-1] for i  n-1 down to 1 do swap s[0] and s[i] “demote” s[0] to its proper place in the heap consisting of the elements s[0]...s[i-1] O( n log n ) O( log n ) operations O( n log n )

About heap sort Build heap phase can be improved to O( n ) if array is rearranged “bottom-up” Overall complexity still O( n log n ) because of second phase O( n log n ) time complexity is guaranteed Note that heap sort is just a more clever version of selection sort since a maximum is repeatedly selected and placed in its proper position

Heapify – builds the heap heapify(Item[]s, int p, int q) { while(p < q) { int c=2*p + 1; // left child if(c > q) break; if(c+1 <= q && s[c] < s[c+1]) ++c; if(s[p] >= s[c]) break; swap(s[p], s[c]); p = c; } }

Heapsort heapsort(Item[]s) { n = s.length; for(j=(n-1)/2, k=n-1; j>=0; --j) heapify(s, j, k); for(j=n-1; j>0; --j) { swap(s[0], s[j]); heapify(s, 0, j-1); } }

Problems Demonstrate, step by step, the operation of Build-Heap on the array A=[5, 3, 17, 10, 84, 19, 6, 22, 9]