Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu

CS 477/6772 The Heap Data Structure Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (heap) property: for any node x Parent(x) ≥ x Heap It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 2

CS 477/6773 Array Representation of Heaps A heap can be stored as an array A. –Root of tree is A[1] –Left child of A[i] = A[2i] –Right child of A[i] = A[2i + 1] –Parent of A[i] = A[  i/2  ] –Heapsize[A] ≤ length[A] The elements in the subarray A[(  n/2  +1).. n] are leaves The root is the maximum element of the heap A heap is a binary tree that is filled in order

CS 477/6774 Maintaining the Heap Property Suppose a node is smaller than a child –Left and Right subtrees of i are max-heaps Invariant: –the heap condition is violated only at that node To eliminate the violation: –Exchange with larger child –Move down the tree –Continue until node is not smaller than children

CS 477/6775 Building a Heap Convert an array A[1 … n] into a max-heap ( n = length[A] ) The elements in the subarray A[(  n/2  +1).. n] are leaves Apply MAX-HEAPIFY on elements between 1 and  n/2  A:

CS 477/6776 Heapsort Goal: –Sort an array using heap representations Idea: –Build a max-heap from the array –Swap the root (the maximum element) with the last element in the array –“Discard” this last node by decreasing the heap size –Call MAX-HEAPIFY on the new root –Repeat this process until only one node remains

CS 477/6777 Alg: HEAPSORT(A) 1. BUILD-MAX-HEAP (A) 2. for i ← length[A] downto 2 3. do exchange A[1] ↔ A[i] 4. MAX-HEAPIFY (A, 1, i - 1) Running time: O(nlgn) O(n) O(lgn) n-1 times

CS 477/6778 Example:A=[7, 4, 3, 1, 2] MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2) MAX-HEAPIFY(A, 1, 1)

CS 477/6779 HEAP-MAXIMUM Goal: –Return the largest element of the heap Alg: HEAP-MAXIMUM (A) 1.return A[1] Running time: O(1) Heap A: Heap-Maximum(A) returns 7

CS 477/67710 HEAP-EXTRACT-MAX Goal: –Extract the largest element of the heap (i.e., return the max value and also remove that element from the heap Idea: –Exchange the root element with the last –Decrease the size of the heap by 1 element –Call MAX-HEAPIFY on the new root, on a heap of size n-1 Heap A: Root is the largest element

CS 477/67711 HEAP-EXTRACT-MAX Alg: HEAP-EXTRACT-MAX (A, n) 1. if n < 1 2. then error “heap underflow” 3. max ← A[1] 4. A[1] ← A[n] 5. MAX-HEAPIFY (A, 1, n-1) remakes heap 6. return max Running time: O(lgn)

CS 477/67712 Example: HEAP-EXTRACT-MAX max = Heap size decreased with Call MAX-HEAPIFY (A, 1, n-1)

CS 477/67713 HEAP-INCREASE-KEY Goal: –Increases the key of an element i in the heap Idea: –Increment the key of A[i] to its new value –If the max-heap property does not hold anymore: traverse a path toward the root to find the proper place for the newly increased key i Key [i] ← 15

CS 477/67714 HEAP-INCREASE-KEY Alg: HEAP-INCREASE-KEY (A, i, key) 1. if key < A[i] 2. then error “new key is smaller than current key” 3. A[i] ← key 4. while i > 1 and A[PARENT(i)] < A[i] 5. do exchange A[i] ↔ A[PARENT(i)] 6. i ← PARENT(i) Running time: O(lgn) i Key [i] ← 15

CS 477/67715 Example: HEAP-INCREASE-KEY i i Key [ i ] ← i i

CS 477/ - MAX-HEAP-INSERT Goal: –Inserts a new element into a max- heap Idea: –Expand the max-heap with a new element whose key is -  –Calls HEAP-INCREASE-KEY to set the key of the new node to its correct value and maintain the max-heap property

CS 477/67717 MAX-HEAP-INSERT Alg: MAX-HEAP-INSERT (A, key, n) 1. heap-size[A] ← n A[n + 1] ← -  3. HEAP-INCREASE-KEY (A, n + 1, key) Running time: O(lgn) --

CS 477/67718 Example: MAX-HEAP-INSERT -- Insert value 15: - Start by inserting -  Increase the key to 15 Call HEAP-INCREASE-KEY on A[11] = The restored heap containing the newly added element

CS 477/67719 Summary We can perform the following operations on heaps: –MAX-HEAPIFY O(lgn) –BUILD-MAX-HEAP O(n) –HEAP-SORT O(nlgn) –MAX-HEAP-INSERT O(lgn) –HEAP-EXTRACT-MAX O(lgn) –HEAP-INCREASE-KEY O(lgn) –HEAP-MAXIMUM O(1)

CS 477/67720 The Search Problem Find items with keys matching a given search key Applications: Keeping track of customer account information at a bank –Search through records to check balances and perform transactions Keep track of reservations on flights –Search to find empty seats, cancel/modify reservations Search engine –Looks for all documents containing a given word

CS 477/67721 Symbol Tables (Dictionaries) Dictionary = data structure that supports two basic operations: insert a new item and return an item with a given key Queries: return information about the set –Search (S, k) –Minimum (S), Maximum (S) –Successor (S, x), Predecessor (S, x) Modifying operations: change the set –Insert (S, k) –Delete (S, k)

CS 477/67722 Implementations of Symbol Tables Key-indexed-array –Key values are distinct, small numbers –Store the items in an array, indexed by keys Ordered/unordered arrays Ordered/unordered linked lists InsertSearch ordered array ordered list unordered array unordered list N N N N 1 1 N N key-indexed array 1 1

CS 477/67723 Binary Search Trees Support many dynamic set operations –SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT Running time of basic operations on binary search trees –On average:  (lgn) The expected height of the tree is lgn –In the worst case:  (n) The tree is a linear chain of n nodes

CS 477/67724 Binary Search Trees Tree representation: –A linked data structure in which each node is an object Node representation: –Key field –Satellite data –Left: pointer to left child –Right: pointer to right child –p: pointer to parent ( p [root [T]] = NIL ) Satisfies the binary-search-tree property Left child Right child LR parent keydata

CS 477/67725 Binary Search Tree Example Binary search tree property: –If y is in left subtree of x, then key [y] ≤ key [x] –If y is in right subtree of x, then key [y] ≥ key [x]

CS 477/67726 Traversing a Binary Search Tree Inorder tree walk: –Prints the keys of a binary tree in sorted order –Root is printed between the values of its left and right subtrees: left, root, right Preorder tree walk: –root printed first: root, left, right Postorder tree walk: left, right, root –root printed last Preorder: Inorder: Postorder:

CS 477/67727 Traversing a Binary Search Tree Alg: INORDER-TREE-WALK (x) 1. if x  NIL 2. then INORDER-TREE-WALK ( left [x] ) 3. print key [x] 4. INORDER-TREE-WALK ( right [x] ) E.g.: Running time: –  (n), where n is the size of the tree rooted at x Output:

CS 477/67728 Searching for a Key Given a pointer to the root of a tree and a key k : –Return a pointer to a node with key k if one exists –Otherwise return NIL Idea –Starting at the root: trace down a path by comparing k with the key of the current node: If the keys are equal: we have found the key If k < key[x] search in the left subtree of x If k > key[x] search in the right subtree of x

CS 477/67729 Searching for a Key Alg: TREE-SEARCH (x, k) 1. if x = NIL or k = key [x] 2. then return x 3. if k < key [x] 4. then return TREE-SEARCH (left [x], k ) 5. else return TREE-SEARCH (right [x], k ) Running Time: O (h), h – the height of the tree

CS 477/67730 Example: TREE-SEARCH Search for key 13: –15  6  7 

CS 477/67731 Iterative Tree Search Alg: ITERATIVE-TREE-SEARCH (x, k) 1. while x  NIL and k  key [x] 2. do if k < key [x] 3. then x  left [x] 4. else x  right [x] 5. return x

CS 477/67732 Finding the Minimum in a Binary Search Tree Goal: find the minimum value in a BST –Following left child pointers from the root, until a NIL is encountered Alg: TREE-MINIMUM (x) 1. while left [x]  NIL 2. do x ← left [x] 3. return x Running time: O(h), h – height of tree Minimum = 2

CS 477/67733 Finding the Maximum in a Binary Search Tree Maximum = 20 Goal: find the maximum value in a BST –Following right child pointers from the root, until a NIL is encountered Alg: TREE-MAXIMUM(x) 1. while right [x]  NIL 2. do x ← right [x] 3. return x Running time: O(h), h – height of tree

CS 477/67734 Successor Def: successor ( x ) = y, such that key [y] is the smallest key > key [x] E.g.: successor (15) = successor (13) = successor (9) = Case 1: right (x) is non empty –successor ( x ) = the minimum in right (x) Case 2: right (x) is empty –go up the tree until the current node is a left child: successor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the largest element

CS 477/67735 Finding the Successor Alg: TREE-SUCCESSOR(x) 1. if right [x]  NIL 2. then return TREE-MINIMUM( right [x] ) 3. y ← p[x] 4. while y  NIL and x = right [y] 5. do x ← y 6. y ← p[y] 7. return y Running time: O (h), h – height of the tree y x

CS 477/67736 Predecessor Def: predecessor ( x ) = y, such that key [y] is the biggest key < key [x] E.g.: predecessor (15) = predecessor (9) = predecessor (13) = Case 1: left (x) is non empty –predecessor ( x ) = the maximum in left (x) Case 2: left (x) is empty –go up the tree until the current node is a right child: predecessor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the smallest element

CS 477/ Insertion Goal: –Insert value v into a binary search tree Idea: –If key [x] < v move to the right child of x, else move to the left child of x –When x is NIL, we found the correct position –If v < key [y] insert the new node as y ’s left child else insert it as y ’s right child –Begining at the root, go down the tree and maintain: Pointer x : traces the downward path (current node) Pointer y : parent of x (“trailing pointer” ) Insert value 13

CS 477/67738 Example: TREE-INSERT x, y=NIL Insert 13: x x x = NIL y = y y

CS 477/67739 Alg: TREE-INSERT (T, z) 1. y ← NIL 2. x ← root [T] 3. while x ≠ NIL 4. do y ← x 5. if key [z] < key [x] 6. then x ← left [x] 7. else x ← right [x] 8. p[z] ← y 9. if y = NIL 10. then root [T] ← z Tree T was empty 11. else if key [z] < key [y] 12. then left [y] ← z 13. else right [y] ← z Running time: O(h)

CS 477/67740 Readings Chapter 6 Chapter 12