1. Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9

2. CS 477/677 - Lecture 9 A Job Scheduling Application Job scheduling –The key is the priority of the jobs in the queue –The job with the highest priority needs to be executed next Operations –Insert, remove maximum Data structures –Priority queues –Ordered array/list, unordered array/list

3. CS 477/677 - Lecture 9 PQ Implementations & Cost Worst-case asymptotic costs for a PQ with N items InsertRemove max ordered array ordered list unordered array unordered list N N N N 1 1 1 1 Can we implement both operations efficiently?

4. CS 477/677 - Lecture 9 Background on Trees Def: Binary tree = structure composed of a finite set of nodes that either: –Contains no nodes, or –Is composed of three disjoint sets of nodes: a root node, a left subtree and a right subtree 2 148 1 16 4 3 910 root Right subtree Left subtree

5. CS 477/677 - Lecture 9 Special Types of Trees Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2. Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2. Full binary tree 2 148 1 16 7 4 3 910 12 Complete binary tree 2 1 16 4 3 910

6. CS 477/677 - Lecture 9 The Heap Data Structure Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (heap) property: for any node x Parent(x) ≥ x Heap 5 7 8 4 It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 2

7. CS 477/677 - Lecture 9 Definitions Height of a node = the number of edges on a longest simple path from the node down to a leaf Depth of a node = the length of a path from the root to the node Height of tree = height of root node =  lgn , for a heap of n elements 2 148 1 16 4 3 910 Height of root = 3 Height of (2)= 1 Depth of (10)= 2

8. CS 477/677 - Lecture 9 Array Representation of Heaps A heap can be stored as an array A. –Root of tree is A[1] –Left child of A[i] = A[2i] –Right child of A[i] = A[2i + 1] –Parent of A[i] = A[  i/2  ] –Heapsize[A] ≤ length[A] The elements in the subarray A[(  n/2  +1).. n] are leaves The root is the maximum element of the heap

9. CS 477/677 - Lecture 9 Heap Types Max-heaps (largest element at root), have the max-heap property: –for all nodes i, excluding the root: A[PARENT(i)] ≥ A[i] Min-heaps (smallest element at root), have the min-heap property: –for all nodes i, excluding the root: A[PARENT(i)] ≤ A[i]

10. CS 477/677 - Lecture 9 Operations on Heaps Maintain the max-heap property –MAX-HEAPIFY Create a max-heap from an unordered array –BUILD-MAX-HEAP Sort an array in place –HEAPSORT Priority queue operations

11. CS 477/677 - Lecture 9 Operations on Priority Queues Max-priority queues support the following operations: –INSERT (S, x) : inserts element x into set S –EXTRACT-MAX (S) : removes and returns element of S with largest key –MAXIMUM (S) : returns element of S with largest key –INCREASE-KEY (S, x, k) : increases value of element x ’s key to k (assume k ≥ current key value at x )

12. CS 477/677 - Lecture 9 Building a Heap Alg: BUILD-MAX-HEAP (A) 1.n = length[A] 2. for i ←  n/2  downto 1 3. do MAX-HEAPIFY (A, i, n) Convert an array A[1 … n] into a max-heap ( n = length[A] ) The elements in the subarray A[(  n/2  +1).. n] are leaves Apply MAX-HEAPIFY on elements between 1 and  n/2  2 148 1 16 7 4 3 910 1 23 4567 89 4132169101487 A:

13. CS 477/677 - Lecture 9 Example: A 4132169101487 2 8 1 16 7 4 3 910 1 23 4567 89 14 28 1 16 7 4 10 93 1 23 4567 89 2 148 1 16 7 4 3 910 1 23 4567 89 14 28 1 16 7 4 3 910 1 23 4567 89 14 28 16 7 1 4 10 93 1 23 4567 89 8 24 14 7 1 16 10 93 1 23 4567 89 i = 5 i = 4 i = 3 i = 2 i = 1

14. CS 477/677 - Lecture 9 Correctness of BUILD-MAX-HEAP Loop invariant: –At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap Initialization: –i =  n/2  : Nodes  n/2  +1,  n/2  +2, …, n are leaves  they are the root of trivial max-heaps 2 148 1 16 7 4 3 910 1 23 4567 89

15. CS 477/677 - Lecture 9 Correctness of BUILD-MAX-HEAP Maintenance: –MAX-HEAPIFY makes node i a max- heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps –Decrementing i in the for loop reestablishes the loop invariant Termination: –i = 0  each node 1, 2, …, n is the root of a max-heap (by the loop invariant) 2 148 1 16 7 4 3 910 1 23 4567 89

16. CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP  It would seem that running time is O(nlgn) This is not an asymptotically tight upper bound Alg: BUILD-MAX-HEAP (A) 1.n = length[A] 2. for i ←  n/2  downto 1 3. do MAX-HEAPIFY (A, i, n) O(lgn) O(n)

17. CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP HEAPIFY takes O(h)  the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree HeightLevel h 0 = 3 (  lgn  ) h 1 = 2 h 2 = 1 h 3 = 0 i = 0 i = 1 i = 2 i = 3 (  lgn  ) No. of nodes 2020 2121 2 2323 h i = h – i height of the heap rooted at level i n i = 2 i number of nodes at level i

18. CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP Cost of HEAPIFY at level i  number of nodes at that level Replace the values of n i and h i computed before Multiply by 2 h both at the nominator and denominator and write 2 i as Change variables: k = h - i The sum above is smaller than the sum of all elements to  and h = lgn The sum above is smaller than 2 Running time of BUILD-MAX-HEAP: T(n) = O(n)

19. CS 477/677 - Lecture 9 Binary Search Trees Tree representation: –A linked data structure in which each node is an object Node representation: –Key field –Satellite data –Left: pointer to left child –Right: pointer to right child –p: pointer to parent ( p [root [T]] = NIL ) Satisfies the binary search tree property Left child Right child LR parent keydata

20. CS 477/677 - Lecture 9 Binary Search Tree Example Binary search tree property: –If y is in left subtree of x, then key [y] ≤ key [x] –If y is in right subtree of x, then key [y] ≥ key [x] 2 3 5 5 7 9

21. CS 477/677 - Lecture 9 Binary Search Trees Support many dynamic set operations –SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT, DELETE Running time of basic operations on binary search trees –On average:  (lgn) The expected height of the tree is lgn –In the worst case:  (n) The tree is a linear chain of n nodes

22. CS 477/677 - Lecture 9 Red-Black Trees “Balanced” binary trees guarantee an O(lgn) running time on the basic dynamic-set operations Red-black tree –Binary tree with an additional attribute for its nodes: color which can be red or black –Constrains the way nodes can be colored on any path from the root to a leaf Ensures that no path is more than twice as long as another  the tree is balanced –The nodes inherit all the other attributes from the binary-search trees: key, left, right, p

23. CS 477/677 - Lecture 9 Red-Black Trees Properties 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black No two red nodes in a row on a simple path from the root to a leaf 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes

24. CS 477/677 - Lecture 9 Example: RED-BLACK TREE For convenience we use a sentinel NIL[T] to represent all the NIL nodes at the leafs –NIL[T] has the same fields as an ordinary node –Color[NIL[T]] = BLACK –The other fields may be set to arbitrary values 26 1741 3047 3850 NIL

25. CS 477/677 - Lecture 9 Black-Height of a Node Height of a node: the number of edges in a longest path to a leaf Black-height of a node x: bh(x) is the number of black nodes (including NIL) on a path from x to leaf, not counting x 26 1741 3047 3850 NIL h = 4 bh = 2 h = 3 bh = 2 h = 2 bh = 1 h = 1 bh = 1 h = 1 bh = 1 h = 2 bh = 1 h = 1 bh = 1

26. CS 477/677 - Lecture 9 Properties of Red-Black Trees Claim –Any node with height h has black-height ≥ h/2 Proof –By property 4, there are at most h/2 red nodes on the path from the node to a leaf –Hence at least h/2 are black 26 1741 3047 3850 Property 4: if a node is red then both its children are black

27. CS 477/677 - Lecture 9 Properties of Red-Black Trees Claim The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Proof: By induction on height of x Basis: height[x] = 0  x is a leaf ( NIL[T] )  bh(x) = 0  # of internal nodes: 2 0 - 1 = 0 NIL x

28. CS 477/677 - Lecture 9 Properties of Red-Black Trees Inductive step: Let height ( x) = h and bh(x) = b Any child y of x has: –bh (y) = 26 1741 3047 3850 b (if the child is red), or b - 1 (if the child is black)

29. CS 477/677 - Lecture 9 Properties of Red-Black Trees Want to prove: –The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Assume true for children of x : –Their subtrees contain at least 2 bh(x) - 1 – 1 internal nodes The subtree rooted at x contains at least: (2 bh(x) - 1 – 1) + (2 bh(x) - 1 – 1) + 1 = 2 · (2 bh(x) - 1 - 1) + 1 = 2 bh(x) - 1 internal nodes x l r

30. CS 477/677 - Lecture 9 Properties of Red-Black Trees Lemma: A red-black tree with n internal nodes has height at most 2lg(n + 1). Proof: n Add 1 to all sides and then take logs: n + 1 ≥ 2 b ≥ 2 h/2 lg(n + 1) ≥ h/2  h ≤ 2 lg(n + 1) root l r height(root) = h bh(root) = b number n of internal nodes ≥ 2 b - 1 ≥ 2 h/2 - 1 since b  h/2

31. CS 477/677 - Lecture 9 Operations on Red-Black Trees The non-modifying binary-search tree operations MINIMUM, MAXIMUM, SUCCESSOR, PREDECESSOR, and SEARCH run in O(h) time –They take O(lgn) time on red-black trees What about TREE-INSERT and TREE-DELETE? –They will still run in O(lgn) –We have to guarantee that the modified tree will still be a red-black tree

32. INSERT INSERT: what color to make the new node? Red? Let’s insert 35! –Property 4: if a node is red, then both its children are black Black? Let’s insert 14! –Property 5: all paths from a node to its leaves contain the same number of black nodes 26 1741 3047 3850 CS 477/677 - Lecture 9

33. DELETE DELETE: what color was the node that was removed? Red? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! OK! Does not create two red nodes in a row OK! Does not change any black heights 26 1741 3047 3850 CS 477/677 - Lecture 9

34. DELETE DELETE: what color was the node that was removed? Black? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! Not OK! Could create two red nodes in a row Not OK! Could change the black heights of some nodes 26 1741 3047 3850 Not OK! If removing the root and the child that replaces it is red CS 477/677 - Lecture 9

35. Rotations Operations for restructuring the tree after insert and delete operations on red-black trees Rotations take a red-black tree and a node within the tree and: –Together with some node re-coloring they help restore the red-black tree property –Change some of the pointer structure –Do not change the binary-search tree property Two types of rotations: –Left & right rotations CS 477/677 - Lecture 9

36. Left Rotations Assumption for a left rotation on a node x : –The right child of x (y) is not NIL Idea: –Pivots around the link from x to y –Makes y the new root of the subtree –x becomes y ’s left child –y ’s left child becomes x ’s right child CS 477/677 - Lecture 9

37. Readings Chapter 8, Chapter 6