1. Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 11

2. CS 477/677 - Lecture 11 Binary Search Trees Support many dynamic set operations –SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT, DELETE Running time of basic operations on binary search trees –On average:  (lgn) The expected height of the tree is lgn –In the worst case:  (n) The tree is a linear chain of n nodes 2

3. CS 477/677 - Lecture 11 Binary Search Trees Tree representation: –A linked data structure in which each node is an object Node representation: –Key field –Satellite data –Left: pointer to left child –Right: pointer to right child –p: pointer to parent ( p [root [T]] = NIL ) Satisfies the binary search tree property Left child Right child LR parent keydata 3

4. CS 477/677 - Lecture 11 Binary Search Tree Example Binary search tree property: –If y is in left subtree of x, then key [y] ≤ key [x] –If y is in right subtree of x, then key [y] ≥ key [x] 2 3 5 5 7 9 4

5. CS 477/677 - Lecture 11 Successor Def: successor ( x ) = y, such that key [y] is the smallest key > key [x] E.g.: successor (15) = successor (13) = successor (9) = Case 1: right (x) is non empty –successor ( x ) = the minimum in right (x) Case 2: right (x) is empty –go up the tree until the current node is a left child: successor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the largest element 3 24 6 7 13 15 18 1720 9 17 15 13 5

6. CS 477/677 - Lecture 11 Finding the Successor Alg: TREE-SUCCESSOR(x) 1. if right [x]  NIL 2. then return TREE-MINIMUM( right [x] ) 3. y ← p[x] 4. while y  NIL and x = right [y] 5. do x ← y 6. y ← p[y] 7. return y Running time: O (h), h – height of the tree 3 24 6 7 13 15 18 1720 9 y x 6

7. CS 477/677 - Lecture 11 Predecessor Def: predecessor ( x ) = y, such that key [y] is the largest key < key [x] E.g.: predecessor (15) = predecessor (9) = predecessor (13) = Case 1: left (x) is non empty –predecessor ( x ) = the maximum in left (x) Case 2: left (x) is empty –go up the tree until the current node is a right child: predecessor ( x ) is the parent of the current node –if you cannot go further (and you reached the root): x is the smallest element 3 24 6 7 13 15 18 1720 9 13 7 9 7

8. CS 477/677 - Lecture 11 Insertion Goal: –Insert value v into a binary search tree Idea: –Begining at the root, go down the tree and maintain: Pointer x : traces the downward path (current node) Pointer y : parent of x (“trailing pointer” ) –If key [x] < v move to the right child of x, else move to the left child of x –When x is NIL, we found the correct position –If v < key [y] insert the new node as y ’s left child else insert it as y ’s right child 13 2 13 5 9 12 18 1519 17 Insert value 13 8

9. CS 477/677 - Lecture 11 Example: TREE-INSERT 2 13 5 9 12 18 1519 17 x, y=NIL Insert 13: 2 13 5 9 12 18 1519 17 x 2 13 5 9 12 18 15 19 17 x x = NIL y = 15 13 2 13 5 9 12 18 1519 17 y y 9

10. CS 477/677 - Lecture 11 Alg: TREE-INSERT (T, z) 1. y ← NIL 2. x ← root [T] 3. while x ≠ NIL 4. do y ← x 5. if key [z] < key [x] 6. then x ← left [x] 7. else x ← right [x] 8. p[z] ← y 9. if y = NIL 10. then root [T] ← z Tree T was empty 11. else if key [z] < key [y] 12. then left [y] ← z 13. else right [y] ← z 2 13 5 9 12 18 1519 17 13 Running time: O(h) 10

11. CS 477/677 - Lecture 11 Deletion Goal: –Delete a given node z from a binary search tree Idea: –Case 0: z has no children Delete z by making the parent of z point to NIL, instead of to z 15 16 20 1823 6 5 12 3 7 10 13 delete 15 16 20 1823 6 5 12 3 7 10 z 11

12. CS 477/677 - Lecture 11 Deletion Case 1: z has one child –Delete z by making the parent of z point to z ’s child, instead of to z –Update the parent of z’s child to be z’s parent 15 16 20 1823 6 5 12 3 7 1013 delete 15 20 1823 6 5 12 3 7 10 z 12

13. CS 477/677 - Lecture 11 Deletion Case 2: z has two children –z ’s successor (y) is the minimum node in z ’s right subtree –y has either no children or one right child (but no left child) –Delete y from the tree (via Case 0 or 1) –Replace z’s key and satellite data with y’s. 15 16 20 1823 6 5 12 3 7 1013 delete z y 15 16 20 1823 7 6 12 3 1013 6

14. CS 477/677 - Lecture 11 Idea for TREE-DELETE (T, z) Determine a node y that has to be deleted –If z has 0 or 1 child  y = z (case 0 or 1) –If z has 2 children  y = TREE-SUCCESSOR( z ) (case 2) –In any case y has at most 1 child!!! Set a node x to the non-nil child of y Delete node y : set the parent of x to be the parent of y If y is the root, x becomes the new root otherwise, update parent pointers accordingly If the deleted node y was the successor of z (case 2), move y ’s key and satellite data onto z The deleted node y is returned for recycling 14

15. CS 477/677 - Lecture 11 TREE-DELETE (T, z) 1. if left[z] = NIL or right[z] = NIL 2. then y ← z 3. else y ← TREE-SUCCESSOR( z ) 4. if left[y]  NIL 5. then x ← left[y] 6. else x ← right[y] 7. if x  NIL 8. then p[x] ← p[y] z has 0 or 1 child z has 2 children 15 16 20 1823 6 5 12 3 7 1013 y x 15

16. CS 477/677 - Lecture 11 TREE-DELETE (T, z) – cont. 9. if p[y] = NIL 10. then root[T] ← x 11. else if y = left[p[y]] 12. then left[p[y]] ← x 13. else right[p[y]] ← x 14. if y  z 15. then key[z] ← key[y] 16. copy y’s satellite data into z 17. return y 15 16 20 1823 6 5 12 3 7 1013 y x 16

17. CS 477/677 - Lecture 11 Binary Search Trees - Summary Operations on binary search trees: –SEARCH O(h) –PREDECESSOR O(h) –SUCCESOR O(h) –MINIMUM O(h) –MAXIMUM O(h) –INSERT O(h) –DELETE O(h) These operations are fast if the height of the tree is small – otherwise their performance is similar to that of a linked list 17

18. CS 477/677 - Lecture 11 Red-Black Trees “Balanced” binary trees guarantee an O(lgn) running time on the basic dynamic-set operations Red-black tree –Binary tree with an additional attribute for its nodes: color which can be red or black –Constrains the way nodes can be colored on any path from the root to a leaf Ensures that no path is more than twice as long as another  the tree is balanced –The nodes inherit all the other attributes from the binary-search trees: key, left, right, p 18

19. CS 477/677 - Lecture 11 Red-Black Trees Properties 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black No two red nodes in a row on a simple path from the root to a leaf 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes 19

20. CS 477/677 - Lecture 11 Example: RED-BLACK TREE For convenience we use a sentinel NIL[T] to represent all the NIL nodes at the leafs –NIL[T] has the same fields as an ordinary node –Color[NIL[T]] = BLACK –The other fields may be set to arbitrary values 26 1741 3047 3850 NIL 20

21. CS 477/677 - Lecture 11 Black-Height of a Node Height of a node: the number of edges in a longest path to a leaf Black-height of a node x: bh(x) is the number of black nodes (including NIL) on a path from x to leaf, not counting x 26 1741 3047 3850 NIL h = 4 bh = 2 h = 3 bh = 2 h = 2 bh = 1 h = 1 bh = 1 h = 1 bh = 1 h = 2 bh = 1 h = 1 bh = 1 21

22. CS 477/677 - Lecture 11 Properties of Red-Black Trees Claim –Any node with height h has black-height ≥ h/2 Proof –By property 4, there are at most h/2 red nodes on the path from the node to a leaf –Hence at least h/2 are black 26 1741 3047 3850 Property 4: if a node is red then both its children are black 22

23. CS 477/677 - Lecture 11 Properties of Red-Black Trees Claim –The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Proof: By induction on height of x Basis: height[x] = 0  x is a leaf ( NIL[T] )  bh(x) = 0  # of internal nodes: 2 0 - 1 = 0 NIL x 23

24. CS 477/677 - Lecture 11 Properties of Red-Black Trees Inductive step: Let height ( x) = h and bh(x) = b Any child y of x has: –bh (y) = 26 1741 3047 3850 b (if the child is red), or b - 1 (if the child is black) 24

25. CS 477/677 - Lecture 11 Properties of Red-Black Trees Want to prove: –The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Assume true for children of x : –Their subtrees contain at least 2 bh(x) - 1 – 1 internal nodes The subtree rooted at x contains at least: (2 bh(x) - 1 – 1) + (2 bh(x) - 1 – 1) + 1 = 2 · (2 bh(x) - 1 - 1) + 1 = 2 bh(x) - 1 internal nodes x l r 25

26. CS 477/677 - Lecture 11 Properties of Red-Black Trees Lemma: A red-black tree with n internal nodes has height at most 2lg(n + 1). Proof: n Add 1 to all sides and then take logs: n + 1 ≥ 2 b ≥ 2 h/2 lg(n + 1) ≥ h/2  h ≤ 2 lg(n + 1) root l r height(root) = h bh(root) = b number n of internal nodes ≥ 2 b - 1 ≥ 2 h/2 - 1 since b  h/2 26

27. CS 477/677 - Lecture 11 Operations on Red-Black Trees The non-modifying binary-search tree operations MINIMUM, MAXIMUM, SUCCESSOR, PREDECESSOR, and SEARCH run in O(h) time –They take O(lgn) time on red-black trees What about TREE-INSERT and TREE-DELETE? –They will still run in O(lgn) –We have to guarantee that the modified tree will still be a red-black tree 27

28. CS 477/677 - Lecture 11 INSERT INSERT: what color to make the new node? Red? Let’s insert 35! –Property 4: if a node is red, then both its children are black Black? Let’s insert 14! –Property 5: all paths from a node to its leaves contain the same number of black nodes 26 1741 3047 3850 28

29. CS 477/677 - Lecture 11 DELETE DELETE: what color was the node that was removed? Red? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! OK! Does not create two red nodes in a row OK! Does not change any black heights 26 1741 3047 3850 29

30. DELETE DELETE: what color was the node that was removed? Black? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! Not OK! Could create two red nodes in a row Not OK! Could change the black heights of some nodes 26 1741 3047 3850 Not OK! If removing the root and the child that replaces it is red 30CS 477/677 - Lecture 11

31. Rotations Operations for restructuring the tree after insert and delete operations on red-black trees Rotations take a red-black tree and a node within the tree and: –Together with some node re-coloring they help restore the red-black tree property –Change some of the pointer structure –Do not change the binary-search tree property Two types of rotations: –Left & right rotations 31CS 477/677 - Lecture 11

32. Left Rotations Assumption for a left rotation on a node x : –The right child of x (y) is not NIL Idea: –Pivots around the link from x to y –Makes y the new root of the subtree –x becomes y ’s left child –y ’s left child becomes x ’s right child 32CS 477/677 - Lecture 11

33. Example: LEFT-ROTATE 33CS 477/677 - Lecture 11

34. Readings Chapters 12, 13 34