Balanced Search Trees CS Data Structures Mehmet H Gunes Modified from authors’ slides

Contents Balanced Search Trees 2-3 Trees Trees Red-Black Trees AVL Trees Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Balanced Search Trees Height of a binary search tree sensitive to order of insertions and removals – Minimum = log 2 (n + 1) – Maximum = n Various search trees can retain balance despite insertions and removals Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Balanced Search Trees (a) A binary search tree of maximum height; (b) a binary search tree of minimum height Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3 Trees A 2-3 tree of height 3 A 2-3 tree is not a binary tree A 2-3 tree never taller than a minimum- height binary tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3 Trees Placing data items in nodes of a 2-3 tree – A 2-node (has two children) must contain single data item greater than left child’s item(s) and less than right child’s item(s) – A 3-node (has three children) must contain two data items, S and L, such that S is greater than left child’s item(s) and less than middle child’s item(s); L is greater than middle child’s item(s) and less than right child’s item(s). – Leaf may contain either one or two data items. Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3 Trees Nodes in a 2-3 tree: (a) a 2-node; (b) a 3-node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013`

2-3 Trees A 2-3 tree View Header file for a class of nodes for a 2-3 tree, Listing 19-1Listing 19-1 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Traversing a 2-3 Tree Traverse 2-3 tree in sorted order by performing analogue of inorder traversal on binary tree: Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree Retrieval operation for 2-3 tree similar to retrieval operation for binary search tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree Possible to search 2-3 tree and shortest binary search tree with approximately same efficiency, because: – Binary search tree with n nodes cannot be shorter than log 2 (n + 1) – 2-3 tree with n nodes cannot be taller than log 2 (n + 1) – Node in a 2-3 tree has at most two items Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree A balanced binary search tree; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree a 2-3 tree with the same entries Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Searching a 2-3 Tree (a) The binary search tree of Figure 19-5a after inserting the sequence of values 32 through 39 (b) the 2-3 tree of Figure 19-5 b after the same insertions Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree After inserting 39 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree The steps for inserting 38 into the tree: (a) The located node has no room; (b) the node splits; (c) the resulting tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree After inserting 37 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree (a), (b), (c) The steps for inserting 36 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree (d) the resulting tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree The tree after the insertion of 35, 34, and 33 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree Splitting a leaf in a 2-3 tree when the leaf is a (a) left child; (b) right child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree Splitting an internal node in a 2-3 tree when the node is a (a) left child; (b) right child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree Splitting the root of a 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree Summary of insertion strategy Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Inserting Data into a 2-3 Tree Summary of insertion strategy Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree A 2-3 tree; (b), (c), (d), (e) the steps for removing 70; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree (f) the resulting tree; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree (a), (b), (c) The steps for removing 100 from the tree; (d) the resulting tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree The steps for removing 80 from the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree The steps for removing 80 from the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree Results of removing 70, 100, and 80 from (a) the 2-3 tree and (b) the binary search tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree Algorithm for removing data from a 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree Algorithm for removing data from a 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree Algorithm for removing data from a 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree (a) Redistributing values; (b) merging a leaf; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree (c) redistributing values and children; (d) merging internal nodes Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Removing Data from a 2-3 Tree (e) deleting the root Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees A tree with the same data items as the 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Rules for placing data items in the nodes of a tree – 2-node (two children), must contain a single data item that satisfies relationships – 3-node (three children), must contain two data items that satisfies relationships –... Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees – 4-node (four children) must contain three data items S, M, and L that satisfy: S is greater than left child’s item(s) and less than middle-left child’s item(s) M is greater than middle-left child’s item(s) and less than middle-right child’s item(s); L is greater than middle-right child’s item(s) and less than right child’s item(s). – A leaf may contain either one, two, or three data items Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees A 4-node in a tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Has more efficient insertion and removal operations than a 2-3 tree Has greater storage requirements due to the additional data members in its 4-nodes Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Searching and Traversing a Tree – Simple extensions of the corresponding algorithms for a 2-3 tree Inserting Data into a Tree – Insertion algorithm splits a node by moving one of its items up to its parent node – Splits 4-nodes as soon as it encounters them on the way down the tree from the root to a leaf Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Inserting 20 into a one-node tree (a) the original tree; (b) after splitting the node; (c) after inserting 20 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees After inserting 50 and 40 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees The steps for inserting 70 into the tree : (a) after splitting the 4-node; (b) after inserting 70 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees After inserting 80 and 15 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees The steps for inserting 90 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees The steps for inserting 100 into the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Splitting a 4-node root during insertion into a tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Splitting a 4-node whose parent is a 2-node during insertion into a tree, when the 4-node is a (a) left child; (b) right child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Splitting a 4-node whose parent is a 3-node during insertion into a tree, when the 4-node is a (a) left child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Splitting a 4-node whose parent is a 3-node during insertion into a tree, when the 4-node is a (b) middle child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Splitting a 4-node whose parent is a 3-node during insertion into a tree, when the 4-node is a (c) right child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

2-3-4 Trees Removing Data from a Tree – Removal algorithm has same beginning as removal algorithm for a 2-3 tree – Locate the node n that contains the item I you want to remove – Find I ’s inorder successor and swap it with I so that the removal will always be at a leaf – If leaf is either a 3-node or a 4-node, remove I. Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Modified from Dr George Bebis and Dr Monica Nicolescu

Red-Black Trees Use a special binary search tree—a red-black tree —to represent a tree Retains advantages of a tree without storage overhead The idea is to represent each 3-node and 4- node as an equivalent binary search tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Red-black representation of (a) a 4-node; (b) a 3-node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees A red-black tree that represents the tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Searching and traversing – Red-black tree is a binary search tree, search and traverse it by using algorithms for binary search tree Inserting, removing with a red-black tree – Adjust the insertion algorithms to accommodate the red-black representation Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red-black representation of a 4-node that is the root Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red-black representation of a 4-node whose parent is a 2-node, when the 4-node is a (a) left child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red-black representation of a 4-node whose parent is a 2-node, when the 4-node is a (b) right child Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red-black representation of a 4-node whose parent is a 3-node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red-black representation of a 4-node whose parent is a 3-node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black Trees Splitting a red- black representation of a 4-node whose parent is a 3-node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

Red-Black-Trees Properties (**Binary search tree property is satisfied**) 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black No two consecutive red nodes on a simple path from the root to a leaf 5.For each node, all paths from that node to a leaf contain the same number of black nodes 68

Example: RED-BLACK-TREE For convenience, we add NIL nodes and refer to them as the leaves of the tree. – Color[NIL] = BLACK NIL 69

Definitions Height of a node: the number of edges in the longest path to a leaf Black-height bh(x) of a node x: the number of black nodes (including NIL) on the path from x to a leaf, not counting x NIL h = 4 bh = 2 h = 3 bh = 2 h = 2 bh = 1 h = 1 bh = 1 h = 1 bh = 1 h = 2 bh = 1 h = 1 bh = 1 70

Height of Red-Black-Trees A red-black tree with n internal nodes has height at most 2log(N+1) 71

Insert Item What color to make the new node? Red? – Let’s insert 35! Property 4 is violated: if a node is red, then both children are black Black? – Let’s insert 14! Property 5 is violated: all paths from a node to its leaves contain the same number of black nodes

Delete Item What color was the node that was removed? Red? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! OK!

Delete Item What color was the node that was removed? Black? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! Not OK! Could create two red nodes in a row Not OK! Could change the black heights of some nodes Not OK! If removing the root and the child that replaces it is red

Rotations Operations for re-structuring the tree after insert and delete operations – Together with some node re-coloring, they help restore the red-black-tree property – Change some of the pointer structure – Preserve the binary-search tree property Two types of rotations: – Left & right rotations 75

Left Rotations Assumptions for a left rotation on a node x : – The right child y of x is not NIL Idea: – Pivots around the link from x to y – Makes y the new root of the subtree – x becomes y ’s left child – y ’s left child becomes x ’s right child 76

Example: LEFT-ROTATE 77

LEFT-ROTATE(T, x) 1.y ← right[x] ► Set y 2.right[x] ← left[y] ► y’s left subtree becomes x’s right subtree 3.if left[y]  NIL 4. then p[left[y]] ← x ► Set the parent relation from left[y] to x 5.p[y] ← p[x] ► The parent of x becomes the parent of y 6.if p[x] = NIL 7. then root[T] ← y 8. else if x = left[p[x]] 9. then left[p[x]] ← y 10. else right[p[x]] ← y 11.left[y] ← x ► Put x on y’s left 12.p[x] ← y ► y becomes x’s parent

Assumptions for a right rotation on a node x : – The left child x of y is not NIL Idea: – Pivots around the link from y to x – Makes x the new root of the subtree – y becomes x ’s right child – x ’s right child becomes y ’s left child Right Rotations 79

Insert Item Goal: – Insert a new node z into a red-black tree Idea: – Insert node z into the tree as for an ordinary binary search tree – Color the node red – Restore the red-black tree properties 80

RB-INSERT (T, z) 1. y ← NIL 2. x ← root[T] 3. while x  NIL 4. do y ← x 5. if key[z] < key[x] 6. then x ← left[x] 7. else x ← right[x] 8. p[z] ← y Initialize nodes x and y Throughout the algorithm y points to the parent of x Go down the tree until reaching a leaf At that point y is the parent of the node to be inserted Sets the parent of z to be y

RB-INSERT (T, z) 9. if y = NIL 10. then root[T] ← z 11. else if key[z] < key[y] 12. then left[y] ← z 13. else right[y] ← z 14. left[z] ← NIL 15. right[z] ← NIL 16. color[z] ← RED 17. RB-INSERT-FIXUP (T, z) The tree was empty: set the new node to be the root Otherwise, set z to be the left or right child of y, depending on whether the inserted node is smaller or larger than y’s key Set the fields of the newly added node Fix any inconsistencies that could have been introduced by adding this new red node

RB Properties Affected by Insert 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! If z is the root  not OK OK! If p(z) is red  not OK z and p(z) are both red OK!

RB-INSERT-FIXUP 84 Case 1: z ’s “uncle” (y) is red (z could be either left or right child) Idea: p[p[z]] ( z ’s grandparent) must be black color p[z]  black color y  black color p[p[z]]  red z = p[p[z]] – Push the “red” violation up the tree

RB-INSERT-FIXUP Case 2: z ’s “uncle” (y) is black z is a left child 85 Case 2 Idea: color p[z]  black color p[p[z]]  red RIGHT-ROTATE(T, p[p[z]]) No longer have 2 reds in a row p[z] is now black

RB-INSERT-FIXUP Case 3: z ’s “uncle” ( y ) is black z is a right child Idea: z  p[z] LEFT-ROTATE( T, z)  now z is a left child, and both z and p[z] are red  case 2 86 Case 3 Case 2

Example 11 Insert y z Case 1 y z and p[z] are both red z’s uncle y is red z z and p[z] are both red z’s uncle y is black z is a right child Case z y Case 2 z and p[z] are red z’s uncle y is black z is a left child z 87

RB-INSERT-FIXUP (T, z) 1.while color[p[z]] = RED 2. if p[z] = left[p[p[z]]] 3. then y ← right[p[p[z]]] 4. if color[y] = RED 5. then Case1 6. else if z = right[p[z]] 7. then Case3 8. Case2 9. else (same as then clause with “right” and “left” exchanged for lines 3-4) 10.color[root[T]] ← BLACK The while loop repeats only when case1 is executed: O(logN) times Set the value of x’s “uncle” We just inserted the root, or The red violation reached the root

Analysis of InsertItem Inserting the new element into the tree O(logN) RB-INSERT-FIXUP – The while loop repeats only if CASE 1 is executed – The number of times the while loop can be executed is O(logN) Total running time of Insert Item: O(logN) 89

Delete Item Delete as usually, then re-color/rotate A bit more complicated though … Demo –

Problems 91

What red-black tree property is violated in the tree below? How would you restore the red-black tree property in this case? – Property violated: if a node is red, both its children are black – Fixup: color 7 black, 11 red, then right-rotate around 11 Problems z

Problems 93 Let a, b, c be arbitrary nodes in subtrees , ,  in the tree below. How do the depths of a, b, c change when a left rotation is performed on node x? – a: increases by 1 – b: stays the same – c: decreases by 1

Problems When we insert a node into a red-black tree, we initially set the color of the new node to red. Why didn’t we choose to set the color to black? Would inserting a new node to a red-black tree and then immediately deleting it, change the tree? 94

AVL Trees Named for inventors – Adel’son-Vel’skii and Landis A balanced binary search tree – Maintains height close to the minimum – After insertion or deletion, check the tree is still AVL tree determine whether any node in tree has left and right subtrees whose heights differ by more than 1 Can search AVL tree almost as efficiently as minimum-height binary search tree. Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

AVL Trees (a) An unbalanced binary search tree; (b) a balanced tree after rotation; (c) a balanced tree after insertion Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013

(a) Before; (b) and after a single left rotation that decreases the tree’s height; (c) the rotation in general Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 AVL Trees

(a) Before; (b) and after a single left rotation that does not affect the tree’s height; (c) the rotation in general Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 AVL Trees

(d) the double rotation in general Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013