BST Trees Saurav Karmakar 5 3 7 1 2 8

Binary search tree (BST) A binary search tree is a binary tree in which every node satisfies the following: the key of every node in the left subtree is smaller than the key of this node the key of every node in the right subtree is larger than the key of this node Note: Duplication nodes ? 5 3 8 1 4 9

Binary search tree The left subtree and the right subtree are also BST (Recursion) No two nodes in a BST have the same key (??) If we traverse a BST inorder, we list the keys in ascending order 5 3 8 1 4 9

Some examples 1 7 9 3 5 8 15 10 2 12 6 14 3 4 16 8 8 4 2 6 1 3 7 5 12 10 14 9 11 15 13

Is this a BST? This is NOT BST! Every node satisfies the following: -- the key of the left node is smaller than the its own key -- the key of the right node is larger than its own key This is an incorrect check of BST. 5 3 8 1 7 9

Search BST 5 Find To find a key in a BST, we start at the root. Compare the target key with the key of the current node. - target == p->key, Done. - target < p->key, go left - target > p->key, go right Note that the principle behind is similar to binary search... 8 4 2 6 1 3 7 5 12 10 14 9 11 15 13 target

Recursive BST Search Algorithm // Algorithm for tree search // Finds the position of a node with the specified key TreeSearch(root, key); begin if root == nil then position := nil else if target == root->key then position := root else if target < root->key then TreeSearch(root->left, target) else TreeSearch(root->right, target); end;

C++ Implementation of BST Search // Internal method to find an item in a subtree. // x is item to search for // t is the node that roots the tree // Return node containing the matched item. template <class Comparable> BinaryNode<Comparable> * BinarySearchTree<Comparable>:: find( const Comparable & x, Node *t ) const { while( t != NULL ) { if( x < t->element ) // element is the key value for each node t = t->left; else if( t->element < x ) t = t->right; else return t; // Match } return NULL; // Not found

Balance tree The efficiency of BST search depends on the shape of the tree. If the tree is balance, (minimum height, very bushy), the search is fast (log(n) steps). If the tree is a long narrow chain, the search is slow (n steps) 1 15 2 14 Compare this with sequential and binary search ... 8 4 2 6 1 3 7 5 12 10 14 9 11 15 13 3 log(16)=4 4 5 13

Binary Search Tree Class - FindMin/Max template <class Comparable> BinaryNode<Comparable> * BinarySearchTree<Comparable>::findMin( Node *t ) const{ if( t != NULL ){ while( t->left != NULL ) t = t->left; // the leftmost node } return t; _ BinaryNode<Comparable> * BinarySearchTree<Comparable>::findMax( Node *t ) const{ If( t != NULL ) while( t->right != NULL ) t = t->right; // the rightmost node

Insert & Delete in BST First we’ll study simple-minded Insert and Delete procedure. They may result in highly unbalanced tree. Then we’ll study a special kind of BST called AVL tree which maintains near balance in inserting and deleting.

Insert in BST (recursive version) // Internal method to insert into a subtree // x is the item to insert // t is the node that roots the tree // Set the new root, if no root present // Throw an exception if x is already in t template <class Comparable> void BinarySearchTree<Comparable>:: insert( const Comparable & x, Node * & t ) const { if( t == NULL ) t = new Node( x, NULL, NULL ); else if( x < t->element ) insert( x, t->left ); else if( x > t->element) insert( x, t->right ); else throw DuplicateItemException( ); } 10 5 12 4 7 14 2 9 17 6 8 1 15 3 16 New nodes are always added as leaves.

Delete in BST  Case 1: If it has no children, that is, it is a leaf, we just remove it.

Delete in BST L  Case 2: If it has only one child, (i.e. only one subtree), we splice out it. That is, we attach that subtree to the parent of the node.

Delete in BST, examples   6 16 15 9 10 5 12 7 14 17 6 16 15 9 10 12 Case 2: delete a node that has only one subtree.

Delete in BST R L  R’ Min in R Case 3: If it has nonempty left and right subtrees. Find the minimum node in right subtree, then copy its key value to X, and remove the min node from right subtree. Or Find the max node in left subtree, then copy its key value to X, and remove the max node from left subtree.

Delete Tree, example 10 10  5 12 6 12 4 8 8 16 3 1 2 4 16 2 6 9 7 9 14 14 1 3 7 15 15 Case 3: delete a node that has nonempty left and right subtrees.

Delete in BST template <class Comparable> void BinarySearchTree<Comparable>:: remove( const Comparable & x, Node * & t ) const { if( t == NULL ) throw ItemNotFoundException( ); if( x < t->element ) remove( x, t->left ); else if( t->element < x ) remove( x, t->right ); else if( t->left != NULL && t->right != NULL ) { //case 3 t->element = findMin( t->right )->element; removeMin( t->right ); // Remove minimum } else{ // case 1, 2 BinaryNode<Comparable> *oldNode = t; t = ( t->left != NULL ) ? t->left : t->right; // Reroot t delete oldNode; // delete old root

Order statistics X SL SL SR SR SR SL findKth smallest element problem based on the consider of relationship of node size and K. X SL SL SR SR SR X X SL If K < SL +1, it is the Kth samllest element in the left subtree If K == SL +1, it is the root If K > SL +1, it is the K-(SL +1)th smallest on the right subtree Note: SL is the size of the left subtree

Binary Search Tree Binary search tree, using simple insert and delete procedures If the tree is nearly balanced add - fast O(log n) delete a target - fast O(log n) search - fast O(log n) If the tree is highly unbalanced, it becomes a singly linked list (the worst case) add, delete and search - slow O(n) effectively using sequential search to find a location

AVL Tree (Adelson-Velskii & Landis) AVL Tree are BST with an additional balance condition to ensure the depth of the tree is O(log2N) Balance Condition: for any node in the tree, the height of the left and right subtrees can differ by at most 1. Must ensure that after insertion and deletion of a node, the balance condition is preserved

Adding/deleteing node from AVL Tree * Add a node connected to some node? * Delete some node? After an insertion, the nodes on the path from the insertion point to the root can have their balances altered. Rotation restores local balance

Signle rotation k2 k1 k1 C k2 A A B B C

Rotate Binary Tree with Left Child template <class comparable> Void BST<comparable>::rotateWithLeftChild(Node * &k2) const { Node *k1 = k2left; k2left = k1right; k1right = k2; k2 = k1; }

Single Rotation y x x y 3 5 4 7 6 9 2 1 8 3 5 4 7 6 9 2 1 8 B A C B C

Single Rotation Doesn’t Work in Some Cases 3 5 4 8 6 9 2 1 7 3 5 4 8 6 9 2 1 7

Double Rotation y x z y x z 3 5 4 6 8 9 2 1 7 3 5 4 8 6 9 2 1 7 D A B C y D x A z B C

Red-black trees This data structure requires an extra one-bit color field in each node. Red-black properties: 1. Every node is either red or black. 2. The root and leaves (NIL’s) are black. 3. If a node is red, then its parent is black. 4. All simple paths from any node x to a descendant leaf have the same number of black nodes = black-height(x).

Example of a red-black tree

Example of a red-black tree 1. Every node is either red or black.

Example of a red-black tree 2. The root and leaves (NIL’s) are black.

Example of a red-black tree 3. If a node is red, then its parent is black.

Example of a red-black tree 4. All simple paths from any node x to a descendant leaf have the same number of black nodes = black-height(x).

Height of a red-black tree Theorem. A red-black tree with n keys has height h ≤ 2 lg(n + 1).

Query operations Corollary. The queries SEARCH, MIN, MAX, SUCCESSOR, and PREDECESSOR all run in O(lg n) time on a red-black tree with n nodes.

Modifying operations The operations INSERT and DELETE cause modifications to the red-black tree: • the operation itself, • color changes, • restructuring the links of the tree via “rotations”.

Rotations Rotations maintain the inorder ordering of keys: • a ∈ α, b ∈ β, c ∈ γ ⇒ a ≤ A ≤ b ≤ B ≤ c. A rotation can be performed in O(1) time.

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 3 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example:

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 3 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15. • Recolor, moving the violation up the tree.

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 3 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18).

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 3 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7) and recolor.

Insertion into a red-black tree IDEA: Insert x in tree. Color x red. Only redblack property 3 might be violated. Move the violation up the tree by recoloring until it can be fixed with rotations and recoloring. Example: • Insert x =15. • Recolor, moving the violation up the tree. • RIGHT-ROTATE(18). • LEFT-ROTATE(7) and recolor.

Pseudocode RB-INSERT(T, x) TREE-INSERT(T, x) color[x] ← RED ⊳ only RB property 3 can be violated while x ≠ root[T] and color[p[x]] = RED do if p[x] = left[p[p[x]] then y ← right[p[p[x]] ⊳ y = aunt/uncle of x if color[y] = RED then 〈Case 1〉 else if x = right[p[x]] then 〈Case 2〉 ⊳ Case 2 falls into Case 3 〈Case 3〉 else 〈“then” clause with “left” and “right” swapped〉 color[root[T]] ← BLACK

Graphical notation Let denote a subtree with a black root. All s have the same black-height.

Case 1 Push C’s black onto (Or, children of A and D, and recurse, since C’s parent may be red. (Or, children of A are swapped.)

Case 2 Transform to Case 3.

Case 3 Done! No more violations of RB property 3 are possible.

Red Black Tree (Insertion Example) Insert 10 – root 10

Red Black Tree (Insertion Example) Insert 10 – root (external nodes not shown) 10

Red Black Tree Insert 85 10 85

Red Black Tree Insert 15 10 85 15

Red Black Tree Rotate – Change colors 15 10 85

Red Black Tree Insert 70 15 10 85 70

Red Black Tree Change Color 15 10 85 70

Red Black Tree Insert 20 (sibling of parent is black) 15 10 85 70 20

Red Black Tree Rotate 15 10 70 85 20

Red Black Tree Insert 60 (sibling of parent is red) 15 10 70 85 20 60

Red Black Tree Change Color 15 10 70 85 20 60

Red Black Tree Insert 30 (sibling of parent is black) 15 10 70 85 20 60 30

Red Black Tree Rotate 15 10 70 85 30 20 60

Red Black Tree Insert 50 (sibling ?) 15 10 70 85 30 20 60 50

Red Black Tree Insert 50 (sibling of 70 is black!) gramps 15 15  Parent 70 10 70 85 30 Child  30 20 60 Oops, red-red. ROTATE! 50

Red Black Tree Double Rotate – Adjust colors 30 15 70 10 20 85 60 Child-Parent-Gramps Middle goes to “top” Previous top becomes child. Its right child is middles left child. 50

Red Black Tree Insert 65 30 15 70 10 20 85 60 50 65

Red Black Tree Insert 80 30 15 70 10 20 85 60 80 50 65

Red Black Tree Insert 90 30 15 70 10 20 85 60 80 90 50 65

Red Black Tree Insert 40 30 15 70 10 20 85 60 80 90 50 65 40

Red Black Tree Adjust color 30 15 70 10 20 85 60 80 90 50 65 40

Red Black Tree Adjust color 30 15 70 10 20 85 60 80 90 50 65 40

Red Black Tree Insert 5 30 15 70 10 20 85 60 5 80 90 50 65 40

Red Black Tree (Insertion Example) 30 15 70 10 20 85 60 5 80 90 50 65 40 55

Analysis • Go up the tree performing Case 1, which only recolors nodes. • If Case 2 or Case 3 occurs, perform 1 or 2 rotations, and terminate. Running time: O(lg n) with O(1) rotations.

Delete demo Delete 30: Swap with 40 and delete red leaf. 30 15 70 10 20 85 60 5 80 90 50 65 40 55

Delete demo 40 15 70 10 20 85 60 5 80 90 50 65 55

Deletion Idea: delete a node and then try to fix the violation of red-black properties. 9 10 7 7 α β 14 9.5 γ δ ε ζ

Deletion z 9 10 7 7 14 9.5 9.7 9 will be deleted. What is x? RB-Delete(T,z) 1. if z→left=nil[T] or z→right=nil[T] 2. then y ← z 3. else y ← Successor(z) 4. if y→left≠nil[T] 5. then x ← y→left 6. else x ← y→right 7.x→p ← y→p 8. if y→p = nil[T] 9. then root[T] ← x 10. else if y = (y→p)→left 11. then (y→p)→left ← x 12. else (y→p)→right ← x 13.if y≠z 14. then key[z] ← key[y] 15. copy y’s data into z 16.if color[y] = BLACK then RB-Deletex-Fixup(T,x) return y Deletion z 9 10 7 7 14 9.5 9.7 9 will be deleted. What is x? what is y?

Deletion z 9 10 7 7 y 14 9.5 9.7 x 9 will be deleted. What is x? 9.7! RB-Delete(T,z) 1. if z→left=nil[T] or z→right=nil[T] 2. then y ← z 3. else y ← Successor(z) 4. if y→left≠nil[T] 5. then x ← y→left 6. else x ← y→right 7.x→p ← y→p 8. if y→p = nil[T] 9. then root[T] ← x 10. else if y = (y→p)→left 11. then (y→p)→left ← x 12. else (y→p)→right ← x 13.if y≠z 14. then key[z] ← key[y] 15. copy y’s data into z 16.if color[y] = BLACK then RB-Deletex-Fixup(T,x) return y Deletion z 9 10 7 7 y 14 9.5 9.7 x 9 will be deleted. What is x? 9.7! what is y? 9.5!

Deletion y 9.5 10 7 7 14 9.7 x After 9 is deleted. RB-Delete(T,z) 1. if z→left=nil[T] or z→right=nil[T] 2. then y ← z 3. else y ← Successor(z) 4. if y→left≠nil[T] 5. then x ← y→left 6. else x ← y→right 7.x→p ← y→p 8. if y→p = nil[T] 9. then root[T] ← x 10. else if y = (y→p)→left 11. then (y→p)→left ← x 12. else (y→p)→right ← x 13.if y≠z 14. then key[z] ← key[y] 15. copy y’s data into z 16.if color[y] = BLACK then RB-Deletex-Fixup(T,x) return y Deletion y 9.5 10 7 7 14 9.7 x After 9 is deleted.

Deletion When the spliced-out (old) y is BLACK, we could have 3 problems: (1)If y was the root and a red child of y becomes the new root,… (2)If both x and p[y] (which is now p[x]) were both red, as in the left figure,… (3)y’s removal causes any path previously contained y to have one fewer black node,… y 9.5 10 7 7 14 9.7 x After 9 is deleted.

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 1. x’s sibling, w, is red

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 1. x’s sibling, w, is red

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 1. x’s sibling, w, is red

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 10 14 9.5 new w 9.7 ε ζ 7 7 x α γ β δ Case 1. x’s sibling, w, is red

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 2. x’s sibling w is black, so are w’s children

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup new x 9.5 w 10 7 7 α β 14 9.7 γ δ ε ζ Case 2. x’s sibling w is black, so are w’s children

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 3. x’s sibling w is black, w’s left child is red, right child is black

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 3. x’s sibling w is black, w’s left child is red, right child is black

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 new w 9.7 7 7 x 10 γ α β 14 δ ε ζ Case 3. x’s sibling w is black, w’s left child is red, right child is black

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 4. x’s sibling w is black, w’s right child is red

While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup 9.5 w 10 7 7 x α β 14 9.7 γ δ ε ζ Case 4. x’s sibling w is black, w’s right child is red

The Fixup ε ζ γ δ α β new x =root, why? 10 14 9.5 9.7 7 7 While x≠root[T] and color[x]=BLACK do if x = left[p[x]] then w ← right[p[x]] if color[w]=RED then color[w] ← BLACK color[p[x]] ← RED Left-Rotation(T,p[x]) w ← right[p[x]] if color[left[w]]=BLACK and color[right[w]]=BLACK then color[w] ← RED x ← p[x] else if color[right[w]]=BLACK then color[left[w]] ← BLACK color[w] ← RED Right-Rotation(T,w) color[w] ← color[p[x]], color[p[x]] ← BLACK color[right[w]] ← BLACK x ← root[T] else (same as then with “right” and “left” exchanged color[x] ←BLACK The Fixup new x =root, why? 10 14 9.5 ε ζ 9.7 7 7 γ δ α β Case 4. x’s sibling w is black, w’s right child is red