1. Binary Search Trees Cmput 115 - Lecture 23 Department of Computing Science University of Alberta ©Duane Szafron 2000 Some code in this lecture is based on code from the book: Java Structures by Duane A. Bailey or the companion structure package Revised 3/19/00

2. ©Duane Szafron 2000 2 LAB EXAM NEXT WEEK! Start on Monday! Please read information on CMPUT 115 website.

3. ©Duane Szafron 2000 3 About This Lecture In this lecture we study an implementation of OrderedStructure called the BinarySearchTree. NOTE: We will not be discussing Huffman coding, priority queues, or heap sort.

4. ©Duane Szafron 2000 4Outline Revisit OrderedStructure Binary Search Tree Definition Binary Search Tree Implementation Unbalanced and Balanced Binary Search Trees

5. ©Duane Szafron 2000 5 Re-visiting OrderedStructures We have two implementations of the OrderedStructure interface: OrderedVector and OrderedList. For OrderedVector –The time for finding an element is O(log n) comparisons due to a binary search. –The time for adding or removing an element is O(log n) comparisons to find it and O(n) assignments to “move” existing elements to the right of it. For OrderedList –The time for finding an element is O(n) comparisons due to a sequential search. –The time for adding or removing an element is O(n) comparisons to find it and O(C) assignments to fix links.

6. ©Duane Szafron 2000 6 Binary Search Tree We want to combine the advantage of a binary search with the advantage of just fixing a few links during adding and removal to obtain an implementation called a BinarySearchTree where: –The time for finding an element is O(log n) comparisons due to a binary search. –The time for adding or removing an element is O(log n) comparisons to find it and O(C) assignments to fix links. A binary search tree (BST) is a binary tree in which every element is greater than or equal to all elements in the left sub-tree of the node containing that element and is less than or equal to all elements in the right sub-tree of the node containing that element.

7. ©Duane Szafron 2000 7 Sort Order in BinarySearchTrees Many different BSTs can be formed from the same set of elements. 40 20 1030 60 5070 50 1060 30 2040 70 50 4060 20 1030 40 10 30 20 50 70 60 However, an inorder traversal always produces the elements in sorted order: 10, 20, 30, 40, 50, 60, 70.

8. ©Duane Szafron 2000 8 OrderedStructure Hierarchy The structure package adds BinarySearchTree as another implementation of OrderedStructure. Store Collection OrderedStructure OrderedVectorOrderedListBinarySearchTree

9. ©Duane Szafron 2000 9 Structure Interface - Store public interface Store { public int size(); //post: returns the number of elements contained in // the store. public boolean isEmpty(); // post: returns the true iff store is empty. public void clear(); // post: clears the store so that it contains no // elements. } code based on Bailey pg. 18

10. ©Duane Szafron 2000 10 Structure Interface - Collection public interface Collection extends Store { public boolean contains(Object anObject); // pre: anObject is non-null // post: returns true iff the collection contains the object public void add(Object anObject); // pre: anObject is non-null // post: the object is added to the collection. The // replacement policy is not specified public Object remove(Object anObject); // pre: anObject is non-null // post: removes object “equal” to anObject and returns it, // otherwise returns nil public Iterator elements(); // post: return an iterator for traversing the collection } code based on Bailey pg. 19

11. ©Duane Szafron 2000 11 Structure Interface - OrderedStructure public interface OrderedStructure extends Collection { } code based on Bailey pg. 173

12. ©Duane Szafron 2000 12 Recall OrderedStructure Example public static void main(String[ ] args) { OrderedStructure container; RandomInt generator; int index; Iterator iterator; container = new BinarySearchTree(); generator = new RandomInt(1); for (index = 0; index < 100; index++) { container.add(new Integer(generator.next(100))); iterator = container.elements(); while(iterator.hasMoreElements()) System.out.print(iterator.nextElement() + ‘ ‘); } code based on Bailey pg. 158 1 1 1 2 2 3 3 3 4 4 5 7 7 7 8 9 9 10 13 13 14 14 15 17... indexOf previousOf

13. ©Duane Szafron 2000 13 Implementation of BST The implementation of a BinarySearchTree parallels the implementation of a BinaryTree. However, addition and removal of elements is different since we don’t provide the user with a cursor to specify location. Instead the element must be added or removed at the correct location at the discretion of our implementation based on the order of the node.

14. ©Duane Szafron 2000 14 BST - State and Constructor class BinarySearchTree implements OrderedStructure { protected BinaryTreeNode root; protected int count; public BinarySearchTree() { // post: constructs an empty binary search tree. this.root = null; this.count = 0; } code based on Bailey pg. 252

15. ©Duane Szafron 2000 15 BST - Store Interface /* Interface Store Methods */ public int size() { //post: returns the number of elements in the store. return this.count; } public boolean isEmpty() { // post: returns the true iff store is empty. return this.size() == 0; // return this.root == null } public void clear(); // post: clears the store so that it contains no elements. this.root = null; this.count = 0; } code based on Bailey SPackage

16. ©Duane Szafron 2000 16 Searching for an Element in a BST To implement contains(Object), add(Object) and remove(Object) we must first search for the object we are looking for. In contains(Object), we are done when we either find Object or ensure it is not in the tree. In add(Object), the search determines not only whether Object is there or not, but the specific location where it belongs. We can use this information to add it. In remove(Object), the search determines not only whether it is there or not, but the specific location where it is, since we need this information to remove it.

17. ©Duane Szafron 2000 17 The Location of an Element in a BST We use a locate method that returns either the node that contains the element or the parent node of the node where the element should be added as a new leaf: –Compare the element to the root element. –If they are equal return the root node. –If element > root then recursively search the right sub-tree. –Otherwise, recursively search the left sub-tree. –If the sub-tree to be searched is null, return the root. Search for 30: 50, 20, 30, return 30 Search for 5 or 15: 50, 20, 10, return 10 Search for 25: 50, 20, 30, return 30 Search for 75: 50, 70, return 70 50 20 1030 70 60 40

18. ©Duane Szafron 2000 18 BST - locate(BinaryTreeNode, Object) 1 /* Protected Methods */ protected BinaryTreeNode locate(BinaryTreeNode root, Object anObject) { // pre: root and anObject are non-null // post: returns an existing tree node containing an Object // equal to the given one or the parent node where the // object should be inserted as a child if not found. BinaryTreeNode child; int compare; rootElement = (Comparable) this.root.value(); compare = anObject.compareTo((Comparable)this.root.value()); if (compare == 0) return root; code based on Bailey pg. 252

19. ©Duane Szafron 2000 19 BST - locate(BTN, Object) else if (compare < 0) child = root.left(); else child = root.right(); if (child == null) return root; else return this.locate(child, anObject); } code based on Bailey pg. 252

20. ©Duane Szafron 2000 20 BST - contains(Object) /* Interface Collection Methods */ public boolean contains(Object anObject) { // pre: anObject is non-null // post: returns true iff the collection contains the object BinaryTreeNode possibleNode; if (this.root == null) return false; possibleNode = this.locate(this.root,(Comparable)anObject); return anObject.equals(possibleNode.value()); } code based on Bailey pg. 253

21. ©Duane Szafron 2000 21 Adding an Element to a BST An element is always added in a new leaf node. We start with a call to locate(BTN, Object). There are two cases: –The element is not already in the tree. Insert it as the appropriate child of the parent node returned from locate. –The element is already in the tree. If the left child of that node is empty, insert it there. If it is full, insert it as the right child of the predecessor of the found node. 50 20 1030 70 60 40 Add 5: locate 10, insert left Add 15: locate 10, insert right Add 75: locate 70, insert right Add 30: locate 30, insert left Add 50: locate 50, find predecessor 40, insert right 5 50 1575 30

22. ©Duane Szafron 2000 22 Finding the Predecessor in a BST To find the predecessor of an element in a Binary Search Tree that has a left child: –Go left once –Go right until there is no right child. 50 20 1030 70 60 40 Predecessor of 50: left 20, right 30, right 40 Predecessor of 70: left 60 Predecessor of 30: no left child, can’t use this algorithm

23. ©Duane Szafron 2000 23 BST - predecessor(BinaryTreeNode) /* Protected Methods */ protected BinaryTreeNode predecessor(BinaryTreeNode root) { // pre: root is non-null and root has a left child // post: returns node whose element is just before the root // node in the order. BinaryTreeNode predecessor; predecessor = root.left(); while (predecessor.right() != null) predecessor = predecessor.right(); return predecessor; } code based on Bailey pg. 254

24. ©Duane Szafron 2000 24 5 15 Alternative Strategy for Add The previous strategy always inserts duplicate elements in the right sub-tree. Alternatively, we could always insert duplicate elements in the left sub-tree by finding the successor of the found node and inserting to the left. 50 20 1030 70 60 40 Add 5: locate 10, insert left Add 15: locate 10, insert right Add 75: locate 70, insert right Add 30: locate 30, find successor 40, insert left Add 50: locate 50, find successor 60, insert left 75 3050

25. ©Duane Szafron 2000 25 BST - add(Object) 1 public void add(Object anObject); // pre: anObject is non-null // post: the object is added at the appropriate position // based on comparing it to the other elements. BinaryTreeNode newNode; BinaryTreeNode insertionNode; Comparable newElement; Comparable nodeElement; newNode = new BinaryTreeNode(val); // if there's no root, add element at root. if (this.root == null) this.root = newNode; else { newElement = (Comparable) anObject; insertionNode = locate(root, newElement); code based on Bailey pg. 253

26. ©Duane Szafron 2000 26 BST - add(Object) nodeElement = (Comparable) insertionNode.value(); if (nodeElement.compareTo(newElement) < 0) insertionNode.setRight(newNode); else if (insertionNode.left() != null) // element is already in tree and left node is not open this.predecessor(insertionNode).setRight(newNode); else insertionNode.setLeft(newNode); } count++; } code based on Bailey pg. 253 See slide 23 for implementation of predecessor

27. ©Duane Szafron 2000 27 Removing from a BST - easy To remove an element in a leaf node, simply remove the leaf node. If the element is in a node that is not a leaf node it has one or two sub-trees. If it has one sub-tree, remove the node and replace it by its single sub-tree. 50 B AA B A B A B

28. ©Duane Szafron 2000 28 Removing from a BST - harder If the element is in a node with two sub-trees, but the left child has no right sub-tree, remove the node containing the element and replace it by the left child. 50 A B C 40 A BC

29. ©Duane Szafron 2000 29 Removing from a BST - hardest 1 If the element is in a node with two sub-trees, and the left child has a right sub-tree, find the predecessor of the node containing the element to be removed, by going left once and then right as many times as possible. Note that the predecessor cannot have a right sub-tree or it would not have been the predecessor. 50 A B C 20 30 D 40 E

30. ©Duane Szafron 2000 30 Removing from a BST - hardest 2 Replace the predecessor node by its left sub-tree. Remove the node containing the element and replace it by the predecessor node. 50 A B C 20 30 D 40 E A B C 20 30 DE

31. ©Duane Szafron 2000 31 BST - remove(Object) 1 public Object remove(Object anObject); // pre: anObject is non-null // post: removes object “equal” to anObject and returns it, // otherwise returns nil Comparable element; BinaryTreeNode newRoot; BinaryTreeNode removeNode; Object result; element = (Comparable)anObject; if (this.isEmpty()) return null; if (element.equals(this.root.value())) { newroot = removeTop(this.root); count--; result = this.root.value(); this.root = newroot; return result; } code based on Bailey SPackage

32. ©Duane Szafron 2000 32 BST - remove(Object) 2 removeNode = locate(root, element); if (element.equals(removeNode.value())) { count--; parent = removeNode.parent(); if (parent.right() == removeNode) parent.setRight(removeTop(removeNode)); else parent.setLeft(removeTop(removeNode)); return removeNode.value(); } return null; } code based on Bailey SPackage

33. ©Duane Szafron 2000 33 BST - removeTop(BTN) 1 protected BinaryTreeNode removeTop(BinaryTreeNode topNode) { // pre: tree is not empty. // post: root of tree (topNode) is disconnected from tree // and a new root is returned, new root has no parent. BinaryTreeNode left; BinaryTreeNode right; BinaryTreeNode predecessor; BinaryTreeNode parent; left = topNode.left(); right = topNode.right(); // disconnect top node topNode.setLeft(null); topNode.setRight(null); // Easy case: only one sub-tree if (left == null) return right; code based on Bailey pg. 256

34. ©Duane Szafron 2000 34 BST - removeTop(BTN) 2 if (right == null) return left; predecessor = left.right(); // Harder case: left node has no right sub-tree if (predecessor == null) { left.setRight(right); return left; } // Hardest case: left node has a right sub-tree parent = left; while (predecessor.right() != null) { parent = predecessor; predecessor = predecessor.right(); } parent.setRight(predecessor.left()); predecessor.setLeft(left); predecessor.setRight(right); return predecessor; } code based on Bailey pg. 256

35. ©Duane Szafron 2000 35 BST - elements() public Iterator elements(); // post: return an iterator for traversing the collection return new BRInorderIterator (this.root); } code based on Bailey pg. 257

36. ©Duane Szafron 2000 36 Efficiency of Binary Search Trees Each of the time consuming operations is O(h), where h is the height of the tree. If the tree is fairly balanced, h = log n so the operations are O(log n). However, since many binary trees can be made from the same elements, we could be unlucky when we make our tree and have a tree with height h = n, which is essentially a linked list and the search time is O(n). What insertion orders cause problems?

37. ©Duane Szafron 2000 37 Unbalanced Binary Search Trees 1 Look at a tree with insertion order: 10, 20, 30, 40: if (nodeElement.compareTo(newElement) < 0) insertionNode.setRight(newNode); else if (insertionNode.left() != null) // element is already in tree and left node is not open this.predecessor(insertionNode).setRight(newNode); else insertionNode.setLeft(newNode); 10 20 10 30 20 10 40 30 20 10

38. ©Duane Szafron 2000 38 Unbalanced Binary Search Trees 2 40 30 40 20 30 40 10 20 30 40 Look at a tree with insertion order: 40, 30, 20, 10: if (nodeElement.compareTo(newElement) < 0) insertionNode.setRight(newNode); else if (insertionNode.left() != null) // element is already in tree and left node is not open this.predecessor(insertionNode).setRight(newNode); else insertionNode.setLeft(newNode);

39. ©Duane Szafron 2000 39 Balanced Binary Search Trees Look at a tree with insertion order: 30, 20, 40, 10, 30: if (nodeElement.compareTo(newElement) < 0) insertionNode.setRight(newNode); else if (insertionNode.left() != null) // element is already in tree and left node is not open this.predecessor(insertionNode).setRight(newNode); else insertionNode.setLeft(newNode); 30 20 30 40 10 20 30 4020 30 10 4020 30