1. Trees Data Structures and Algorithms (60-254)

2. Recursive Definition of a Tree A tree T is either empty or it consists of a root and zero or more nonempty subtrees T 1,T 2,…T k each of whose roots are connected by an edge from the root of T. 2 … nodes, vertices arcs, edges

3. The terminology of trees 3 B HGF EDC K JI A: Root of the tree B,C,D,E: Children of the root B,C,D,E: Sibblings E is an ancestor of K K is a descendant of E F,G,C,H,I,K are the leaves of the tree The depth of a node is the length of the path from the root to the node Length of path = # of edges Depth of E =1 Height of a node = length of the longest path from that node to a leaf Height of E = 2 Height of tree = height of root Height of example tree is 3 A

4. Applications Trees are a ubiquitous data structure in Computer Science. The Unix directory structure is a tree. Allows user to store files in a logical/organized fashion. Issues: How to implement tree structure How to insert and delete nodes in a tree structure How to find nodes in a tree structure How to traverse the entire tree structure How to organize structure for efficiency These issues will be at the core of much of the discussion of trees. 4

5. Binary Trees 5 A B ED C H GF Root T Right T Left A binary tree T is either empty or it consists of a root and two (possibly empty) binary subtrees T Left and T Right each of whose roots are connected by an edge from the root of T. If the left (right) binary subtree is not empty, its root is called the left (right) child of the root of the tree. The degree of a node is the number of children it has.

6. Problem Show by induction that the number of degree 2 nodes (n T 2) in any binary tree is 1 less than the number of leaves (n T 0). Proof: The induction is on the number of nodes n in the binary tree. For n = 1, there is only the root node with 0 children. Hence, 1 leaf node and 0 nodes of degree 2. Suppose the claim is true for all binary trees with n or less nodes. Consider a binary tree with n+1 nodes. It has one of the following structures: Let n L and n R be the number of nodes in the left and right subtrees respectively. 6

7. Proof - continued 7 Since n L  n and n R  n, the claim is true for the left subtree in case (i) and the right subtree in case (ii). Since in both these cases the degree 2 nodes and the leaves belong to these subtrees alone, the claim holds for the entire tree. In case (iii), the claim is true for both subtrees. If we add the root to the count of degree 2 nodes of these trees, the claim follows.

8. Exercise Show by induction that a binary tree with n nodes has height at least  log n . 8

9. Operations that we would like to do on a binary tree Find its height Find its size (number of nodes) Visit its nodes in a systematic way Etc. 9

10. Height of a binary tree Height(T) = 1 + max(Height(T L ),Height(T R )) Height(  ) = -1 Write a recursive function based on the above definition. height(T) if (T is empty) return –1 else return 1 + max(height(T.left),height(T.right)) 10

11. Size of a binary tree Size(  ) = 0 Size(T) = 1 + Size(T L ) + Size(T R ) Write a recursive function based on the above definition. size(T) if (T is empty) return 0 else return 1 + size(T.left) + size(T.right) 11

12. Tree traversals Many problems require that we visit the nodes of a binary tree structure in a systematic way. Three such ways are: 1.Preorder traversal 2.Postorder traversal 3.Inorder traveral 12

13. Preorder traversal 1.Visit the root 2.Visit the left subtree in preorder 3.Visit the right subtree in preorder 13 For example, printing the labels of the nodes in the given tree using a preorder traversal gives: A,B,D,E,C,F,G,H

14. Postorder traversal 1.Visit the left subtree in postorder 2.Visit the right subtree in postorder 3.Visit the root For example, printing the labels of the nodes in the given tree using a postorder traversal gives: D,E,B,F,H,G,C,A 14

15. Inorder traversal 1.Visit the left subtree in inorder 2.Visit the root 3.Visit the right subtree in inorder For example, printing the labels of the nodes in the given tree using an inorder traversal gives: D,B,E,A,F,C,H,G 15

16. Printing printPreorder(T) print T.data// visit root if (T.left is not empty) printPreorder(T.left)// visit left if (T.right is not empty) printPreorder(T.right)// visit right printPostorder(T) if (T.left is not empty) printPostorder(T.left)// visit left if (T.right is not empty) printPostorder(T.right)// visit right print T.data// visit root printInorder(T) if (T.left is not empty) printInorder(T.left)// visit left print T.data// visit root if (T.right is not empty) printInorder(T.right)// visit right 16

17. Heapsort We can define another sorting algorithm that uses a binary tree. Definition: An (essentially) complete binary tree is a binary tree that is complete at all levels except possibly the last. Examples: Definition: A heap is an essentially complete binary tree with the additional property: The key stored at any node is  the keys stored at its children, if any. 17

18. Replacing  by  Example: Array Representation of a heap: Key values placed in array level-by-level and left to right. Children of node at index i placed at indices 2i+1 and 2i+2. Given, an array of n elements, how can we heapify them? 18

19. Algorithm Heapify Input: Indices i and j. An array A[0..n-1] of key values. Output: A[i] sinks to appropriate place in A[i..j] Procedure Heapify(i, j) if A[i] is a leaf node then return Set A[k]  max{A[2i+1],A[2i+2]} // k is index of max. where 2i+1 and 2i+2  j if (A[i] < A[k]) then Exchange(A[i],A[k]) Heapify(k, j) 19

20. Example 20 Heapify(1,6): causes 10 to sink Heapify(0,6): causes 12 to sink

21. Example 21 Heapify(2,6): causes 12 to continue sinking

22. Heapify and BuildHeap Heapify(0,n-1) is O(log n) Heapify(i,n-1) only fixes element i To build the entire heap, we must apply heapify to all of the elements in turn. 22

23. Algorithm BuildHeap Input: An array A[0..n-1] of key values. Output: An array A[0..n-1] that satisfies the heap property. Procedure BuildHeap(A) for i  n div 2 - 1 downto 0 Heapify(i, n-1) Initial array: 23 i: 7 div 2 – 1 = 2 BuildHeap -Heapify(2,6): no change

24. BuildHeap Example 24 Heapify(1,6): causes 10 to sink Heapify(2,6): causes 12 to continue sinking

25. Algorithm Heapsort Input: An array A[0..n-1] of key-values. Output: A sorted array A[0..n-1] of key-values. BuildHeap(A) for i  n-1 downto 1 Exchange(A[0],A[i]) Heapify(0,i-1) 25

26. Example 26

27. Example Continued 27

28. 28

29. 29

30. 30

31. Complexity of Heapsort BuildHeap is O(n) : True but difficult to prove!! Heapify is O(log n) Heapify is called n times in heapsort. Therefore, Heapsort is O(n log n) 31

32. Binary Search Trees Definition: A Binary Search Tree (BST) is a binary tree that satisfies the property that for every node x in the BST, x.Key > every key in x.Left (left subtree), and x.Key < every key in x.Right (right subtree). A node of a binary search tree may be represented with 5 fields: For simplicity, we usually omit the data field in examples. Also, the parent field can often be omitted depending on the intended application and/or the particular implementation. 32

33. An example of a BST BST property ⇒ Inorder traversal of a BST visits the nodes in sorted order: 2,3,4,5,7,8 (ascending order) 33

34. 1.Searching for a node with a given key, k: TreeSearch(x,k) // x is the node where search starts if (x == NULL or k == x.Key) return x else if (k < x.Key) return TreeSearch(x.Left,k) else return TreeSearch(x.Right,k) 34

35. 2. Minimum and Maximum keys TreeMin(x) while (x.Left != NULL) x = x.Left return x; TreeMax(x) while (x.Right != NULL) x = x.Right return x; 35

36. 3. Successor of a key Definition: The successor of a node x is the node with the smallest key greater than x.Key TreeSuccessor(x) if (x.Right != NULL) return TreeMin(x.Right) else p = x.Parent while (p != NULL and x == p.Right) x = p p = x.Parent return p 36

37. Example: Successor of 13? Note that 13 is the maximum key in the BST rooted at 6 37

38. 4. Predecessor of a key Definition: The predecessor of a node x is the node with thelargest key smaller than x.Key. TreePredecessor(x) if (x.Left != NULL) return TreeMax(x.Left) else p = x.Parent while (p != NULL and x == p.Left) x = p p = x.Parent return p 38

39. Example: Predecessor of 13? Note that 13 is the minimum key in the BST rooted at 16 39

40. Insertion Node TreeInsert(x, k) if (x == NULL) Create new node t with key k else if (k < x.Key) x.Left ← TreeInsert(x.Left, k) else if (k > x.Key) x.Right ← TreeInsert(x.Right, k) else Throw exception “Existing key!” return t 40

41. Example: Insert key 13 into existing BST 41

42. Deletion Assumption: The node to be deleted is given. Case 1: Node has no children Simply remove the node. Example: Delete 13 42

43. Deletion Case 2: Node has one child Replace node with its child. Example: Delete 13 43

44. Deletion Case 3: Node has two children Replace node with minimum key m in right subtree and recursively delete m in right subtree. Example: Delete 13 44

45. Balanced Search Trees Definition: Binary Search Trees with a worst-case height of O(log n) are called balanced trees We can guarantee O(log n) performance for each search tree operation for balanced trees. We will discuss two kinds of balanced trees (a) AVL trees (b) Red-black trees 45

46. AVL Trees Named after its discoverers: Adelson-Velskii and Landis, 1962 Definition: An empty binary tree is an AVL tree. If T is a non-empty binary tree with T L and T R as its left and right sub-trees, then T is an AVL tree iff (a) T L and T R are AVL trees and (b) | h L − h R | ≤ 1 where h L and h R are the heights of T L and T R respectively. 46

47. AVL search tree Definition: An AVL search tree is a binary search tree that is also an AVL tree Henceforth, by an AVL tree we shall mean an AVL search tree. balance = height of left subtree – height of right subtree Node of an AVL tree: 47

48. Examples of AVL search trees 48 Balance factor = left balance– right balance

49. Height of an AVL tree Let N h be the minimum number of nodes in an AVL tree of height h. ThenN h = N h−1 + N h−2 +1, N 0 =1, N 1 =2 assuming the worst-case scenario in which one sub-tree has height h-1 and the other h-2 The solution to the recurrence given above is: N h = F h+3 -1, where F n is the n th Fibonacci number 49

50. 50

51. Insertion into an AVL tree Inserting key 32 causes the tree to become unbalanced 51

52. Imbalance due to: Insertion into left sub-tree of left child of a node 52

53. Balanced by LL-rotation 53

54. Tree balanced by LL rotation 54

55. Pseudocode: LL rotation Input: Binary Node k2 Procedure withLeftChild( k2) Set k1 ← k2.Left if (k2 is Left child) k2.Parent.Left ← k1 else k2.Parent.Right ← k1 k2.Left ← k1.Right k1.Right ← k2 // Also, update Parent fields! return k1 55

56. Imbalance due to Insertion into right sub-tree of right child of a node 56

57. Balanced by RR-rotation 57

58. Example of RR rotation: Insert 45 Tree unbalanced after insertion 58

59. After the RR rotation: 59

60. Pseudocode: RR rotation Input: Binary Node k1 Procedure withRightChild( k1) Set k2 ← k1.Right if (k1 is Left child) k1.Parent.Left ← k2 else k2.Parent.Right ← k2 k1.Right ← k2.Left k2.Left ← k1 // Update Parent fields too! return k2 60

61. Imbalance caused by Insertion into right sub-tree of the left child of a node 61

62. Balanced by LR-rotation 62

63. Example of LR rotation: 63

64. Remark: An LR rotation is often called a double rotation because it can be viewed as an RR rotation, followed by an LL rotation On the example: After the RR-rotation 64

65. After the LL-rotation 65

66. Imbalance caused by Insertion into left sub-tree of right child Balance is restored by an RL rotation Details are the same as in the case of a LR rotation 66

67. An example of RL rotation: 67

68. After the RL rotation: 68

69. 69

70. Pseudocode: LR rotation Input: Binary Node k3 Procedure doubleWithLeftChild( k3) Set k3.Left ← withRightChild(k3.Left) return withLeftChild(k3) 70

71. Pseudocode: RL rotation Input: Binary Node k3 Procedure doubleWithRightChild( k3) Set k3.Right ← withLeftChild(k3.Right) return withRightChild(k3) 71

72. Balance: 72

73. Balance: 73

74. 74

75. Deletion from an AVL tree We delete a node just as in a binary search tree Let q be the parent of the physically deleted node. We might need to go all the way from q to the root in order to balance the tree and restore AVL-balance 75

76. Three cases arise: C1: Balance at q has changed to 0 ⇒ height has decreased by 1 (of the tree rooted at q) and we need to check the balance factor of the ancestors of q. C2: Balance at q has changed to +1 or –1 ⇒ height of subtrees at q is unchanged, so the balances of all the ancestors of q remain unchanged C3: Balance at q is ±2 ⇒ tree needs to be balanced at q 76

77. Three cases: example Case C1, delete root Case C2, delete node circled in green Case C3, delete node circled in blue 77

78. Types of Imbalance Let A be the first node on the path from q to the root where the balance has changed to ±2 The imbalance is of type L if deletion has taken from the left sub-tree of A R if deletion has taken from the right sub-tree of A We discuss the R-type imbalance 78

79. 79

80. 80

81. 81

82. 82

83. 83

84. 84

85. 85

86. A Red-Black Tree 86

87. 87

88. 88

89. Important conclusion For a red-black tree with n nodes 89

90. Insertion into a red-black tree (Method 1) Insert node as a leaf Restore properties P1-P4, bottom-up OK? Observations Inserted node must be colored red Red-black property P3 is violated if it has a red parent 6 cases. We’ll study 3 of them: Inserted node is in the “left” subtree of G. 90

91. Case 1: Inserted node has a red parent, which has a black sibling Sub-case 1.1: X is an outer child 91 Single rotation restoration (LL)

92. Sub-case 1.2: X is an inner child Double rotation restoration (LR) 92

93. Case 2: Inserted node has a red parent, which has a red sibling. 93 Single rotation restoration: LL

94. More on Case 2 This case is problematic because: P could have a red parent, and we need to propagate the restoration up to the root 94

95. 95

96. 96

97. 97

98. 98

99. Top-down insertion Rule: Walk down the tree (starting from root). If we hit a node X with (two) red children, then flip the color of X and its children. Claim. If the parent of X is red, Then X’s parent’s sibling cannot be red, and the single rotation (LL) or the double rotation (LR) can be applied. 99

100. 100

101. 101

102. 102

103. Deletion from a red-black tree Deletion is as the case of a binary search tree, but we need to guarantee that the color of the physically deleted node is red Start from the root… As we search down the tree, we maintain the invariant that the color of the current node X is red 103

104. 104

105. Let X be the current node Let P be its parent Let T be its sibling Always ensure that, P is red ⇒ both X and T are black … why? Two cases: Case 1: X has 2 black children 105

106. Case 1.1: T has 2 black children 106

107. Case 1.2: T has an outer child that is red 107

108. Case 1.3: T has an inner red child 108

109. Case 2: X has a red child 109

110. Deletion from a red-black tree Note that the rotation is the main case when X has two black children, always when X red. When X has a red child we have to move one level down on the search path (whether the child along the search path is red, the rotation is both). Note that in Case 1.2, if both children of T are red we can still apply the case. 110