1. Balanced Search Trees CS 3110 Fall 2010

2. Some Search Structures Sorted Arrays –Advantages Search in O(log n) time (binary search) –Disadvantages Need to know size in advance Insertion, deletion O(n) – need to shift elements Lists –Advantages No need to know size in advance Insertion, deletion O(1) (not counting search time) –Disadvantages Search is O(n), even if list is sorted

3. Balanced Search Trees Best of both! –Search, insert, delete in O(log n) time –No need to know size in advance Several flavors –AVL trees, 2-3 trees, red-black trees, skip lists, random treaps,...

4. Review – Binary Search Trees Every node has a left child, a right child, both, or neither Data elements are drawn from a totally ordered set Every node contains one data element Data elements are ordered in inorder

5. A Binary Search Tree 25 476 1 2029 80 5491 48 13

6. Binary Search Trees In any subtree: all elements smaller than the element at the root are in the left subtree all elements larger than the element at the root are in the right subtree 25 476 1 2029 80 5491 48 13

7. Search To search for an element x: if tree is empty, return false if x = object at root, return true If x < object at root, search left subtree If x > object at root, search right subtree

8. Search 25 476 1 2029 80 5491 48 13 Example: search for 13

9. 25 Search 476 1 2029 80 5491 48 13 13 ?

10. 6 Search 25 47 1 2029 80 5491 48 13 13 ?

11. 20 Search 25 476 1 29 80 5491 48 13 13 ?

12. 13 Search 25 476 1 2029 80 5491 48 13 ?

13. 13 Search 25 476 1 2029 80 5491 48

14. Search type 'a tree = Node of 'a * 'a tree * 'a tree | Leaf let rec contains (t :'a tree) (x :'a) : bool = match t with Leaf -> false | Node (y, l, r) -> if x = y then true else if x < y then contains l x else contains r x

15. Insertion To insert an element x: search for x – if there, just return when arrive at a leaf y, make x a child of y –left child if x < y –right child if x > y

16. Insertion 25 476 1 2029 80 5491 48 13 Example: insert 15

17. 25 Insertion 476 1 2029 80 5491 48 13 15 ?

18. 6 Insertion 25 47 1 2029 80 5491 48 13 15 ?

19. 20 Insertion 25 476 1 29 80 5491 48 13 15 ?

20. 13 Insertion 25 476 1 2029 80 5491 48 15 ?

21. 15 13 Insertion 25 476 1 2029 80 5491 48

22. Insertion let rec insert (x : 'a) (t : 'a tree) : 'a tree = match t with Leaf -> Node (x, Leaf, Leaf) (* if at a leaf, put new node there *) | Node (y, l, r) as t -> (* recursively search for insert point *) if x = y then t else if x > y then Node (y, l, insert x r) else (* x < y *) Node (y, insert x l, r)

23. Deletion To delete an element x: remove x from its node – this creates a hole if the node was a leaf, just delete it find greatest y less than x in the left subtree (or least y greater than x in the right subtree), move it to x's node this creates a hole where y was – repeat

24. Deletion To find least y greater than x: follow left children as far as possible in right subtree 19 25 47615423331 27 22

25. Deletion To find greatest y less than x: follow right children as far as possible in left subtree 19 25 47615423331 27 22

26. Deletion 25 476 1 2029 80 5491 48 13 Example: delete 25

27. 25 Deletion 476 1 2029 80 5491 48 13

28. Deletion 476 1 2029 80 5491 48 13

29. Deletion 476 1 2029 80 5491 48 13

30. Deletion 476 1 2029 80 5491 48 13

31. Deletion 476 1 20 29 80 5491 48 13

32. Deletion 476 1 20 29 80 5491 48 13

33. Deletion 476 1 20 29 80 5491 48 13

34. Deletion 476 1 20 29 80 5491 48 13

35. Deletion 476 1 20 29 80 5491 48 13 Example: delete 47

36. Deletion 476 1 20 29 80 5491 48 13

37. Deletion 6 1 20 29 80 5491 48 13

38. Deletion 6 1 20 29 80 5491 48 13

39. Deletion 6 1 20 29 80 5491 48 13

40. Deletion 6 1 20 29 80 5491 48 13

41. Deletion 6 1 20 29 80 5491 48 13 Example: delete 29

42. Deletion 6 1 20 29 80 5491 48 13

43. Deletion 6 1 20 80 5491 48 13

44. Deletion 6 1 20 80 5491 48 13

45. Deletion 6 1 20 80 5491 48 13

46. Deletion 6 1 20 80 5491 48 13

47. Deletion 6 1 20 80 5491 48 13

48. Deletion 6 1 20 80 5491 48 13

49. These operations take time proportional to the height of the tree (length of the longest path) O(n) if tree is not sufficiently balanced 20 8091 48 54 Bad case for search, insertion, and deletion – essentially like searching a list Observation

50. 54 Solution Try to keep the tree balanced (all paths roughly the same length) 6 1 20 80 48 91 13

51. Balanced Trees Size is exponential in height Height = log 2 (size) Search, insert, delete will be O(log n) 54 6 1 20 80 48 91 13

52. Creating a Balanced Tree Creating one from a sorted array: Find the median, place that at the root Recursively form the left subtree from the left half of the array and the right subtree from the right half of the array 54 6 1 20 80 48 13 16 20485480

53. Keeping the Tree Balanced Insertions and deletions can put tree out of balance – we may have to rebalance it Can we do this efficiently?

54. AVL Trees Adelson-Velsky and Landis, 1962 AVL Invariant: The difference in height between the left and right subtrees of any node is never more than one

55. 27 29 An AVL Tree 25 476 2 20 80 5491 48 13 Nonexistent children are considered to have height –1 Note that paths can differ in length by more than 1 (e.g., paths to 2, 48)

56. AVL Trees are Balanced The AVL invariant implies that: Size is at least exponential in height n   d, where  = (1 +  5)/2 ~ 1.618, the golden ratio! Height is at most logarithmic in size d  log n / log  ~ 1.44 log n

57. AVL Trees are Balanced To see that n   d, look at the smallest possible AVL trees of each height AVL Invariant: The difference in height between the left and right subtrees of any node is never more than one A0A0 A1A1 A2A2 A3A3

58. AVL Trees are Balanced AVL Invariant: The difference in height between the left and right subtrees of any node is never more than one A0A0 A1A1 A2A2 A3A3 A2A2 A1A1 To see that n   d, look at the smallest possible AVL trees of each height

59. AVL Trees are Balanced AVL Invariant: The difference in height between the left and right subtrees of any node is never more than one A0A0 A1A1 A2A2 A3A3 A2A2 A1A1... AdAd A d–1 A d–2 To see that n   d, look at the smallest possible AVL trees of each height

60. AVL Trees are Balanced A0A0 A1A1 A2A2 A3A3 A2A2 A1A1... AdAd A d–1 A d–2 A 0 = 1 A 1 = 2 A d = A d–1 + A d–2 + 1, d  2

61. AVL Trees are Balanced A 0 = 1 A 1 = 2 A d = A d–1 + A d–2 + 1, d  2 1 2 4 7 12 20 33 54 88...

62. AVL Trees are Balanced A 0 = 1 A 1 = 2 A d = A d–1 + A d–2 + 1, d  2 1 2 4 7 12 20 33 54 88... 1 1 2 3 5 8 13 21 34 55... The Fibonacci sequence

63. AVL Trees are Balanced A 0 = 1 A 1 = 2 A d = A d–1 + A d–2 + 1, d  2 1 2 4 7 12 20 33 54 88... 1 1 2 3 5 8 13 21 34 55... A d = F d+2 – 1 = O(  d )

64. Rebalancing Insertion and deletion can invalidate the AVL invariant May have to rebalance

65. Rebalancing Rotation A local rebalancing operation Preserves inorder ordering of the elements The AVL invariant can be reestablished with at most O(log n) rotations up and down the tree v uw x yx yw v u rotate  AB C BC A

66. Rebalancing 27 29 25 476 2 20 80 5491 48 13 Example: delete 27

67. Rebalancing 29 25 476 2 20 80 5491 48 13

68. 80 5491 48 Rebalancing 29 25 476 2 20 13

69. 54 4880 91 Rebalancing 29 25 476 2 20 13

70. 54 4880 91 29 47 Rebalancing 25 6 2 20 13

71. 80 4891 47 54 Rebalancing 25 6 2 20 13 29

72. 2-3 Trees Another balanced tree scheme Data stored only at the leaves Ordered left-to-right All paths of the same length Every non-leaf has either 2 or 3 children Each internal node has smallest, largest element in its subtree (for searching)

73. 2-3 Trees smallest 2-3 tree of height d = 3 2 d = 8 data elements largest 2-3 tree of height d = 3 3 d = 27 data elements number of elements satisfies 2 d  n  3 d height satisfies d  log n

74. Insertion in 2-3 Trees

75. want to insert new element here

76. Insertion in 2-3 Trees

77. want to insert new element here

78. Insertion in 2-3 Trees

82. Deletion in 2-3 Trees want to delete this element

83. Deletion in 2-3 Trees

84. want to delete this element

85. Deletion in 2-3 Trees If neighbor has 3 children, borrow one

86. Deletion in 2-3 Trees If neighbor has 3 children, borrow one

87. Deletion in 2-3 Trees If neighbor has 2 children, coalesce with neighbor

88. Deletion in 2-3 Trees If neighbor has 2 children, coalesce with neighbor

89. Deletion in 2-3 Trees This may cascade up the tree!

90. Deletion in 2-3 Trees This may cascade up the tree!

91. Deletion in 2-3 Trees This may cascade up the tree!

92. Deletion in 2-3 Trees This may cascade up the tree!

93. Conclusion Balanced search trees are good Search, insert, delete in O(log n) time No need to know size in advance Several different versions –AVL trees, 2-3 trees, red-black trees, skip lists, random treaps, Huffman trees,... –find out more about them in CS4820