1. 1 COSC 2P03 Lecture #5 – Trees Part III, Heaps

2. 2 Today Take up the quiz Assignment Questions Red-Black Trees Binary Heaps Heap sort D-Heaps, Leftist Heaps Brief: Skew, Binomial, Fibonacci

3. 3 Review From Last Class Why balance trees? How to DSW balanced tree? What is the idea behind AVL balancing? What is the idea behind splay trees? When should we use AVL? Splay? What is amortized analysis?

4. 4 Red Black Trees Is really a 2-4 tree, that has been represented as a binary tree Comparing with 2-4 tree  Still has O(lgN) performance  Simpler to implement! 2 5 8 5 28

5. 5 Red-Black Trees Is a BST where  Every node is either colored “Red” or “Black”  Every null pointer is Black Meaning that every “real” node has 2 children  If a node is Red, then its children are Black So, can’t have 2 consecutive reds on a path  Every path from root to null contains the same # of Black nodes Meaning that all leaves have the same “Black Height”  The root is always black The BLACK HEIGHT of a node N is the number of Black Nodes on any path to null, not including N – but including null as black

6. 6 The Black-Height Black Height of the root? 33 Of “X” 22 X

7. 7 16 1410 8793 241 Coloring an RBT 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

8. 8 16 1410 8793 241 Coloring an RBT 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

9. 9 16 1410 8793 241 Coloring an RBT 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

10. 10 16 1410 8793 241 Coloring an RBT 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

11. 11 16 1410 8793 241 Coloring an RBT 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

12. 12 16 1410 8793 241 Coloring an RBT H = 4 B = 2 H = 2 B = 2 H = 1 B = 1 H = 1 B = 1 H = 2 B = 1 H = 3 B = 2 H = 2 B = 1 H = 1 B = 1 1.Every node is either red or black 2.Every leaf (null) is black 3.If a node is red, then both its children are black 4.All paths from node to descendants contain the same number of black nodes 5.The root is black

13. 13 Theorem 1 Any RBT (Red-Black Tree) rooted at X has n ≥ 2 bh(X) – 1 nodes  bh(X) is black height of node X Proof: (idea behind inductive proof)  Consider a node with 2 children  Each child has a bh(X) or bh(X)-1, depending on if it is red or black  Since height of child < height of X, can apply 2 bh(X) – 1 for the children node: 2 bh(X)-1 – 1  So, subtree rooted at X has at least: (2 bh(X)-1 – 1)+(2 bh(X)-1 – 1) + 1= 2 bh(X) – 1 internal nodes

14. 14 Theorem 2 In an RBT at least ½ the nodes on any path from the root to null must be Black Proof:  If there is a red node, by definition of RBT there must exist a black node  Meaning that bh(X) ≥h/2

15. 15 Theorem 3 There does not exist a path from any X to null that is more than twice as long as any other path from X to any another null Proof:  Any path from X to any null contains the same number of black nodes  From Theorem 2, we know that at least ½ of the nodes on any path are black  This means that the length of any path from X is no more than twice as long as any other

16. 16 Theorem 4 A RBT with N nodes has a height of h ≤ 2 lg(N + 1) Proof:  Let h be the height of an RBT rooted at X  From Theorem 2: bh(X) ≥ h/2  From Theorem 1: N ≥ 2 bh(X) – 1  Thus, N ≥ 2 h/2 – 1 N + 1 ≥ 2 h/2 lg(N + 1) ≥ h/2 2lg(N + 1) ≥ h  Implying that the height of an RBT is O(lgN)

17. 17 Insertion Insert a node Z uses the same insertion algorithm as BST, where Z is given the color red (unless it is the root)  Preserve root, external and depth properties  If parent of Z is black, also preserves internal property  If parent of Z is red, it causes a violation of internal property and the tree must be reorganized How to fix this?

18. 18 Fixing the Double Red Let Z be the newly inserted node, P its parent and S its sibling (uncle/aunt of Z), then there are 2 cases that may arise Case 1: S is red  Double red corresponds to an overflow  Recolor the nodes  In 2-4 tree same as a split Case 2: S is black  Double red is an incorrect placement of a 4-node  Restructure the 4-node

19. 19 Re-coloring Fixes the case where the parent node P and its sibling S are both red P and S become black and parent G of P and S becomes red, unless it is the root It is possible that this can propagate back to the grandparent of G 2 5 9 7 P S G 2 5 9 7 P S G

20. 20 Restructuring Fixes the case where the Parent (P) is red and its sibling (S) is black Same as restoring the correct location of a 4-node Restores internal property There are 4 cases to consider, depending on structure of subtree  Same as the ones used in AVL trees, but slightly modified to keep the color properties

21. 21 Ex. Restructuring 123 4 => 3 6 8 3 6 8 3 8 6 8 3 6 38 6

22. 22 Insertion Algorithm Search for location to insert node Z Add Z to the tree and color it red while doubleRed(Z) if sibling of Parent(Z) is black Z = restructure(Z) else Z = recolor(Z) Searching is O(lgN) Adding Z is O(1) At most 1 restructure: O(lgN), O(lgN) recolorings each is O(1) So, insertion is O(lgN)

23. 23 Insertion Example Insert 10, 3, 15, 20, 1 10 5 15 20 1

24. 24 Insertion Example Insert 5 10 3 15 20 1. 10 3 15 20 1 5

25. 25 Insertion Example Insert 7 10 3 15 20 1 5 Red  red parent has red child 10 3 15 20 1 5 7 Recolor 7 Color red, then fix by recoloring. 10 3 15 20 1 5 7 Recolor parent, uncle to black. Recolor grandparent to red.

26. 26 Insertion Example 10 3 15 20 1 5 7 Insert 8 10 3 15 20 1 5 7 8 Previous recoloring doesn’t work. Uncle (a leaf) already black. Recolor & Rotate 10 3 15 20 1 7 8 5 Recolor parent black. Recolor grandparent red. Rotate grandparent.

27. 27 Insertion Example 10 3 15 20 1 7 8 5 Insert 9 10 3 15 20 1 7 8 5 9 Uncle is so… Recolor parent, uncle to black. Recolor grandparent to red. Recolor 10 3 15 20 1 7 8 5 9 Grandparent & its parent might now both be red. Continue.

28. 28 Insertion Example 10 3 15 20 1 7 8 5 9 Uncle of current node is black, so… Need to recolor parent & grandparent. But current node is a right child while parent is a left child. Want both to be same side. Rotate 10 3 15 20 1 7 8 59

29. 29 Insertion Example Recolor & Rotate 10 3 15 20 1 7 8 59 103 15 20 1 7 85 9

30. 30 Deletion Basically the same as insertion Delete as you would in a BST, then check to maintain properties Read about this in Weiss (12.2) and/or Cormen (13)

31. 31 Conclusion Red Black vs Splay Trees  Fewer Rotations  1 extra color bit Where are they used?  Generally for look-up tables (key, value) data  In Java: The TreeMap and TreeSet classes in the java.util package When would you use?  AVL?  B-Tree?  Red-Black?  Splay?

32. 32

33. 33 Priority Queues Unordered Linked List  Insert O(1), delete O(n) Ordered Linked List  Insert O(n), delete O(1) Balanced BST  Insert, delete, remove O(log n)  Merge O(n) A better way exists…Heaps!!!

34. 34 Binary Heaps Binary Heaps Are Complete and Partially ordered trees  Complete: Filled on all levels, except the last one where it is filled left to right (so between 2 h and 2 h+1 -1 nodes)  Partially ordered: Key of each node is = parent (max-heap) Note: left child is not necessarily less than right child Since they are complete, can be easily stored as an array

35. 35 Binary Heaps Min/Max element is in the root Heap with N elements has height 6 14 78 18 81 7791 45 5347 64 84 99 83

36. 36 Insertion Idea:  Put into next available leaf node  Recursively, if the new node is < parent then swap (if min-heap) This may “bubble up” to the root, also called “sifting up” Complexity?  O(lgN), but on average O(1)

37. 37 Insertion Insert element X into heap  Insert into next available slot  Bubble up until obeys heap ordering rule (min/max heap) 6 14 78 18 81 7791 45 5347 64 84 99 83 42 next free slot

38. 38 Insertion Insert element X into heap  Insert into next available slot  Bubble up until obeys heap ordering rule (min/max heap) 6 14 78 18 81 7791 45 5347 64 84 99 83 42 swap with parent

39. 39 Insertion Insert element X into heap  Insert into next available slot  Bubble up until obeys heap ordering rule (min/max heap) 6 14 78 18 81 7791 45 4247 64 84 99 83 42 53 swap with parent

40. 40 Insertion Insert element X into heap  Insert into next available slot  Bubble up until obeys heap ordering rule (min/max heap) 6 14 78 18 81 7791 42 4547 64 84 99 83 53 stop: heap ordered

41. 41 Deletion Idea:  Remove the root from the tree  Remove the last (rightmost) leaf and place it at the root  Perform the “sift down” operation If the key is larger then swap with the smallest child, repeat. Complexity?  O(lgN), on average O(lgN)

42. 42 Binary Heap: Delete Min Delete minimum element from heap.  Exchange root with rightmost leaf.  Bubble root down until it's heap ordered. 6 14 78 18 81 7791 42 4547 64 84 99 83 53

43. 43 Binary Heap: Delete Min Delete minimum element from heap.  Exchange root with rightmost leaf.  Bubble root down until it's heap ordered. 53 14 78 18 81 7791 42 4547 64 84 99 83 6

44. 44 Binary Heap: Delete Min Delete minimum element from heap.  Exchange root with rightmost leaf.  Bubble root down until it's heap ordered. 53 14 78 18 81 7791 42 4547 64 84 99 83 exchange with left child

45. 45 Binary Heap: Delete Min Delete minimum element from heap.  Exchange root with rightmost leaf.  Bubble root down until it's heap ordered. 14 53 78 18 81 7791 42 4547 64 84 99 83 exchange with right child

46. 46 Binary Heap: Delete Min Delete minimum element from heap.  Exchange root with rightmost leaf.  Bubble root down until it's heap ordered. 14 18 78 53 81 7791 42 4547 64 84 99 83 stop: heap ordered

47. 47 Building a Heap Start at the root Recursively travel to each of the leaves. From each leaf, perform the “sift up” operations Pseudo-code:  for i= downto 1 minHeapify(A,i)//or maxHeapify  Where A is an array representation of the heap, and i is the current node being inspected for the heap property

48. 48 Fix the bottom level Fix the next to bottom level Fix the top level

49. 49 Building a Heap

50. 50 Complexity of Heap Building Theorem: Given a perfect binary tree of height h, containing 2 h+1 -1 nodes the sum of the heights of the nodes is 2 h+1 -1-(h+1) What is the complexity?  O(n)  Why? Sum of the heights of the nodes is N Proof:

51. 51 Sorting How would you use heaps to sort?

52. 52 Heap Sort First Build the heap Continually call the Delete method until no more items are in the heap  Complexity? O(N lgN)

53. 53 H1H1 H2H2 14 78 18 81 7791 11 6253 64 84 99 41 Union  Combine two binary heaps H 1 and H 2 into a single heap.  No easy solution -  (N) operations apparently required  Can support fast union with fancier heaps.

54. 54

55. 55 D-heaps Same as a binary heap except each node has d children So, a binary heap is a d-heap or order 2 A 3-heap could look like:

56. 56 D-Heaps Since more children per node it is shallower than a 2-heap  Improves insert running time  O(log d n) If d is large then deletion becomes costly  Shallower tree but must find the minimum of d children which is a linear algorithm  Result is O(dlog d n)

57. 57 D-Heaps A speedup is possible if the number of insertions is much greater than the number of deletions If the heap becomes so large that it cannot entirely fit in memory Possible that 4-heap is better than a 3-heap

58. 58 Union Problem: Merging 2 array implemented heaps is linear in time  Why? Must copy 1 heap into another  Could use a linked list, but then other operations may be slower because we are using linear lists Is it possible to use arrays and still be able to merge heaps in O(lgn)?

59. 59 Leftist Heap Ordering is same as for 2-heaps Structure is a bit more complicated  Leftist heaps not perfectly balanced, rather make a point not to be balanced! The Null Path Length (NPL) of a node X is the length of the shortest path from X to a node without 2 children Requirement for Leftist heap:  For every node X, the NPL(X.left) >= NPL(X.right)

60. 60 Theorem A leftist tree with R nodes on the right path must have at least 2 R -1 nodes Proof:  If R=1, then there must be >= 1 node  Assume the theorem is true for 1,2,…R  Consider leftist tree with R+1 nodes on the right path So, root has a right subtree with R nodes on the right path Left subtree with at least R nodes on the right path (or not leftist) Applying 2 R -1 hypothesis to each subtree results in a minimum of 2 R -1 in each subtree. If we also add in the root, we get 2 R+1 -1 total nodes. QED.

61. 61 Operations Merge: main reason for existence! O(lgN) Insert: merges a 1 node heap with an existing heap Delete: removes the root then merges the subtrees

62. 62 Merge Algorithm Merge(H1, H2)  If H1 or H2 is empty, return non-empty heap  Recursively merge the heap with the larger root with the right subheap of the other (smaller rooted) heap  Make the new heap the right child of the root of H1 Satisfies heap property, but not leftist property – except the right subtree of the root is leftist Done by swapping the root’s left and right children and updating the NPL

63. 63 Merge Example 86 9 685 43 6

64. 64 Merge Example 86 9 685 43 6 Merge right subtree of tree with smaller root and the other tree

65. 65 Merge Example 86 9 685 43 6 Merge right subtree of tree with smaller root and the other tree

66. 66 Merge Example 86 68 4 6 Merge right subtree of tree with smaller root and the other tree

67. 67 Merge Example 8 6 Merge right subtree of tree with smaller root and all of other tree. Right subtree of 6 is empty. So, result of melding right subtree of tree with smaller root and other tree is the other tree.

68. 68 Merge Example Swap left and right subtree if NPL(left) < NPL(right) Make merged subtree right subtree of smaller root 6 8 6 8

69. 69 Merge Example 86 66 4 886 6 4 6 8 Make merged subtree right subtree of smaller root

70. 70 Merge Example Make melded subtree right subtree of smaller root, then swap children 9 5 3 86 66 4 8 1 2

71. 71 Merge Example 9 5 3 86 66 4 8 Final Result!

72. 72 Other Types Skew Heaps  Like leftist, but no left balancing condition  Always swaps children after a merge Binomial Queues  Collection of heap ordered “binomial trees” each with size of power 2  Merge is like adding integers in base 2  Very flexible Fibonacci Heaps  Variation of binomial queues  “sift up” runs in O(1) amortized time  Other operations in O(lgN) amortized time