1. Advanced Data Structures CHAPTER 2

2. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures2 Outlines Binary search trees B-trees Heaps and priority queues Skip list

3. Binary Search Trees

4. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures4 Binary Search Trees: Nodes leftright keyparent leftrightkeyparent leftrightkeyparent internal node root nodeleaf node

5. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures5 Property of Binary Search Trees For any node n in a binary search tree T, if p is a node in the left subtree of n, then p.key ≤ n.key; if q is a node in the right subtree of n, then n.key ≤ q.key. n pq T

6. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures6 Example of Binary Search Trees 20 196 10 9412 17 8 5 94

7. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures7 Inorder Traversal of Binary Search Trees 20 196 10 9412 17 8 5 94 inorder(T: node) if T is not null, then inorder(T.left) print(T.key) inorder(T.right) return. }

8. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures8 Search in Binary Search Trees search(n: node, k: key) if n is null or n.key = k, then return(n) if k < n.key, then search(n.left, k) else search(n.right, k) return. 20 196 10 9412 17 8 5 94

9. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures9 Found the place, and insert as the left child. Found the place, and insert as the right child. insert(T: node, z: node) if T is null then T.root = z return if key[z] < key[T] then if T.left is null then T.left = z z.p = T else insert(x.left, z) else if T.right is null then x.right = z z.p = T else insert(x.right, z) return Insertion in Binary Search Trees insert(T: node, z: node) y = null x = T while x is not null y = x if z.key < x.key then x = x.left else x = x.right z.parent= y if y = NIL then T.root= z else if z.key < y.key then y.left= z else y.right = z The tree T is empty.

10. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures10 Deletion in Binary Search Trees See textbook

11. B-trees

12. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures12 B-trees Index structures for large amount of data. All data cannot be resided in the main memory. Data is stored in a disk.

13. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures13 Disk Operations In a disk, data is organized in disk pages. To read data in a disk, The disk rotates, and the R/W head moves to the page containing the data. Then, the whole disk page is read. From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

14. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures14 B-trees: Nodes number of keys leaf key 1 key n key 2 key 3 number of keys leaf key 1 key n key 2 key 3 false true Internal node leaf node Pointers to other nodes Pointers to data pages

15. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures15 Properties of B-trees For any node n in a B-tree T n.key i ≤ c.key 1 ≤ c.key 2 ≤ … ≤ c.key q ≤ n.key i+1 If n is a leaf node, the depth of n is h, where h is the tree’s height. n c p key 1 key i key i +1 … false … q key 1 key i key i +1 … false …

16. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures16 Properties of B-trees The minimum degree, t, of the B-tree is the lower bound on the number of keys a node can contain. (t ≥ 2) Every node other than the root must have at least t − 1 keys. Every internal node other than the root thus has at least t children. If the tree is nonempty, the root must have at least one key. Every node can contain at most 2t − 1 keys. Therefore, an internal node can have at most 2t children. We say that a node is full if it contains exactly 2t − 1 keys.

17. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures17 Search in B-trees 86121660268875676617117168656970

18. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures18 Search in B-trees B-TREE-SEARCH(x, k) i = 1 ► find a proper child while i ≤ x.n and k > x.key i do i =i + 1 if i ≤ x.n and k = x.key i then return (x, i ) if x.leaf then return NIL ► recursively search in the proper subtree else DISK-READ (x.c i ) return B-TREE-SEARCH (x.c i, k)

19. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures19 Creating B-trees B-TREE-CREATE ( T ) x = ALLOCATE-NODE () x.leaf = TRUE x.n = 0 DISK-WRITE (x) T.root = x

20. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures20 8 Insertion in B-trees 2665 17 8 11 65 11 1726

21. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures21 Insertion in B-trees 11812166526 16 17 12 6 11

22. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures22 Insertion in B-trees 861216652671 1711 60 6571 65

23. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures23 Insertion in B-trees 861216602688757167 171165 66 8875 71

24. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures24 Insertion in B-trees 86121660268875676669 17116571 70 6968

25. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures25 Insertion in B-trees 8612166026887567661711656917116571 68 70

26. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures26 Insertion in B-trees 861216602688756766171171686569 70

27. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures27 Insertion in B-trees B-TREE-INSERT ( T, k) r = T.root if r.n = 2t − 1 ► The root node r is full, then create a new root node then s = ALLOCATE-NODE () T.root = s; s.leaf = FALSE s.n = 0; s.c 1 = r ► Split old root into two, and put under the new root B-TREE-SPLIT-CHILD (s, 1, r) B-TREE-INSERT-NONFULL (s, k) else B-TREE-INSERT-NONFULL (r, k)

28. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures28 Splitting Nodes in B-trees B-TREE-SPLIT-CHILD (x, i, y) ► Create a new node z = ALLOCATE-NODE () z.leaf = y.leaf z.n = t − 1 ► Move 1/2 of old node to new for j = 1 to t − 1 do z.key j = y.key j+t if not y.leaf then for j = 1 to t do z.c j = y.c j+t y.n = t − 1 ► Move up other 1/2 of old node for j = x.n + 1 downto i + 1 do x.c j+1 = x.c j x.c i+1 = z for j = x.n downto i do x.key j+1 = x.key j x.key i = y.key t x.n = x.n + 1 DISK-WRITE (y) DISK-WRITE (z) DISK-WRITE (x)

29. Binary Heaps

30. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures30 Binary Heaps Binary heap is an array can be viewed as a nearly complete binary tree each node of the tree corresponds to an element of the array that stores the value in the node. The tree is completely filled on all levels except possibly the lowest, which is filled from the left up to a point. 1 5 3 10 11 2 4 89 7 6 1213 45769810111212313 leftright

31. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures31 Representation of Biinary Heaps An array A that represents a heap is an object with two attributes: length[A], which is the number of elements in the array heap-size[A], the number of elements in the heap stored within array A. 123……nx…x lengthheap-size

32. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures32 Properties of Binary Heaps If a heap contains n elements, its height is lg 2 n. In a max-heaps For every non-root node i, A[PARENT(i)] ≥ A[i] In a min-heaps For every non-root node i, A[PARENT(i)] ≤ A[i]

33. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures33 Examples 3 15 6 9 12 5 8 21 46 15 5 12 11 9 18 7 1920 Max heap Min heap

34. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures34 Heapify 3 7 6 9 12 5 8 21 46

35. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures35 Heapify 3 7 6 9 12 5 8 21 46 7

36. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures36 Heapify 3 12 6 9 7 5 8 21 46 9 7

37. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures37 Heapify MAX-HEAPIFY(A, i ) l = LEFT(i ) r = RIGHT(i ) ► Find node containing the largest value among i and its valid children. if l ≤ heap-size[A] and A[l] > A[i ] then largest = l else largest = i if r ≤ heap-size[A] and A[r] > A[largest] then largest = r ► Swap the largest node and node i if the node i is not the largest if largest  i then exchange A[i ] and A[largest] MAX-HEAPIFY(A, largest)

38. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures38 Time Complexity for Heapify worst-case The left subtree is one-level taller than the right subtree, and both are complete binary trees The left subtree contains (2n-1)/3 nodes and the right subtree contains (n-2)/3. Let T(n) be time spent to heapify an n-node heap. T(n) = T((2n-1)/3) + k T(n)  Ο(lg n) T(n)  Ο(h) since the height h of a heap is in Ο(lg n)

39. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures39 Building Max-heaps: BUILD-MAX-HEAP(A) 8 1 9 4 2 7 3 1011 56 5 9 4 7 32 2 10

40. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures40 Building Max-heaps: BUILD-MAX-HEAP(A) 8 1 2 69 4 7 3 11 5 10 11 110 15 1

41. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures41 Building Max-heaps: BUILD-MAX-HEAP(A) heap-size[A] = length[A] ► Start from the rightmost node in the level above leaves for i =  length[A]/2  downto 1 do MAX-HEAPIFY(A, i ) 8 1 9 4 2 7 3 1011 56

42. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures42 Time to Build Max-heaps Let n be the heap size, T(n) be the time spent in BUILD-MAX-HEAP for heap of size n, t(h) be the time spent in MAX-HEAPIFY for the heap of height h, k(h, n) be the number of subtrees of height h in the heap of size n.  lg n   lg n   lg n  T(n)   t(h) k(h, n) =  c h  n/2h+1  = c n  (h/2h) h=0 h=0 2 h=0  lg n  T(n) = O(n  (h/2h)) h=0  Since  (h/2h)) = 2, T(n) = O(n). h=0

43. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures43 Heapsort HEAPSORT(A) BUILD-MAX-HEAP(A) for i = length[A] downto 2 ► Swap the max node with last in heap, and ► heapify heap, excluding the last (old max) do exchange A[1] and A[i ] heap-size[A] = heap-size[A] − 1 MAX-HEAPIFY(A, 1)

44. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures44 Time for Heapsort The call to BUILD-MAX-HEAP takes O(n) Each of the n − 1 calls to MAX-HEAPIFY takes O(lg n) HEAPSORT takes O(n lg n).

45. Priority Queues

46. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures46 Priority Queues A priority queue is a max-heap with operations for queues. Insert Extract-max Increase-key HEAP-MAXIMUM(A) return A[1]

47. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures47 Extract-Max In Priority Queues HEAP-EXTRACT-MAX(A) if heap-size[A] < 1 thenerror “heap underflow” ► Take the maximum element from the root max = A[1] ► Move the last element in the heap to the root A[1] = A[heap-size[A]] heap-size[A] = heap-size[A] − 1 ► Heapify MAX-HEAPIFY(A, 1) return max

48. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures48 Increase Key HEAP-INCREASE-KEY(A, i, key) if key < A[i ] then error “new key < current key” A[i ] = key ► Swap the increased element with its parent up toward ► the root until it is not greater than its parent while i > 1 and A[PARENT(i)] < A[i ] do exchange A[i] and A[PARENT(i)] i = PARENT(i)

49. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures49 Insert MAX-HEAP-INSERT(A, key) ► Insert the minimum element heap-size[A] = heap-size[A] + 1 A[heap-size[A]]=−∞ ► Increase the minimum element HEAP-INCREASE-KEY(A, heap-size[A], key)

50. Binomial Heaps

51. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures51 Why Binomial Heaps Union two binary heaps takes  (n). Can it be reduced? Use binomial heaps A binomial heap is a binomial tree that has the property of heaps.

52. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures52 Binomial Trees The binomial tree B k is an ordered tree which consists of two binomial trees B k−1 that are linked together. the root of one is the leftmost child of the root of the other. The binomial tree B 0 consists of a single node.

53. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures53 Examples From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

54. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures54 Examples From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001. level

55. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures55 Properties of Binomial Trees For the binomial tree B k, the number of nodes = 2k the height of the tree = k the number of nodes at depth i = k C i the degree of root = k (maximum degree) if the children of the root are numbered from left to right by k − 1, k − 2,..., 0, child i is the root of a subtree B i. See the proof in the textbook.

56. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures56 Binomial Heaps 101 12 18 25 8 11 27 17 6 14 38 29 head[H] B3B3 B2B2 B0B0 A binomial heap is a set of binomial trees with heap properties.

57. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures57 Nodes in Binomial Heaps parent child degree key sibling

58. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures58 Examples of Nodes in Binomial Heaps 101 12 18 25 8 11 27 17 6 14 38 29 8 6 1117 14 3 21 10 \ 38 0 \\\ \ \

59. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures59 Find Minimum in Binomial Heaps BINOMIAL-HEAP-MINIMUM(H) y = NIL x = H.head min = ∞ while x  NIL doif x.key < min thenmin = x.key y = x x = x.sibling return y 101 12 18 258 11 27 17 6 14 38 29 h e a d Maximum number of trees =  lg n  + 1, the running time of BINOMIAL-HEAP-MINIMUM = O(lg n)

60. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures60 Link Binomial Heaps Make node y the new head of the linked list of node z’s children in O(1) time. BINOMIAL-LINK(y, z) y.p = z y.sibling = z.child z.child = y z.degree = z.degree + 1 8 11 27 17 y 6 14 38 29 z B2B2 B3B3 B2B2

61. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures61 Union Binomial Heaps Link the roots of all trees in both heaps, in a non-decreasing order of the degree. Go through each tree in the heap, and combine two trees of equal degree in the heaps, according to one of the following four cases.

62. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures62 Union Binomial Heaps: case 1 x.degree < next-x.degree move up to the next tree. From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

63. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures63 Union Binomial Heaps: case 2 x.degree = next-x.degree = next-x.sibling.degree move up to the next tree (and they will be linked in next step) From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

64. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures64 Union Binomial Heaps: case 3 x.degree = next-x.degree Link x and next-x. Use the node with smaller key as the root. x.key<next-x.key x.key  next-x.key From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

65. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures65 Binomial-Heap-Union BINOMIAL-HEAP-UNION(H1, H2) H = MAKE-BINOMIAL-HEAP() H.head = BINOMIAL-HEAP-MERGE(H1, H2) Free the objects H1 and H2 but not the lists they point to if H.head = NIL then return H prev-x = NIL x = H.head next-x = x.sibling while next-x  NIL do if (x.degree  next-x.degree) or (next-x.sibling  NIL and next-x.sibling.degree=x.degree) ► Cases 1 and 2 then prev-x = x x = next-x else if x.key ≤ next-x.key ► Case 3 then x.sibling = next-x.sibling BINOMIAL-LINK(next-x, x) ► Case 4 else if prev-x = NIL then H.head = next-x else prev-x.sibling = next-x BINOMIAL-LINK(x,next-x) x = next-x next-x = sibling[x] return H

66. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures66 Binomial Heaps: Insert a Node BINOMIAL-HEAP-INSERT(H, x) H ’ = MAKE-BINOMIAL-HEAP() x.p = NIL x.child = NIL x.sibling = NIL x.degree = 0 H ’.head = x H = BINOMIAL-HEAP-UNION(H, H ’ )

67. Fibonacci Heaps

68. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures68 Why Fibonacci Heaps Every insertion in a binomial heap requires the structure adjustment. Take long time (O(lg n)) for that. Do we need to adjust it that often? No, if you use Fibonacci heaps instead!

69. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures69 Fibonacci Heaps A Fibonacci heap is a collection of min- heap-ordered trees. The trees in a Fibonacci heap are not constrained to be binomial trees. Trees within Fibonacci heaps are rooted but unordered. (unlike trees within binomial heaps, which are ordered) Circular, doubly linked lists are used in Fibonacci heaps.

70. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures70 Fibonacci Heaps:Example From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

71. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures71 Properties of unordered binomial tree U k Number of nodes = 2k Height of the tree = k Number of nodes at depth i = k C i The degree of root = k, which is greater than that of any other node The children of the root are roots of subtrees U 0,U 1,...,U k−1 in some order.

72. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures72 Nodes in Fibonacci Heaps parent child degree key mark rightleft

73. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures73 Links in Fibonacci Heaps From: Cormen, T., C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2001.

74. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures74 Fibonacci Heaps: Insert a node Link the node in the root list of H Update min[H] if necessary

75. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures75 Consolidate Heaps Find two roots x and y in the root list with the same degree, where key[x] ≤ key[y]. Link y to x, using FIB-HEAP-LINK, i.e. remove y from the root list, and make y a child of x. The field degree[x] is incremented, and the mark on y, if any, is cleared.

76. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures76 Time Complexity ProcedureBinary Heap (worst-case) Binomial Heap (worst-case) Fibonacci Heap (amortized) Make-heap  (1) Insert  (lg n) Ο (lg n)  (1) Minimum  (1) Ο (lg n)  (1) Extract-min  (lg n) Ο (lg n) Union (n)(n) Ο (lg n)  (1) Decrease-key  (lg n)  (1) delete  (lg n) Ο (lg n)

77. Skip Lists

78. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures78 Skip Lists Alternative to binary search trees Probabilistic data structure Easy to implement Good performance in average-case

79. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures79 Structure of Skip Lists Max. level H of the skip list An array H of header pointers A set of nodes such that a node N contains a key k and a set of values v, the height of h node N, where h<=H, and an array of h pointers. 26912 4 7 11 nil H

80. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures80 Skip List v.s. Binary search tree 7 4 21296 11 26912 4 7 11 nil

81. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures81 Search in a Skip List Search(list, searchkey) cn = list.head ► Start at the list head ► Start skip from top level to bottom for i=list.level downto 1 do ► Forward until key of node exceeds search key while ((cn.fwd[i]).key < searchkey) do cn = cn.fwd[i] cn = cn.fwd[1] if cn.key = searchkey then return(cn) else return(null)

82. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures82 Search in a Skip List 26912 4 7 11 nil H Search for 11 7 4 21296 11

83. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures83 Search in a Skip List 26912 4 7 11 nil H Search for 12 7 4 21296 11

84. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures84 Insert in a Skip List 2 6 912 4 7 11 nil cn update[3] cn update[2] update[1] Insert 5 5

85. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures85 Insert in a Skip List cn = list.head ► Search for a place to insert for i=list.maxlevel downto 1 do while ((cn.fwd[i]).key < node.key) do cn = cn.fwd[i] update[i]=cn ► The key is already in the list if cn.fwd[1].key = node.key then return(false) ► Insert the node in level 1 i=1 node.fwd[i] = update[i].fwd[i] update[i].fwd[i] = node ► Insert the node from level 2 up ► Toss a coin while (rand() >.5) do addLevel(node) i++ ► Update link in level i if (i <= list.level) then node.fwd[i] = update[i].fwd[i] update[i].fwd[i] = node else addLevel(list.head) (list.head).fwd[i] = node

86. Jaruloj ChongstitvatanaChapter 2: Advanced Data Structures86 Delete a node Only remove the node, and connect the lost links. The list can become uneven. But it is more efficient than deletion in binary search trees.