1. Franklin University 1 COMP 620 Algorithm Analysis Module 3: Advanced Data Structures Trees, Heap, Binomial Heap, Fibonacci Heap

2. Franklin University 2 COMP 620 Algorithm Analysis Trees Definition: A tree (which is one type of a data structure) is a finite set of one or more nodes such that: There is a specially designated node called root. The remaining nodes are divided into n>=0 disjoint sets T1, T2,..... Tn where each of these sets is a tree. T1, T2,.... Tn are called the subtrees of the root. Node - represents an item of information stored in the tree. Branches - represent the links between the nodes. Root of the tree or root node - node at the top of the tree, the start of the tree. Degree of a node - the number of subtrees of the node (i.e., the number of children from any one node). Degree of a tree - the maximum degree of any of the nodes in the tree. Leaf or terminal nodes - nodes with degree 0.

3. Franklin University 3 COMP 620 Algorithm Analysis Tree Terminology (contd.) 7.Child/children - the roots of the subtrees of the parent node. 8.Siblings - children of the same parent. 9.Parent - a node that has subtrees (i.e., a node that has children). 10.Level of a node - defined by root = 1, children of root = 2, grandchildren of root = 3, etc. (In some textbooks, the root is defined to be at level 0, children of the root at level 1, etc.) 11.Depth or height of a tree - the maximum level of any node in the tree. 12.Binary trees are trees with no more than two-way branching from each node in the tree. A binary tree is either empty or consists of a root node and two disjoint binary trees called left and right subtrees.

4. Franklin University 4 COMP 620 Algorithm Analysis Tree - Example A L B K O C E F N M J I HG D Degree of the tree = Height (depth) of the tree = Terminal nodes of the tree are:

5. Franklin University 5 COMP 620 Algorithm Analysis Binary Trees: Properties The maximum number of nodes on level i of a binary tree is 2 (i-1), where i >= 1. The maximum number of nodes in a binary tree of depth k is 2 k –1,where k >= 1. For any non-empty binary tree, T, if n0 represents the number of leaf nodes and n2 represents the number of nodes with degree 2, then n0 = n2 + 1. A full binary tree of depth k is a binary tree of depth k having 2 k - 1 nodes, where k >= 1. A binary tree with n nodes and depth k is complete if and only if its nodes correspond to the nodes numbered from 1 to n in the full binary tree of depth k, filling in from left to right on each level.

6. Franklin University 6 COMP 620 Algorithm Analysis Full and Complete Binary Trees C GF E D B A 4 5 32 1 Full Binary Tree Complete Binary Tree Array Representing Above Tree ABCDEFG 0 1 2 3 4 5 6 7 Leftchild(i) is at 2i if 2i >n no left child Rightchild(i) is at 2i + 1 if 2i +1 > n no right child

7. Franklin University 7 COMP 620 Algorithm Analysis Neither Full nor Complete Trees A D B C E F G Array Representation of the Above Tree ABCDEFG 0 1 2 3 4 5.. 10 11.. 23 The array representation of the tree that is neither full nor complete is very wasteful of memory

8. Franklin University 8 COMP 620 Algorithm Analysis Linked Representation template Class BtNode { public: BaseData info; BtNode *leftChild, *rightChild; };

9. Franklin University 9 COMP 620 Algorithm Analysis Traversing Trees Inorder traversal 1.Traverse the left subtree 2.Visit the root 3.Traverse the right subtree Void BinTree::inord( BtNode *rt) { if (rt != NULL) { inord(rt->leftChild); processNode(rt->info); inord(rt->rightChild); }

10. Franklin University 10 COMP 620 Algorithm Analysis Traversing Trees Preorder traversal 1.Visit root 2. Traverse the left subtree 3. Traverse the right subtree Void BinTree::preord( BtNode *rt) { if (rt != NULL) { processNode(rt->info); preord(rt->leftChild); preord(rt->rightChild); }

11. Franklin University 11 COMP 620 Algorithm Analysis Traversing Trees Postorder traversal 1.Traverse the left subtree 2.Traverse the right subtree 3.Visit the root Levelorder traversal Every node is visited in turn from left to right on every level, starting at level 1, then level 2, etc., until level n, where n represents the height (depth) of the tree.

12. Franklin University 12 COMP 620 Algorithm Analysis Binary Search Trees Binary search tree is of significant importance in Computer Science. Has better performance than many other data structures especially for operations like insertion, deletion, and searching. Definition: A binary search tree is a binary tree. It may be empty. If not empty, it satisfies the following properties: Every element has a key, and traditionally no two elements have the same key. Keys in a non-empty left subtree must be smaller than the key in the root of the subtree. Keys in a non-empty right subtree must be larger than the key in the root of the subtree. Left and right subtrees are also binary search trees.

13. Franklin University 13 COMP 620 Algorithm Analysis J O H E A F G I D B C M P N L K Binary Search Tree

14. Franklin University 14 COMP 620 Algorithm Analysis Binary Search Trees: Operations Searching a binary search tree: Begin at the root. If the key of the element to be searched = root key, then the search is successful. If the key of the element to be searched < root key, then search the left subtree. If the key of the element to be searched > root key, then search the right subtree. Inserting into a binary search tree: - Search for the key - If the key is not present, locate the parent node - Insert the new node as the left/right child of the parent node All insertions take place at leaf nodes.

15. Franklin University 15 COMP 620 Algorithm Analysis Binary Search Trees: Operations Deleting from a binary search tree: The deletion process begins with a search to find the node to be deleted from the binary search tree. Three possible scenarios exist in the deletion process: 1.Delete a leaf node. - Make the appropriate pointer in x’s parent a null pointer 2.Delete a node with 1 child. - Set the appropriate pointer in x’s parent to point to this child 3.Delete a node with 2 children. - Replace the value stored in the node x by its inorder successor (predecessor) and then delete the successor (predecessor)

16. Franklin University 16 COMP 620 Algorithm Analysis Binary Search Trees: Variations AVL (or height-balanced) trees A binary search tree in which the balance factor of each node is 0, 1, or –1, where the balance factor of a node x is defined as the height of the left subtree of x minus the height of x’s right subtree. 2-3-4 trees:A tree with the following properties: Each node stores at most 3 data values. Each internal node is a 2-node, 3-node, or a 4- node. All the leaves are on the same level.

17. Franklin University 17 COMP 620 Algorithm Analysis Motivation for AVL Tree 2 1 3 4 5 5 3 41 2 The height of AVL tree is approximately Log 2 n What is the value of balance of each node?

18. Franklin University 18 COMP 620 Algorithm Analysis 2-3-4 Tree 53 2738 6070 16 25 3336 41 46 48 5559 6568 73 75 79

19. Franklin University 19 COMP 620 Algorithm Analysis Red Black Trees Red Black trees A binary search tree with two kinds of nodes, red and black, which satisfy the following properties: Every node is either red or black. Root is black. Every leaf (NIL) is black. If a node is red, both its children are black. Every simple path from node to descendant leaf contains the same number of black nodes. A red-black tree with n internal nodes has height at most 2log(n+1).

20. Franklin University 20 COMP 620 Algorithm Analysis 11 14 2 15 1 8 7 5 RED-BLACK TREE

21. Franklin University 21 COMP 620 Algorithm Analysis B-trees B-trees of minimum degree t A tree with the following properties: - Each node stores at most 2t -1 data values. - Every node other than the root must have at least t- 1 data values. - n[x] keys in each node x stored in nondecreasing order, so that key 1 [x] <= key 2 [x].. <= key n[x] [x]. - Each internal node contains n[x] + 1 children. X contains n +1 pointers c 1 [x], c 2 [x] … c n[x]+1 [x].

22. Franklin University 22 COMP 620 Algorithm Analysis B-trees - The keys key i [x] separates the ranges of keys stored in each subtree: if k i is any key stored in the subtree with root c i [x] then, then k 1 <= key1[x] <= key 2 [x] <= ….<= key n[x] [x] <= k n[x]+1 - All the leaves are on the same level. - B alanced and designed to work efficiently on disks and other direct access secondary storage devices. Many database systems use B-tree. - the maximum height of a n-key B tree is log t ((n+1)/2) - insert, delete, and search time on a B-Tree is  (log n)

23. Franklin University 23 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

24. Franklin University 24 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

25. Franklin University 25 COMP 620 Algorithm Analysis B-tree of minimum degree t = 2 M Q T X D H Y Z V W N P F GB C J K L Is 2-3-4 tree a B-tree? R S

26. Franklin University 26 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

27. Franklin University 27 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

28. Franklin University 28 COMP 620 Algorithm Analysis B-Tree Insert - locate the node where the key should be inserted - split any full nodes encountered while descending the tree Deletion Deletion is similar to Insertion. Make sure that the tree is still a B-Tree after deletion. Detailed discussion of deleting from a B-tree, refer to Section 18.3, pages 450-453, of Cormen, Leiserson, and Rivest.

29. Franklin University 29 COMP 620 Algorithm Analysis G M P X A C D E J K N O R S T U V Y Z (a) Initial Tree G M P X A B C D E J K N O R S T U V Y Z (b) B Inserted B-Tree Insertion - Inserts only in leaf node

30. Franklin University 30 COMP 620 Algorithm Analysis B-Tree Insertion - Inserts only in leaf node G M P T X A B C D E J K N O U V Y Z Q R S P G M T X A B C D E J K L N O Q R S U V Y Z (C ) Q Inserted (d ) L Inserted

31. Franklin University 31 COMP 620 Algorithm Analysis P C G M T X A B J K L N O Q R S U V Y Z B-Tree Insertion - Inserts only in leaf node (e ) F Inserted D E F

32. Franklin University 32 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

33. Franklin University 33 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

34. Franklin University 34 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

35. Franklin University 35 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

36. Franklin University 36 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

37. Franklin University 37 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

38. Franklin University 38 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

39. Franklin University 39 COMP 620 Algorithm Analysis Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

40. Franklin University 40 COMP 620 Algorithm Analysis Heaps Heaps data structures are mostly used to support the following operations (not efficient to use for search): Insert element x Return min element Delete minimum element Union of two heaps Application: Dijkstra’s Shortest Path Prim’s MST Huffman Encoding Heap Sort

41. Franklin University 41 COMP 620 Algorithm Analysis Binary Heap A heap is a complete binary tree such that the value of the key in the root is greater than the value of the key in each of its children, and that both subtrees are also heaps (a recursive definition). http://homepages.ius.edu/rwisman/C455/html/notes/Chapte r6/HeapSort.htm http://homepages.ius.edu/rwisman/C455/html/notes/Chapte r6/HeapSort.htm Heapsort: (1) Make a heap of the elements to be sorted. (2) Convert the heap into a sorted list. 10 6 3 2 9 5

42. Franklin University 42 COMP 620 Algorithm Analysis Heap Sort HeapSort(A) 1Build_Max_Heap(A) 2for i ← length[A] downto 2 do 3 exchange A[1] ↔ A[i] 4heap-size[A] ← heap-size[A] – 1 5Max_Heapify(A, 1) Build_Max_Heap(A) 1heap-size[A] ← length[A] 2for i ← floor(length[A]/2) downto 1 do 3Max_Heapify (A, i)

43. Franklin University 43 COMP 620 Algorithm Analysis HEAP SORT Heapify(A, i) 1l ← LEFT(i) 2r ← RIGHT(i) 3if l ≤ heap-size[A] and A[l] > A[i] 4then largest ← l 5else largest ← i 6if r ≤ heap-size[A] and A[r] > A[largest] 7then largest ← r 8if largest ≠ i 9then exchange A[i] ↔ A[largest] 10Max_Heapify (A, largest)

44. Franklin University 44 COMP 620 Algorithm Analysis HEAPS OperationLinked List Binary Heap Binomial Heap Fibonacci Heap make-heap1111 insert1Log N 1 find-minN1Log N1 delete-minNLog N decrease-key1Log N 1 deleteNLog N union1NLog N1

45. Franklin University 45 COMP 620 Algorithm Analysis BINOMIAL TREE Recursive Definition : B k consists of 2 binomial trees B k-1 that are linked together; the root of the one is the leftmost Child of the root of the other. B 0 B 2 B 3 B 1

46. Franklin University 46 COMP 620 Algorithm Analysis Binomial Tree Properties Number of nodes = 2 k Height of the tree = k There are nodes nodes at level i The root has degree k and it children are B k-1, B k-2,.. B 0 from left to right.

47. Franklin University 47 COMP 620 Algorithm Analysis Binomial Heap A binomial heap is a set of binomial trees Each binomial tree in H is heap-ordered key (x) >= key (parent(x)). There never exist 2 or more trees with same degree in the heap. Linked list of roots in order of increasing degree Binomial tree is stored in a left child-right sibling representation.

48. Franklin University 48 COMP 620 Algorithm Analysis Binomial Heap 5 7 10 3 17 6 4 B 0 B 2 B 1 View View View the operations on binomial heap at the class web site Head Sibling Parent Child degree Key Roots of the trees connected with singly linked list Head points to the first node in the linked list

49. Franklin University 49 COMP 620 Algorithm Analysis Binomial Heap MAKE_BINOMIAL_HEAP() allocate(H) head[H] = NIL Insert(H, x) H’ = MAKE-BINOMIAL-HEAP() set x’s field appropriately head[H’] = x n[H’] = 1 H = BINOMIAL-HEAP-UNION(H, H’]

50. Franklin University 50 COMP 620 Algorithm Analysis Binomial Heap Binomial-Heap-Minimum(H) y = NIL x = head[H] min = inf while x <> NIL do if key[x] < min then min = key[x] y = x x = sibling[x] return y;

51. Franklin University 51 COMP 620 Algorithm Analysis Binomial Heap BINOMIAL-HEAP-UNION (H1, H2) Merge the root lists of binomial heaps H1 and H2 into a single linked linked list H that is sorted by degree into monotonically increasing order. Links roots of equal degree until at most one root remains of each degree. Binomial-Link (y,z) p[y] = z sibling[y] = child[z] child[z] = y degree [z] = degree[z] + 1

52. Franklin University 52 COMP 620 Algorithm Analysis Binomial Heap BINOMIAL-HEAP-EXTRACT-MIN(H) find the root x with the minimum key in the root list of H, and remove x from the root list of H H’ = MAKE-BINOMIAL-HEAP() reverse the order of the linked list of x’s children and set head[H’] to point to the head of the resulting list H = BINOMIAL-HEAP-UNION (H, H’) return (x)

53. Franklin University 53 COMP 620 Algorithm Analysis Fibonacci Heap A set of min-heap-ordered trees Roots of trees are connected with circular doubly linked list. Children of a node are connected with circular doubly linked list. Pointer to root of tree with minimum element. Parent Mark: Newly created nodes are unmarked. This field becomes true if the node has lost a child Since the node became a child Of another node. Left Right Child Key Degree

54. Franklin University 54 COMP 620 Algorithm Analysis Fibonacci Heap View the operations on Fibonacci heap at the class web site 23 7 3 17 24 18 52 38 41 39 46 30 26 35 Min[H]

55. Franklin University 55 COMP 620 Algorithm Analysis Fibonacci Heap Make_heap() allocate(H) min[H] = NIL n[H]= 0 Insert(H, x) set x’s field appropriately add x to root list of H reset min[H] if needed n[H] = n[H] + 1

56. Franklin University 56 COMP 620 Algorithm Analysis Fibonacci Heap Union(H1, H2) H = new heap whose root list contains roots from H1 and H2 Min[H] = min[H1] If ( min[H1] == NIL) or (min [H2] <> NIL and min[H2] < min[H1]) then min [H] = min[H2] N[H] = n[H1] + n[H2] free the objects H1 and H2 return H

57. Franklin University 57 COMP 620 Algorithm Analysis Fibonacci Heap Extract-min (H) z = min[H] Add z’s children to root list Remove z from root list If root list <> {} then Consolidate H else min[H] = NIL n[H] = n[H] - 1

58. Franklin University 58 COMP 620 Algorithm Analysis Fibonacci Heap Consolidate (H) While 2 trees in H (T1, T2) have the same degree: if root(T1) < root (T2) then make T1 child of T2 else make T2 child of T1 for i = 0 to D(n[H] ) // D is max possible trees if tree T of degree i has root-key < min[H] then min[H] = T

59. Franklin University 59 COMP 620 Algorithm Analysis FIB-HEAP-DECREASE-KEY(H, x, k) 1 if k > key[x] 2 then error "new key is greater than current key" 3 key[x] ← k 4 y ← p[x] 5 if y ≠ NIL and key[x] < key[y] 6 then CUT(H, x, y) 7 CASCADING-CUT(H, y) 8 if key[x] < key[min[H]] 9 then min[H] ← x CUT(H, x, y) 1 remove x from the child list of y, decrementing degree[y] 2 add x to the root list of H 3 p[x] ← NIL 4 mark[x] ← FALSE CASCADING-CUT(H, y) 1 z ← p[y] 2 if z ≠ NIL 3 then if mark[y] = FALSE 4 then mark[y] ← TRUE 5 else CUT(H, y, z) 6 CASCADING-CUT(H, z) Fibonacci Heap Decrease Key

60. Franklin University 60 COMP 620 Algorithm Analysis Fibonacci Heap Decrease Key

61. Franklin University 61 COMP 620 Algorithm Analysis Fibonacci Heap Decrease Key a.The initial Fibonacci heap b.The node with key 46 has its key decreased to 15. The node becomes a root, and its parent ( with key 24) which had previously been unmarked, becomes marked. c-e The node with key 35 has its key decreased to 5. In part (c), the node now with key 5, becomes a root. Its parent, with key 26, is cut from its parent and made an unmarked root in (d). Another cascading cut occurs, since the node with key 24 is marked as well. This node is cut from its parent and made an unmarked root in part (e). The cascading cuts stop at this point, since the node with key 7 is a root.

62. Franklin University 62 COMP 620 Algorithm Analysis Fibonacci Heap Delete Node FIB-HEAP-DELETE(H, x) 1 FIB-HEAP-DECREASE-KEY(H, x, -∞) 2 FIB-HEAP-EXTRACT-MIN(H)