1. Trees Part of this lecture was organized by the instructors at the University of Manitoba in Canada and has been modified by Dr. Ahmad Reza Hadaegh

2. Trees A tree is a linked data structure which reflects some hierarchical structure There are several varieties of trees Each with its own characteristics A family of related data structures

3. Trees A tree gets its name from its structure which is tree-like A tree has a root, branches, leaves A J I H GF E D C B Root node Leaf nodes Branches

4. Trees A tree consists of a number of connected nodes The nodes are organized in a top-down hierarchical manner A tree is an instance of a general structure called a graph A graph consists of vertices (nodes) and edges (branches), but is not necessarily hierarchical in nature Sometimes tree and graph terminology is interchanged

5. Trees This tree has 10 nodes, 9 branches, 1 root node, and 6 leaf nodes A J I H GF E D C B

6. Trees Other defining characteristics If node A points to node B, then A is the parent of B Furthermore, B is the child of A Children of the same parent are siblings All nodes “beneath” a given node (children, children of children, …) are descendants of that node All nodes on the path between the root and a given node are ancestors of that node

7. Trees The parent of G is D and the parent of E is B The children of A are B, C, and D B, C, and D are sibling The ancestors of J are G, D, and A The descendants of D are G, H, I and J A J I H GF E D C B

8. Trees Some typical operations upon a tree Insert Add a new node into a tree Delete Remove a node from a tree Search Find a node (if it exists) that contains a given key Traverse Systematically examine (an operate on) all nodes in a tree

9. Binary Trees Binary trees A binary tree is a tree in which no node has more than two children A H G F E D C B

10. Ordered Binary Trees Ordered binary trees An ordered binary tree is a binary tree in which left descendants < given node < right descendants Applied recursively throughout the tree structure

11. Ordered Binary Trees The following binary tree is not ordered A H G F E D C B

12. Ordered Binary Trees The following binary tree is ordered D H F G C A E B

13. Ordered Binary Trees The following binary tree is also ordered D H E G B A F C

14. Ordered Binary Trees The following binary tree is also ordered B E A D H F C G

15. Ordered Binary Trees Ordered binary trees are often referred to as binary search trees (BST) They tend to be a “co-operative” structure on which to work

16. Searching a BST Algorithm Start with root node if search key = node key then stop else if search key < node key then go left else go right Stop if at any time during the search, you fall off the tree, stop

17. Searching a BST Find node with key C ` D H F G C A E B

18. Searching a BST Find node with key X D H F G C A E B

19. Searching a BST Given an existing BST, searching for a key is simple Effectively a binary search Is it as efficient as a binary search? Algorithm is recursive, but can be trivially implemented iteratively Could be implemented as follows:

20. Searching a BST bool Search(nodeptr curr, char key) { bool found; nodeptr curr; curr = root; found = false; while ((curr != NULL) && (! found)) if (curr->data == key) found = true; else if (key data) curr = curr->left; else curr = curr->right; return found; } // We can also do the search recursively

21. Traversing a Tree Tree traversal is a common operation Traversal of a tree can be divided into two category: Depth-First Traversal and Breadth-First Traversal Three common types of depth-first traversal are: Pre-order, post-order, and in-order traversal Depth-First Traversal can be easily done recursively We can also use stack operation to print a traverse a tree Breath-First traversal can be done with Queue

22. Recursive Depth-First Traversal In-order traversal ● Perform an In-order traversal of the left sub-tree ● Visit the current node ● Perform an In-order traversal of the right sub-tree Pre-order traversal ● Visit the current node ● Perform a pre-order traversal of the left sub-tree ● Perform a pre-order traversal of the right sub-tree Post-order traversal ● Perform a Post-order traversal of the left sub-tree ● Perform a Post-order traversal of the right sub-tree ● Visit the current node

23. void InOrderTraversal (NodePtr root) { if (root = = NULL) return; InOrder(root->left); cout data << endl; InOrder(root->right); } void PreOrderTraversal (NodePtr root) { if (root = = NULL) return; cout data << endl; PreOrder(root->left); PreOrder(root->right); } void PostOrderTraversal (NodePtr root) { if (root = = NULL) return; PostOrder(root->left); PostOrder(root->right); cout data << endl; }

24. Depth-First Traversal with Stack In Order - Traversal void Tree::PrintInOrderTreeWithStack(TreeStructPtr Root) { Stack stk; TreeStructPtr p = Root; while (p!= NULL) { while (p != NULL) { if (p->Right != NULL) stk.push(p->Right); stk.push(p); p=p->Left; } p = stk.pop(); while (!stk.empty() && p->Right == NULL) { cout Number "; p = stk.pop(); } cout Number "; if (!stk.empty()) p = stk.pop(); else p = NULL; }

25. void Tree::PrintPreOrderTreeWithStack(TreeStructPtr Root) { Stack stk; TreeStructPtr p = Root; if (p != NULL) { stk.push(p); while (!stk.empty()) { p = stk.pop(); cout Number "; if (p->Right != NULL) stk.push(p->Right); if (p->Left != NULL) stk.push(p->Left); } Depth-First Traversal with Stack Pre Order - Traversal

26. void Tree::PrintPostOrderTreeWithStack(TreeStructPtr Root) { Stack stk; TreeStructPtr p = Root, q = Root; while (p!= NULL) { for ( ; p->Left != NULL; p = p->Left) stk.push(p); while (p != NULL && (p->Right == NULL || p->Right == q)) { cout Number "; q = p; if (stk.empty()) return; p = stk.pop(); } stk.push(p); p = p->Right; } Depth-First Traversal with Stack Post Order - Traversal

27. Breadth-First Traversal ` D H F G C A E B In Breadth-First traversal, all the nodes in the first level are printed, followed by the nodes in second level, followed by the nodes in the third level and so on. For example, the breadth-first traversal of the above graph gives: D B E A C G F H Queue can be used to implement the Breadth-first traversal

28. void Tree::PrintBreadthFirstWithQueue(TreeStructPtr Root) { Queue que; TreeStructPtr p = Root; if (p != NULL) { que.push(p); while (!que.empty()) { p = que.pop(); cout Number "; if (p ->Left != NULL) que.push(p->Left); if (p->Right != NULL) que.push(p->Right); } Breadth-First Traversal with Queue

29. Stackless Depth-First Traversal There are also algorithms that can be used to do depth-first traversal of a tree without using stack or recursion. One of these methods is Threaded Trees Threads are pointers to the successors and predecessors of the nodes according to a particular traversal. For example, if we want to do in-order traversal, threads become pointers to the successors and predecessors of the nodes according to the in-order traversal. In this case every node’s left and right pointer is pointing to something. Even leaves of the tree point to something. If a node has a left child, its left pointer points to its left child; otherwise its left pointer is pointing to its predecessor If a node has a right child, its right pointer points to its right child; otherwise it points to its successor

30. An example of threaded tree for in-Order traversal is shown below The threads shown in broken blue color pointing to the successors and the threads shown in solid red color are pointing to the predecessors A G F E D C B J I H In order Traversal gives us : G, D, B, H, E, I, A, F, J, C

31. To do the in-order traversal, it is sufficient to keep the successor threads only. The broken red line shows the path of in-order traversal If a node has a right child, its successor pointer is NULL (Example: node B in the following tree) However, if the node does not have a right child, its right pointer points to its successor node (Example: Node A in the following tree) A B

32. TreeStructPtr p = Root; prev; If (p != NULL) { while (p->Left != NULL) p = p->Left; while (p ! = NULL) { cout Number "; } prev = p; p = p->Right; if (p != NULL && prev->Successor == NULL) while (p->Left != NULL) p = p->Left; } Here is the algorithm to do the stackless in-order traversal of a binary tree

33. Another method of stackless in-order traversal Using threaded tree is not the only way to do stackless in-order traversal of a binary search tree. Another stackless algorithm is called Morris method In this case, we change the structure of the tree such that the tree has no left descendants Then we traverse the tree, print the content Next, we charge the structure of the tree to its original shape. For in-order traversal of the tree, this algorithm is called MorrisInOrder

34. MorrisInOrder() { P = Root while not finished if P has no left descendant { visit it; P = P->Right; } else { Temp = the right child of the rightmost node in its left descendant of P Temp->Right = P; P = Root } P Temp P P

35. 10 7 3 20 5 P Temp 10 7 3 20 5 Temp P 10 7 3 20 5 P Temp 10 7 3 20 5 PTemp Now we can print the linked list. The nodes are organized in in-order form like a singular linked list Step 1Step 2Step 3 Step 4

36. 10 7 3 20 5 P 10 7 3 20 5 P Temp 10 7 3 20 5 P 10 7 3 20 5 Temp P 10 7 3 20 5 P Temp 10 7 3 20 5 P=NULL Temp Step 5Step 6Step 7 Step 8Step 9Step 10 Now the tree is back to its original shape after the traversal is done Temp

37. Insertions Binary trees are normally created by inserting nodes into an initially empty tree ● If tree empty, first insert will create the root node ● Otherwise, tack new node onto the (unique) appropriate node that has less than 2 children ● This will require a search routine

38. Insert 15, 8, 12, 19, 5, and 23 into an empty BST 8 15 8 12 19 8 15 12 23 19 8 15 12 5 Step 1Step 2Step 3 Step 4Step 5Step 6 19 8 15 12 5

39. // This procedure implements a recursive insert. // it can also be done iteratively. void insert (nodeptr& root, nodeptr newnode) { if (root == null) root = newnode; else if (newnode->data data) insert (root->left, newnode); else insert (root->right, newnode); }

40. void Tree::InsertWithNoRecursion(TreeStructPtr& Root, int x) { TreeStructPtr p = Root, prev=Root; while (p != NULL) {prev = p; if (x Number) p = p->Left; else if (x > p->Number) p = p->Right; else { cout << "\n!!!! " << x << " is already in the tree " << endl; return; } TreeStructPtr NewNode; NewNode = new (TreeStruct); NewNode->Number = x; NewNode->Left = NULL; NewNode->Right = NULL; if (Root = = NULL) Root = NewNode; else if (prev->Number < x) prev->Right = NewNode; else prev->Left = NewNode; return; } This is the implementation of the insert function without using Recursion

41. Deletions Deleting is a little more difficult. Why? When deleting from a BST, the algorithm must often reorganize tree while maintaining the inherent ordering Three cases to consider

42. 23 19 8 15 12 5 ` 23 19 8 15 12 5 ` 23 19 8 15 5 Case 1: Deleting a leaf node ● This is easy -- detach leaf node from tree by setting its parent’s pointer to nil ● For example, suppose we want to delete 12

43. Case 2: Deleting a node with a single child ● Easy -- replace the node to be deleted with its child For example, suppose we want to delete 19 23 19 8 15 12 5 23 19 8 15 12 5 23 8 15 12 5

44. 26 22 23 10 7 21 8 17 19 14 These nodes must be properly linked back into the tree Case 3: Deleting a node with two Children For example, suppose we want to delete 21

45. There are Two possible algorithms for delete Method 1: Replace the deleted node with one of its children, say the left child (i.e. node 17) The remaining right child (i.e. node 23) must now be linked back into the tree Place it on the right of the largest node of the new left subtree

46. 12 26 22 23 10 7 21 8 17 19 14 Replace the deleted node with one of its children, say the left child (i.e. node 17)

47. 12 26 22 23 10 7 8 17 19 14

48. 12 26 22 23 10 7 8 17 19 14 Place it (node 23) on the right of the largest node of the new left subtree

49. 12 23 19 10 7 17 8 14 Done! 23 22

50. Deletions Method 2: Locate the in-order successor of the node to be deleted (the node that contains the next highest value) To locate the in-order successor, go right, then go left as far as possible Replace the contents of the node to be deleted with the contents of its in-order successor Delete the successor node This algorithm is possible since the in-order successor node will never have more than one child

51. 12 26 22 23 10 7 21 8 17 19 14 Delete 21

52. 12 26 22 23 10 7 21 8 17 19 14 Replace the contents of the node to be deleted with the contents of its in-order successor

53. 12 26 21 23 10 7 22 8 17 19 14 Now swap the value of the node with the value of its successor node

54. Delete the successor node 12 26 21 23 10 7 22 8 17 19 14

55. Deletions 12 26 23 10 7 22 8 17 19 14 Done!

56. An alternative approach to the second method is to find predecessor of a node instead of the successor node. This mainly works the same as the second method Another issue to consider is that the second method does not change the structure of the tree as much as the first method does. ● The height of the tree does not change as much in the second method compare to the first method However, if we keep applying the second method for deletion and choose successor node to replace with the node, the tree eventually becomes unbalanced One way to improve the second method is we can write the algorithm to alternatively delete the predecessor of the node from the left subtree and delete the successor from the right subtree

57. Balancing a Binary Tree A binary tree is height-balanced or simply balanced if the difference between the left subtree and right subtree of any node is either 0 or 1 For example, consider the following binary tree: B R P K D M 330B 121K 011P DifferenceHeight of the right subtree Height of the left subtree Node Because there is at least one node (node B) with the difference value of 3, this tree is not balanced

58. Suppose we receive a set f data all in one shot and we want to insert them into a binary tree such that the end result is a balanced binary search tree. If we insert the data one by one randomly, the tree may not turn to be a balanced binary search tree. One method to solve this problem is to ● First sort the data using the best sort algorithm available ● Designate the middle element to be the root of the binary tree ● The array would consist of two sub arrays: ● one from the first element to the middle element (the root) but not including the middle element ● Another consists of middle + 1 till the last element ● Now the left child of the tree is taken from the middle of the first sub-array and its right child from the middle of the second sub-array ● Now divide the first sub-array into two other sub-arrays and repeat the same process ● Similarly divide the other sub-array into two other sub-arrays and repeat the same process

59. void Tree:: Balance (T data[ ], int first, int last) { if (first <=last) { int middle = (first + last) /2; insert (data[middle]); Balance (data, first, middle-1) Balance(data, middle+1, last); } The algorithm is:

60. Example: ● Stream of data:5 1 9 8 7 0 2 3 4 6 ● Sorted Data:0 1 2 3 4 5 6 7 8 9 4 Step 1 Step 2 0 1 2 3 4 5 6 7 8 9 1 4

61. Step 3 1 4 0 0 1 2 3 4 5 6 7 8 9 1 4 2 0 Step 4 0 1 2 3 4 5 6 7 8 9

62. Step 5 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 Step 6 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 7

63. Step 7 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 7 5 Step 8 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 7 5 6

64. Step 9 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 7 5 6 8 Step 10 0 1 2 3 4 5 6 7 8 9 1 4 2 0 3 7 5 6 8 9

65. What if we already have a balanced binary search tree and we want to insert another element such that after insertion the tree remains balanced One solution is to sort all elements including the new one again and then re-insert the elements again to the tree as we discussed before. Another solution proposed by C. Day and later improved by Q. Stout and B. Warren. Their solution is known as DSW algorithm DSW algorithm avoids sorting. It acquires deconstructing and reconstructing of the tree The building block for the tree transformation in this algorithm is the rotation

66. There are two types of rotations: Left rotation and Right Rotation which are symmetrical to one another The right rotation of a node called “Child” around its parent node called “Parent” is performed according to the following algorithm RotateRight (Grandparent, Parent, Child) { - If Parent is not the root of the tree Grandparent of child becomes child’s parent by replacing the parent - Right subtree of child becomes left subtree of child’s parent - Node child acquire parent as its right child }

67. Gr c a Par Ch b Gr c a Par Ch b The following is an example of right rotation Gr Ch Par Right rotation of Ch around Par Left rotation of Ch around Par Gr Ch Par a bc ab c

68. DSW algorithm transforms an arbitrary binary search tree into a linked list like tree called a backbone tree Then the backbone tree is transformed in a series of passes into a perfectly balanced tree by repeatedly rotating every second node of the backbone around its parent 5 30 10 20 40 15 2823 25 5 20 10 15 23 25 30 40 28 25 23 2030 40 10 15 5 28

69. In the first phase, the backbone is created using the following algorithm: createBackbone( root, n) { tmp = root; while (tmp != NULL) if tmp has a left child { rotate this child around tmp; set tmp to the child which just became parent } else set tmp to its right child }

70. 5 30 10 20 40 15 2823 25 tmp 530 10 20 40 15 282325 tmp 5 30 10 20 40 15 2823 25 tmp

71. 53010 20 40 15 282325 tmp 5 30 10 20 40 15 2823 25 tmp 5 30 10 20 40 15 28 23 25 tmp

72. 5 30 10 20 40 15 28 23 25 tmp 5 30 10 20 40 15 28 23 25 tmp 5 30 10 20 40 15 28 23 25 tmp

73. 5 30 10 20 40 15 28 23 25 tmp 5 30 10 20 40 15 28 23 25 tmp 5 30 10 20 40 15 28 23 25 tmp=NULL 5 30 10 20 40 15 28 23 25 tmp

74. In the second phase, the Backbone is transformed into a tree but this time the tree is perfectly balanced by having leaves on two adjacent levels In each pass down the backbone, every second node down to a certain point is rotated around its parent The algorithm is as follows: createPerfectTree(n) { m = 2 lg(n+1) -1 Make n – m rotations starting from the top of the backbone while (m >1) { m = m/2 make m rotations starting from the top of the backbone }

75. 5 20 10 15 23 25 30 40 28 n = 9 m = 7 So we need to do n – m = 2 rotations 5 20 10 15 23 25 30 40 28 m = 7/2 = 3 So we need to do three rotations 20 25 30 28 23 5 10 15 40 25 20 23 30 2840 5 10 15 m = 3/2 = 1 So we need to do one rotation m = 1/2 = 0 No more rotation is necessary Step 1Step 2Step 3 Step 4

76. Can you do this example now. 1 9 2 4 5 3 86 7 10 Some of the code in this lecture is placed in example 1 for the tree lecture in the web