1. AVL Trees and Heaps

2. AVL Trees So far balancing the tree was done globally Basically every node was involved in the balance operation Tree balancing can however be done locally if only a portion of the tree is affected. An example of this is called AVL tree AVL tree was proposed by Adel’son, Vel’skii and Landis

3. AVL Trees An AVL tree is one in which the height of the left and right sub-trees of every node differ by at most one For example consider the following tree 19| 0 8| +1 15| -1 12| 0 There are two values in each node: one is the actual value of the node and the other is called balancing factor of the node. The balancing factor of every node in an AVL tree is either 0, +1, or -1. For each node, P, the balancing factor of P = height of right sub-tree of P – height of left sub-tree of P

4. AVL Trees -- Continue If a new node is inserted to an AVL tree or a node is deleted from an AVL tree, the balancing factor of some nodes changes. In order to make the new modified tree an AVL tree again we may have to rotate some nodes Insertion or deletion may cause the balance factor of some nodes to be +2 or -2. These nodes have to be tracked and based on appropriate algorithms they should be properly rotated to right/left to make the tree a balanced AVL tree

5. Inserting into an AVL Tree Algorithm: Insert the new node, node N, into the tree the same way you insert N into a binary search tree Follow the path from node N to the root. This is your insertion path Insertion of node N may change the balancing factors of some nodes in the insertion path After inserting node N, start updating the balancing factors of the nodes in the insertion path until you reach the first node with balancing factor of + 2 or -2. If such a node does not exist in the insertion path, the tree is already balanced; otherwise mark that node, node P and go to the next step

6. R = P -> Right L = P -> Left If the new node that was just inserted, Node N, is in the right sub-tree of R Left Rotate R around P else if Node N is in the left sub-tree of R C = R ->Left Right rotate C around R Left rotate C around P else if Node N, is in the left sub-tree of L Right Rotate L around P else if Node N is in the right sub-tree of L C = L ->Right Left rotate C around L Right rotate C around P Case 1 Case 2 Case 3 Case 4

7. L R P C C...... Right rotate of C around R then, left rotate of C around P Left rotate of R around P Right rotate of L around P Left rotate of C around L then, right rotate of C around P Suppose node P is the first node on the insertion path in which its balance factor is changed to +2 or -2

8. Example of Case 1: 23| 0 19| +1 8 | 0 15| 0 12 | 0 5 | 0 23| ? 19| ? 8 | 0 15| ? 12 | 0 5 | 0 30| 0 Step 1 Step 2 Original Tree Inserting the node 30 into the tree

9. 23| +1 19| +2 8 | 0 15| ? 12 | 0 5 | 0 30| 0 Stop Here P R 23| 0 19| 0 8 | 0 15| 0 12 | 0 5 | 0 30| 0 R P Step 3 Step 4 Following the insertion path to find node P with balancing factor +2 or -2 R is rotated around P to balance the tree (case 1)

10. Example of Case 2: 23| 0 19| +1 8 | 0 15| 0 12 | 0 5 | 0 23| ? 19| ? 8 | 0 15| ? 12 | 0 5 | 0 21| 0 Original Tree Inserting the node 21 into the tree Step 1 Step 2

11. 23| -1 19| +2 8 | 0 15| ? 12 | 0 5 | 0 21 | 0 Stop Here P R C 21| 0 19| 0 8 | 0 15| 0 12 | 0 5 | 0 23| 0 C P R 21| ? 19| ? 8 | 0 15| ? 12 | 0 5 | 0 23| ? R P C Following the insertion path to find node P with balancing factor +2 or -2 Making a double rotations to balance the tree (case 2) Step 3 Step 4

12. Example of Case 3: 19| 0 8 | 0 15| -1 12 | 0 5 | 0 19| 0 8 | ? 15| ? 12 | 0 5 | ? 2| 0 Step 1 Step 2 Original Tree Inserting the node 2 into the tree

13. P L 5 | -1 8| 0 12 | 0 2 | 0 P Step 3 Step 4 Following the insertion path to find node P with balancing factor +2 or -2 L is rotated around P to balance the tree (case 3) 19| 0 8 | -1 15| -2 12 | 0 5 | -1 2| 0 Stop Here 15| 0 19| 0 L

14. Example of Case 4: 19| 0 8 | -1 15| -1 5 | 0 19| 0 8 | ? 15| ? 5 | ? 6| 0 Step 1 Step 2 Original Tree Inserting the node 6 into the tree

15. Step 3 Step 4 Following the insertion path to find node P with balancing factor +2 or -2 P L 19| 0 8 | -2 15| ? 5 | +1 6 | 0 Stop Here C P L 19| 0 6 | 0 15| -1 5 | 0 8 | 0 C Making a double rotations to balance the tree (case 4) P L 19| 0 8 | ? 15| ? 6 | ? 5 | ? C

16. Inserting a series of numbers into a AVL Tree We want to insert numbers 2, 8, 6, 10, 15, 4, 3, 20, 13, 18 2 | 0 Insert 2 Insert 8 2 | +1 8 | 0 No rotation is required because there is no node with balance factor of +2 or -2

17. Insert 6 Insert 10 2 | +2 8 | -1 6 | 0 2 | +26 | +18 | 0 P R C 6 | 08 | 02 | 0 P C R C R P Double Rotation is required 6 | +1 8 | +1 2 | 0 10 | 0 No rotation is required because there is no node with balance factor of +2 or -2

18. Insert 15 Insert 4 6 | ? 8 | +2 2 | 0 10 |+1 15 | 0 P R 6 | +1 10 | 0 2 | 0 15 |0 8 | 0 R P 6 | 0 10 | 0 2 | +1 15 |0 8 | 0 4 | 0 No rotation is required because there is no node with balance factor of +2 or -2 Single rotation is required

19. Insert 3 6 | ? 10 | 0 2 | +2 15 |0 8 | 0 4 | -1 3 | 0 P R C 6 | ? 10 | 0 2 | +2 15 |0 8 | 0 4 | 0 3 | +1 P R C 6 | 0 10 | 0 3 | 0 15 |0 8 | 0 2 | 0 4 | 0 P R C

20. Insert 20 6 | +1 10|+1 3 | 0 15|+1 8 | 0 2 | 0 4 | 0 20 |0 Insert 13 6 | +1 10|+1 3 | 0 15|0 8 | 0 2 | 0 4 | 0 20 |0 13 | 0 No rotation is required because there is no node with balance factor of +2 or -2

21. Insert 18 6 | +1 10|+2 3 | 0 15|+1 8 | 0 2 | 0 4 | 0 20 |-1 13 | 0 18 | 0 R L P 6 | +1 15|0 3 | 0 20|-1 10 | 0 2 | 0 4 | 0 8 |-1 13 | 018 | 0 Single rotation is required

22. Deleting From an AVL Tree This can be more time consuming than insertion. First we need to apply a delete method to get rid of the node We can use the delete method that finds either the successor or predecessor node to replace a node that has both left and right children After the node is deleted, the balance factors are updated from the parent of the deleted node up to the root. This is the “delete path” For each node on the delete path whose balance factor becomes +2 or -2, a single or a double rotations has to be performed to restore the balance of the tree IMPORTANT NOTE: The balancing of the tree DOES NOT STOP after the first node P with balancing factor +2 or -2 found in the delete path. You still need to continue searching for other nodes with balancing factor +2 or -2

23. Delete Algorithm -- AVL Tree 1.Start from the parent of the deleted node and move up the tree (delete path) 2.If you find a node with balance factor +2 or -2 then stop and do the balancing as follows 3.Mark the node P 4.If the deleted node is in the left sub-tree of P Q = P -> Right If balance factor of Q is +1 or 0 rotate Q around P else // the balance factor is -1 R = Q -> Left rotate R around Q rotate R around P endif 1.else if the …. > Case 1 Case 2

24. Delete Algorithm -- AVL Tree – Cont. 1.else if the deleted node is in the right sub-tree of P Q = P -> Left if balance factor of Q is -1 or 0 rotate Q around P else // the balance factor is +1 R = Q -> Right rotate R around Q rotate R around P end if 2.Continue moving up the tree on the delete path. If you find another node with balancing factor +2 or -2 go to step 3; otherwise, you are done Case 3 Case 4

25. Example of Case 1: 25| 0 20| +1 5 | 0 10| +1 8 | 0 3 | 0 Step 1 Original Tree Delete 10 Delete method: Find successor of 10 which is 15 15| 0 30| 0 23| 0 25| 0 20| +1 5 | 0 10| +1 8 | 0 3 | 0 Step 2 15| 0 30| 0 23| 0

26. Start from node 20 (the parent of deleted node) and find the first node with balancing factor +2 or -2 25| 0 20| +2 5 | 0 15| ? 8 | 0 3 | 0 Step 4 30| 0 23| 0 25| 0 20| ? 5 | 0 15| ? 8 | 0 3 | 0 10| 0 30| 0 23| 0 Swap 10 and 15 And Delete 10 P Step 3

27. Because the deleted node was in the left side of P, we have Q = P -> Right Note: Balance factor of Q is 0 Step 6 25| -1 23| 0 5 | 0 15| +1 8 | 0 3 | 0 30| 0 20|+1 Q 25| 0 20| +2 5 | 0 15| ? 8 | 0 3 | 0 30| 0 23| 0 P Q Because the balancing factor of Q was 0, we rotate Q around P Step 5 P

28. Example of Case 2: 25| -1 20| +1 5 | 0 10| +1 8 | 0 3 | 0 Step 1 Original Tree Delete 10 Delete method: Find successor of 10 which is 15 15| 0 23| 0 25| -1 20| +1 5 | 0 10| +1 8 | 0 3 | 0 Step 2 15| 0 23| 0

29. Start from node 20 (the parent of deleted node) and find the first node with balancing factor +2 or -2 25| -1 20| +2 5 | 0 15| ? 8 | 0 3 | 0 Step 4 23| 0 25| -1 20| ? 5 | 0 15| ? 8 | 0 3 | 0 10| 0 23| 0 Swap 10 and 15 And Delete 10 P Step 3

30. Because the deleted node was in the left side of P, we have Q = P -> Right Note: Balance factor of Q is -1, so R = Q -> Left Because the balancing factor of Q was -1, we do two rotations 25| -1 20| +2 5 | 0 15| ? 8 | 0 3 | 0 23| 0 P Q Step 5 R Step 6 20| ? 25| ? 5 | 0 15| ? 8 | 0 3 | 0 23| ? P R Q 25| 0 23| 0 5 | 0 15| 0 8 | 0 3 | 0 20| 0 R Q P

31. Example of Case 3: 25| 0 20| 0 5 | -1 10| -1 8 | 0 3 | 0 Step 1 Original Tree Delete 10 Delete method: Find predecessor 10 which is 8 15| 0 4| 0 1 | 0 25| 0 20| 0 5 | -1 10 | -1 8 | 0 3 | 0 Step 2 15| 0 4| 0 1 | 0

32. Start from node 5 (the parent of deleted node) and find the first node with balancing factor +2 or -2 Step 4 Swap 10 and 8 and Delete 10 Step 3 25| 0 20| 0 5 | ? 8| ? 10 | 0 3 | 0 15| 0 4 | 0 1 | 0 25| 0 20| 0 5 | -2 8| ? 3 | 0 15| 0 4 | 0 1 | 0 P

33. Because the deleted node was in the right side of P, we have Q = P -> Left Note: Balance factor of Q is 0 Step 6 Because the balancing factor of Q was 0, we rotate Q around P Step 5 25| 0 20| 0 5 | -2 8| ? 3 | 0 15| 0 4 | 0 1 | 0 P Q 25| 0 20| 0 5 | -1 8| -1 3 | +1 15| 0 4 | 0 1 | 0 P Q

34. Example of Case 4: 25| 0 20| 0 5 | -1 10| -1 8 | 0 3 | +1 Step 1 Original Tree Delete 8 Delete method: Because node 8 is a leaf, it can be simply deleted 15| 0 4 | 0 Step 2 25| 0 20| 0 5 | ? 10| ? 8 | 0 3 | +1 15| 0 4 | 0

35. Start from node 5 (the parent of deleted node) and find the first node with balancing factor +2 or -2 Step 4 Step 3 25| 0 20| 0 5 | -2 10| ? 3 | +1 15| 0 P 25| 0 20| 0 5 | -2 10| ? 3 | +1 15| 0 4 | 0 P Q Because the deleted node was in the right side of P, we have Q = P -> Left Note: Balance factor of Q is +1, so R = Q -> Right 4 | 0 R

36. 25| 0 20| 0 5 | ? 10| ? 4 | ? 15| 0 3 | ? P Q R 25| 0 20| 0 4 | 0 10| 0 3 | 0 15| 0 5 | 0 P Q R Because the balancing factor of Q was +1, we do two rotations Step 5

37. Priority Queue A priority queue is a data structure that schedules the items in a queue based on certain priority Thus an item that is just arrived may not necessarily be placed at the end of the queue and the item that is removed from the queue may not be the one that has been in the queue the longest For example, in a multi-tasking environment the operating system scheduler may place the arriving programs (tasks) in a priority queue based on their size, or their importance. Smaller tasks do not have to wait for longer ones to finish first Important programs can get a chance to be executed quickly faster than the ones that are not relatively important

38. Two main operations of a priority queue are insert and deleteMin (deleting the minimum value from the queue) Several methods can be used to implement a priority queue: Simple linked list: Insert: can be done very fast. Just insert on the top O(1) deleteMin: may require searching the entire list O(n) Sorted linked list: Insert: may require traveling through the entire linked list O(n) deleteMin: can be done in constant time O(1) Ordered Binary Tree: Insert: can be done in O(log n) if tree is balanced deleteMin: can be done in O(log n) if tree is balanced Problem: a sequence of deleteMin can change the balance of the tree because the minimum is always on the left subtree

39. Heaps A particular kind of binary tree called “min-Heap” or just “Heap” has the following properties: The value of each node is less than the values stored in each of its children The tree is perfectly balanced and leaves in the last level are all in the leftmost position Similarly, in a “max Heap” tree the value of any node is greater than the values of its children. Therefore, the root has the highest value in the tree.

40. An example of a Min-Heap A E D C B I H J G F JIHGFEDCBA 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Note that for every element i, its left child is located at 2*i and its right child is located at 2*i+1 and its parent is at location i/2

41. Inserting into a Max-Heap Tree To add an element, the element is added at the end of the heap as the last leaf Restoring the heap property in the case of inserting a new node is achieved by moving from the last node (the node that was just added) to the root. This is called percolating up the tree Thus to insert a new node, node P, to the heap Put P at the end of the heap While P is not the root and value of P > value of parent (p) Swap P with its parent

42. Example: Insert 15 to the following heap tree 20 7 8 18 10 5 2 6 14 13 20 7 8 18 10 5 2 6 14 13 15 Insert 15 (call it node P): 15 is added as the last leaf. Now we need to swap P with its parent because P’s value is greater P 20 7 8 18 10 5 2 6 14 13 15 P

43. 20 15 8 18 10 5 2 6 14 13 7 20 10 8 18 15 5 2 6 14 13 7 P is still greater than its parent. Thus we need to do another swap P P is not greater than its parent anymore. So we are done P

44. Another way of looking at it (array form) Lets consider the Min-Heap this time To insert an element P into a heap, Create a hole in the next available location If P can be place in the hole without violating heap order, then we do so and we are done; Otherwise, we slide the element that is in the hole’s parent into the hole, thus bubbling the hole up toward the root We continue until P can be placed into the hole

45. 13 31 24 16 21 26 65 32 68 19 13 24 16 21 26 65 32 68 19 31 21 13 24 16 26 65 32 68 19 31 21 13 24 16 14 2665 32 68 19 31 Trying to insert 14 into the following heap 14 < 21 – move parent to the hole 14 > 13 – move 14 to the hole 14 < 31 – move parent to the hole

46. Insert Algorithm void BinaryHeap::insert(const int& x) { if (isfull()) throw Overflow(); // percolate up int hole = ++currentsize; // creating a new hole (position) for (; hole > 1 && x < array[hole/2]; hole=hole/2) array[hole] = array [hole/2]; // bubbling the hole up array[hole] = x; // Placing the value into the hole } Compares the value with its parent

47. Algorithm for Deleting From a Max-Heap Tree Locate the node to be deleted and call it node P Locate the last node in the heap Swap the values of P with the last node Delete the last node While P is not a leaf, and P < any of its children Swap P with the larger child

48. Example of deleting an element from a max-heap tree 20 7 8 15 10 5 2 6 14 13 6 7 8 15 10 5 2 20 14 13 We swap the last element with 20 first Suppose we want to delete 20

49. 6 7 8 15 10 5 2 20 14 13 Swap P with its larger child if the value of P is lower than the larger child Now we delete 20 P 6 7 8 15 10 5 2 14 13 P 15 7 8 6 10 5 2 14 13 P

50. Swap P with its larger child if the value of P is lower than the larger child 15 7 8 6 10 5 2 14 13 P 15 7 8 14 10 5 2 6 13 P

51. Other operations include: DescreaseKey(p, delta): Lowers the values of the item at position p by the positive amount “delta” This may violate the heap. We may need to percolate up to restore the heap in a min-heap IncreaseKey(p, delta): Increases the values of the item at position p by the positive amount “delta” This may violate the heap. We may need to percolate down to restore the heap in a min-heap Remove(p) To remove an item in position p, we can Apply Decreasekep(p, infinity) to move it up to the root Apply deleteMin to remove it from the min-heap

52. 13 21 19 16 14 2665 32 68 19 31 Trying to delete 13 from the heap 21 19 16 14 26 65 32 68 19 31

53. 14 21 19 16 26 65 32 68 19 14 19 21 16 26 65 32 68 19 Both children, 14 and 16 are smaller than 31 – we move 14 to the hole Both children, 19 and 21 are smaller than 31 – we move 19 to the hole 31

54. 14 19 26 21 16 65 32 68 19 14 19 26 21 16 31 65 32 68 19 One child, 26, is smaller than 31 – we move 26 to the hole This is the final tree after 13 is deleted and heap is restored 31

55. deleteMin Algorithm void BinaryHeap::deleteMin (int& minItem) { if (isEmpthy()) throw Underflow(); minItem = array[1]; array[1] = array [currentsize - -]; // moves the value of the last node into the first node percolateDown(1); } //--------------------------------------------------------------------------------------------------------------------- void BinaryHeap ::precolateDown(int hole) { int child; int tmp = array[hole]; // stores the value of the last element in the heap into a temp variable for (; hole*2 <=currentSize; hole=child) { child = hole*2; // sets the child to the left child if (child != currentSize && array[child+1] < array[child]) // compares left with right child array[hole] = array[child]; // bubbling the hole down the tree else break; } array[hole] = tmp; // hole is found and the last item is placed there }

56. Another way of looking at it (array form) Lets again consider the Min-Heap For the deleteMin operation Since the minimum is at the root, removing the minimum creates a hole at the root of the tree. Let P be the last element in the heap If P can be place in the hole without violating heap order, then we do so and we are done; (This is unlikely) Otherwise, compare the children of the hole and move the smaller one to the hole bubbling the hole down toward the bottom We continue until P can be placed into the hole