Trees (Revisited) CHAPTER 15 6/30/15 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Chapter Contents 15.1 Case Study: Huffman Codes 15.2 Tree Balancing: AVL Trees Trees Red-Black Trees B-Trees Other Trees 15.4 Associative Containers in STL – Maps (optional) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Chapter Objectives  Show how binary trees can be used to develop efficient codes  Study problem of binary search trees becoming unbalanced  Look at classical AVL approach for solution  Look at other kinds of trees, including trees, red-black trees, and B-trees  Introduce the associate containers in STL and see how red-black trees are used in implementation Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Case Study: Huffman Codes  Recall that ASCII, EBCDIC, and Unicode use same size data structure for all characters  Contrast Morse code  Uses variable-length sequences  Some situations require this variable-length coding scheme  Can save space vs set data size Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Variable-Length Codes  Each character in such a code  Has a weight (probability) and a length  Expected length: expected length of the code for any one “codeword”  The expected length is the sum of the products of the weights and lengths for all the characters (0.2 x 2) + (0.1 x 4) + (0.1 x 4) + (0.15 x 3) + (0.45) x 1 = 2.1 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Immediate Decodability  When no sequence of bits that represents a character is a prefix of a longer sequence for another character  Can be decoded without waiting for remaining bits  Note how previous scheme is not immediately decodable  And this one is Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved CharacterCode A01 B10 C010 D0000

Huffman Codes  We seek codes that are  Immediately decodable  Each character has minimal expected code length  For a set of n characters { C 1.. C n } with weights { w 1.. w n }  We need an algorithm which generates n bit strings representing the codes Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Huffman's Algorithm 1. Initialize list of n one-node binary trees containing a weight for each character 2. Do the following n – 1 times a. Find two trees T' and T" in list with minimal weights w' and w" b. Replace these two trees with a binary tree whose root is w' + w" and whose subtrees are T' and T" and label points to these subtrees 0 and 1 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Huffman's Algorithm 3. The code for character C i is the bit string labeling a path in the final binary tree form from the root to C i Given characters The end with codes result is Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved CharacterHuffman Code A011 B000 C001 D010 E1

Overall Result  Each time we traverse a left branch, we add a 0 to the code  Each time we traverse a right branch, we add a 1 to the code  These variable length codes, take advantage of the frequencies of the symbols  This results in time and memory efficiencies Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Huffman Decoding Algorithm 1. Initialize pointer p to root of Huffman tree 2. While end of message string not reached do the following a. Let x be next bit in string b. if x = 0 set p equal to left child pointer else set p to right child pointer c. If p points to leaf i. Display character with that leaf ii. Reset p to root of Huffman tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Huffman Decoding Algorithm  For message string  Using Hoffman Tree and decoding algorithm Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Click for answer D E C A D E Initial Leaves:{ (A20) (B10) (C10) (D15) (E45) } Merge { (A20) ({BC} 20) (D15) (E45) } Merge { ( {AD}35 ) ( {BC}20 ) (E45) } Merge { ( {ADBC}55 ) (E45) } Merge { ( { ADBCE}100 ) }

Exercise #1: Huffman Tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Create the Huffman tree following the algorithm. Take the two lowest weight nodes and merge them 2. Label the vertices with the weight and letter 3. Label the 0/1 4. Create a table with overall encoding CharacterHuffman Code A0 B100 C1010 D1011 E1100 F1101 G1110 H ABCDEFGH Ref:

Tree Balancing: AVL Trees  Consider a BST of state abbreviations  When tree is nicely balanced, search time is O(log 2 n)  If abbreviations entered in order DE, GA, IL, IN, MA, MI, NY, OH, PA, RI, TX, VT, WY  BST degenerates into linked list with search time O(n) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

AVL Trees  A height balanced tree  Named with initials of Russian mathematicians who devised them  Georgii Maksimovich Adel’son-Vel’ski and Evgenii Mikhailovich Landis  Defined as  Binary search tree  Balance factor of each node is 0, 1, or -1 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved (Balance factor of a node = left height of sub tree minus right height of subtree)

AVL Trees  Consider linked structure to represent AVL tree nodes  Includes data member for the balance factor Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

AVL Trees  Insertions may require adjustment of the tree to maintain balance  Given tree  Becomes unbalanced  Requires right rotation on subtree of RI  Producing balanced tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Insert DE

Basic Rebalancing Rotations 1. Simple right rotation When inserted item in left subtree of left child of nearest ancestor with factor Simple left rotation When inserted item in right subtree of right child of nearest ancestor with factor Left-right rotation When inserted item in right subtree of left child of nearest ancestor with factor Right-left rotation When inserted item in left subtree of right child of nearest ancestor with factor -2 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Basic Rebalancing Rotations  Rotations carried out by resetting links  Right rotation with A = nearest ancestor of inserted item with balance factor +2 and B = left child  Reset link from parent of A to B  Set left link of A equal to right link of B  Set right link of B to point A  Next slide shows this sequence Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Basic Rebalancing Rotations Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Basic Rebalancing Rotations Right-Left rotation Item C inserted in right subtree of left child B of nearest ancestor A 1. Set left link of A to point to root of C 2. Set right link of B equal to left link of C 3. Set left link of C to point to B Now the right rotation 4. Reset link from parent of A to point to C 5. Set link of A equal to right link of C 6. Set right link of C to point to A Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Basic Rebalancing Rotations  When B has no right child before node C insertion Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Another way to look at rotations… Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Exercise #2: Apply a Left-Rotate operation on node 15

Exercise #3: Create an AVL Tree  Create an AVL tree using the following numbers:  4, 8, 12, 1, 2, 3  Show each step/rotation Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Steps 1-3: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  Step 1: Initial tree created with value 4. By default this is balanced  Step 2: The node 8 is added which is greater than 4, so it becomes a right node. It by itself is balanced, but at the root, there is no left subtree. Left subtree = 0, Right subtree = 1 so root balance factor = -1  Step 3: The node 12 is added, which is greater than 4 and 8, so it becomes the right node of 4. It by itself is balanced, but 8 now has an unbalanced subtree because it does not have a left subtree. Now the root, 4, has no left subtree but a height of two on it’s right subtree, so it’s balance factor is 0 – 2 =

Step 4: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  Step 4: Because we inserted a right child on the right subtree that caused the imbalance, we need to apply a left rotation. The imbalance occurs at the root (4) so therefore we rotate at node (8)  The result is a balanced tree with 8 as the new root

Step 5: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  Next we insert node (1) to the left of 4. We now have an additional level of height in the left subtree, but no additional rotation is needed yet.  Next, we insert node (2). This node is less than 8, less than 4 and greater than 1, so it gets attached as the right child in the left subtree. As you can see we now have an imbalance which we will handle by a R-L rotation next

Step 6: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  The imbalance is at node (4), so therefore the rotation will be at node (1). First we apply a left rotation  We can see after the left rotation, that there is still an imbalance, which is why we need to apply the right rotation at the location where node (2) now resides

Step 7: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  The imbalance is still at node (4), and we need to complete our second step of our L-R rotation, so therefore the rotation will be at node (2). We apply the second step, the Right rotation.  We can now see that the tree is again balanced

Step 8: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  On our last step, we insert node 3. This is less than 8, but greater than 2 and less than 4, so it becomes the left child of 4.  Left-right rotation When inserted item in right subtree of left child of nearest ancestor with factor +2  Note that the imbalance occurs at the root, and that the offending insertion was on the right subtree (4 – 3) of the left child (2) of the nearest ancestor with a factor of 2 (node 8). So therefore we need a L-R rotation with the rotation occurring at node(2).  We apply the first part, the left rotation

Step 9: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved  Finally we need to apply the last part, the “right” rotation (of the L- R rotation). This rotation is applied at node (4).  This is the last step. We end with a balanced tree

Non Binary Trees  Some applications require more than two children per node  Genealogical tree  Game tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

2-3-4 Trees  We extend searching technique for BST  At a given node, we now have multiple paths a search path may follow  Define m-node in a search tree  Stores m – 1 data values k 1 < k 2 … < k m-1  Has links to m subtrees T 1..T m  Where for each i all data values in T i < k i ≤ all values in T k+1 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

2-3-4 Trees  Example of m-node  Define tree  Each node stores at most 3 data values  Each internode is a 2-node, a 3-node, or a 4-node  All the leaves are on the same level Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

What is a 2, 3 or 4 node?  A 2-node will have one data element and two separate child nodes  Like a binary search tree  A 3-node will have two data elements and 3 separate child nodes  A 4-node will have three data elements (this is the max), and 4 separate child nodes  Note that the child nodes represent values in between the parent node values Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

2-3-4 Trees  Example of a node which stores integers  To search for 36  Start at root … 36 < 53, go left  Next node has 27 < 36 < 38. take middle link  Find 36 in next lowest node Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Building a Balanced Tree If tree is empty  Create a 2-node containing new item, root of tree Otherwise 1. Find leaf node where item should be inserted by repeatedly comparing item with values in node and following appropriate link to child 2. If room in leaf node Add item Otherwise … (ctd) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Building a Balanced Tree Otherwise: a. Split 4-node into 2 nodes, one storing items less than median value, other storing items greater than median. Median value moved to a parent having these 2 nodes as children b. If parent has no room for median, spilt that 4- node in same manner, continuing until either a parent is found with room or reach full root c. If full root 4-node split into two nodes, create new parent 2-node, which = the new root Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Exercise #4: Building a Balanced Tree  Take the following list of numbers and using the algorithm, construct a tree  53, 27, 75, 25, 70, 41, 38, 16, 59, 36, 73, 65, 60, 46, 55, 33, 68, 79, 48 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Building a Balanced Tree  Note that last step in algorithm may require multiple splits up the tree  Instead of bottom-up insertion  Can do top-down insertion  Do not allow any parent nodes to become 4-nodes  Split 4-nodes along search path as encountered Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Note that trees stay balanced during insertions and deletions

Implementing Trees class Node234 { public DataType data[3]; // type of data item in nodes Node234 * child[4]; // Node234 operations }; typedef Node234 * Pointer234 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Red-Black Trees  Defined as a BST which:  Has two kinds of links (red and black)  Every path from root to a leaf node has same number of black links  No path from root to leaf node has more than two consecutive red links  Data member added to store color of link from parent Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Red-Black Trees enum ColorType {RED, BLACK}; class RedBlackTreeNode { public: DataType data; ColorType parentColor; // RED or BLACK RedBlackTreeNode * parent; RedBlackTreeNode * left; RedBlackTreeNode * right; }  Now possible to construct a red-black tree to represent a tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Red-Black Trees  tree represented by red-black trees as follows:  Make a link black if it is a link in the tree  Make a link red if it connects nodes containing values in same node of the tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Red-Black Trees  Example: given tree Represented by this red-black tree Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Constructing Red-Black Tree  Do top-down insertion as with tree 1. Search for place to insert new node – Keep track of parent, grandparent, great grandparent 2. When 4-node q encountered, split as follows: a. Change both links of q to black b. Change link from parent to red: Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Constructing Red-Black Tree 3. If now two consecutive red links, (from grandparent gp to parent p to q ) Perform appropriate AVL type rotation determined by direction (left-left, right-right, left- right, right-left) from gp -> p-> q Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

B-Trees  Previous trees used in internal searching schemes  Tree sufficiently small to be all in memory  B-tree useful in external searching  Data stored in 2ndry memory  B-tree has properties  Each node stores at most m – 1 data values  Each internal nodes is 2-node, 3-node, …or m-node  All leaves on same level Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

B-Trees  Thus a tree is a B-tree of order 4  Note example below of order 5 B-tree  Best performance found to be with values for 50 ≤ m ≤ 400 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

Representing Trees & Forests as Binary Trees  Consider a node with multiple links const int MAX_CHILDREN = … ; class TreeNode { public: DataType data TreeNode * child[MAX_CHILDREN]; // TreeNode operations } typedef TreeNode * TreePointer; Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Note problem with wasted space when not all links used

Representing Trees & Forests as Binary Trees  Use binary (two link) tree to represent multiple links  Use one of the links to connect siblings in order from left to right  The other link points to first node in this linked list of its children Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Note right pointer in root always null

Representing Trees & Forests as Binary Trees  Now possible to represent collection of trees (a forest!)  This collection becomes Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved