Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9.

Slides:



Advertisements
Similar presentations
Chapter 13. Red-Black Trees
Advertisements

Comp 122, Spring 2004 Binary Search Trees. btrees - 2 Comp 122, Spring 2004 Binary Trees  Recursive definition 1.An empty tree is a binary tree 2.A node.
BY Lecturer: Aisha Dawood. Heapsort  O(n log n) worst case like merge sort.  Sorts in place like insertion sort.  Combines the best of both algorithms.
Analysis of Algorithms
Jan Binary Search Trees What is a search binary tree? Inorder search of a binary search tree Find Min & Max Predecessor and successor BST insertion.
Analysis of Algorithms CS 477/677 Binary Search Trees Instructor: George Bebis (Appendix B5.2, Chapter 12)
Binary Search Trees Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 10.
1 Brief review of the material so far Recursive procedures, recursive data structures –Pseudocode for algorithms Example: algorithm(s) to compute a n Example:
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 12.
Red-Black Trees CIS 606 Spring Red-black trees A variation of binary search trees. Balanced: height is O(lg n), where n is the number of nodes.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 11.
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2009 Heap Lecture Chapter 6 Use NOTES feature to see explanation.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.
CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.
Analysis of Algorithms CS 477/677
Analysis of Algorithms CS 477/677 Red-Black Trees Instructor: George Bebis (Chapter 14)
Comp 122, Spring 2004 Heapsort. heapsort - 2 Lin / Devi Comp 122 Heapsort  Combines the better attributes of merge sort and insertion sort. »Like merge.
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 7 Heapsort and priority queues Motivation Heaps Building and maintaining heaps.
1 COSC 2P03 Lecture #5 – Trees Part III, Heaps. 2 Today Take up the quiz Assignment Questions Red-Black Trees Binary Heaps Heap sort D-Heaps, Leftist.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2007 Heap Lecture Chapter 6 Use NOTES feature to see explanation.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Heap Lecture 2 Chapter 7 Wed. 10/10/01 Use NOTES feature to.
Heapsort CIS 606 Spring Overview Heapsort – O(n lg n) worst case—like merge sort. – Sorts in place—like insertion sort. – Combines the best of both.
Design & Analysis of Algorithms Unit 2 ADVANCED DATA STRUCTURE.
Balanced Trees Balanced trees have height O(lg n).
Sorting Algorithms (Part II) Slightly modified definition of the sorting problem: input: A collection of n data items where data item a i has a key, k.
David Luebke 1 9/18/2015 CS 332: Algorithms Red-Black Trees.
Ch. 6: Heapsort n nodes. Heap -- Nearly binary tree of these n nodes (not just leaves) Heap property If max-heap, the max-heap property is that for every.
Heaps, Heapsort, Priority Queues. Sorting So Far Heap: Data structure and associated algorithms, Not garbage collection context.
Red-Black Trees CS302 Data Structures Dr. George Bebis.
Mudasser Naseer 1 10/20/2015 CSC 201: Design and Analysis of Algorithms Lecture # 11 Red-Black Trees.
Computer Sciences Department1. Sorting algorithm 3 Chapter 6 3Computer Sciences Department Sorting algorithm 1  insertion sort Sorting algorithm 2.
Lecture 10 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea.
2IL50 Data Structures Fall 2015 Lecture 7: Binary Search Trees.
CS 473Lecture X1 CS473-Algorithms Lecture RED-BLACK TREES (RBT)
1 Analysis of Algorithms Chapter - 03 Sorting Algorithms.
David Luebke 1 12/23/2015 Heaps & Priority Queues.
Computer Algorithms Lecture 9 Heapsort Ch. 6, App. B.5 Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR.
Heapsort. What is a “heap”? Definitions of heap: 1.A large area of memory from which the programmer can allocate blocks as needed, and deallocate them.
Chapter 6: Heapsort Combines the good qualities of insertion sort (sort in place) and merge sort (speed) Based on a data structure called a “binary heap”
Red-Black Trees. Review: Binary Search Trees ● Binary Search Trees (BSTs) are an important data structure for dynamic sets ● In addition to satellite.
Red Black Trees. History The concept of a balancing tree was first invented by Adel’son-Vel’skii and Landis in They came up with the AVL tree. In.
David Luebke 1 2/5/2016 CS 332: Algorithms Introduction to heapsort.
Analysis of Algorithms CS 477/677 Lecture 8 Instructor: Monica Nicolescu.
COSC 3101A - Design and Analysis of Algorithms 3 Recurrences Master’s Method Heapsort and Priority Queue Many of the slides are taken from Monica Nicolescu’s.
Analysis of Algorithms CS 477/677 Red-Black Trees Instructor: George Bebis (Chapter 14)
CS6045: Advanced Algorithms Sorting Algorithms. Heap Data Structure A heap (nearly complete binary tree) can be stored as an array A –Root of tree is.
Sept Heapsort What is a heap? Max-heap? Min-heap? Maintenance of Max-heaps -MaxHeapify -BuildMaxHeap Heapsort -Heapsort -Analysis Priority queues.
Heapsort Lecture 4 Asst. Prof. Dr. İlker Kocabaş.
"Teachers open the door, but you must enter by yourself. "
Binary Search Trees What is a binary search tree?
Heaps, Heapsort, and Priority Queues
Heaps, Heap Sort and Priority Queues
Red Black Trees
CS200: Algorithms Analysis
Red-Black Trees.
CS200: Algorithm Analysis
CMSC 341 (Data Structures)
Lecture 9 Algorithm Analysis
Lecture 9 Algorithm Analysis
Lecture 9 Algorithm Analysis
"Teachers open the door, but you must enter by yourself. "
CS 583 Analysis of Algorithms
Analysis of Algorithms CS 477/677
Binary Search Trees Comp 122, Spring 2004.
Chapter 12&13: Binary Search Trees (BSTs)
Red-Black Trees CS302 Data Structures
Presentation transcript:

Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9

CS 477/677 - Lecture 9 A Job Scheduling Application Job scheduling –The key is the priority of the jobs in the queue –The job with the highest priority needs to be executed next Operations –Insert, remove maximum Data structures –Priority queues –Ordered array/list, unordered array/list

CS 477/677 - Lecture 9 PQ Implementations & Cost Worst-case asymptotic costs for a PQ with N items InsertRemove max ordered array ordered list unordered array unordered list N N N N Can we implement both operations efficiently?

CS 477/677 - Lecture 9 Background on Trees Def: Binary tree = structure composed of a finite set of nodes that either: –Contains no nodes, or –Is composed of three disjoint sets of nodes: a root node, a left subtree and a right subtree root Right subtree Left subtree

CS 477/677 - Lecture 9 Special Types of Trees Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2. Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2. Full binary tree Complete binary tree

CS 477/677 - Lecture 9 The Heap Data Structure Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (heap) property: for any node x Parent(x) ≥ x Heap It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 2

CS 477/677 - Lecture 9 Definitions Height of a node = the number of edges on a longest simple path from the node down to a leaf Depth of a node = the length of a path from the root to the node Height of tree = height of root node =  lgn , for a heap of n elements Height of root = 3 Height of (2)= 1 Depth of (10)= 2

CS 477/677 - Lecture 9 Array Representation of Heaps A heap can be stored as an array A. –Root of tree is A[1] –Left child of A[i] = A[2i] –Right child of A[i] = A[2i + 1] –Parent of A[i] = A[  i/2  ] –Heapsize[A] ≤ length[A] The elements in the subarray A[(  n/2  +1).. n] are leaves The root is the maximum element of the heap

CS 477/677 - Lecture 9 Heap Types Max-heaps (largest element at root), have the max-heap property: –for all nodes i, excluding the root: A[PARENT(i)] ≥ A[i] Min-heaps (smallest element at root), have the min-heap property: –for all nodes i, excluding the root: A[PARENT(i)] ≤ A[i]

CS 477/677 - Lecture 9 Operations on Heaps Maintain the max-heap property –MAX-HEAPIFY Create a max-heap from an unordered array –BUILD-MAX-HEAP Sort an array in place –HEAPSORT Priority queue operations

CS 477/677 - Lecture 9 Operations on Priority Queues Max-priority queues support the following operations: –INSERT (S, x) : inserts element x into set S –EXTRACT-MAX (S) : removes and returns element of S with largest key –MAXIMUM (S) : returns element of S with largest key –INCREASE-KEY (S, x, k) : increases value of element x ’s key to k (assume k ≥ current key value at x )

CS 477/677 - Lecture 9 Building a Heap Alg: BUILD-MAX-HEAP (A) 1.n = length[A] 2. for i ←  n/2  downto 1 3. do MAX-HEAPIFY (A, i, n) Convert an array A[1 … n] into a max-heap ( n = length[A] ) The elements in the subarray A[(  n/2  +1).. n] are leaves Apply MAX-HEAPIFY on elements between 1 and  n/2  A:

CS 477/677 - Lecture 9 Example: A i = 5 i = 4 i = 3 i = 2 i = 1

CS 477/677 - Lecture 9 Correctness of BUILD-MAX-HEAP Loop invariant: –At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap Initialization: –i =  n/2  : Nodes  n/2  +1,  n/2  +2, …, n are leaves  they are the root of trivial max-heaps

CS 477/677 - Lecture 9 Correctness of BUILD-MAX-HEAP Maintenance: –MAX-HEAPIFY makes node i a max- heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps –Decrementing i in the for loop reestablishes the loop invariant Termination: –i = 0  each node 1, 2, …, n is the root of a max-heap (by the loop invariant)

CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP  It would seem that running time is O(nlgn) This is not an asymptotically tight upper bound Alg: BUILD-MAX-HEAP (A) 1.n = length[A] 2. for i ←  n/2  downto 1 3. do MAX-HEAPIFY (A, i, n) O(lgn) O(n)

CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP HEAPIFY takes O(h)  the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree HeightLevel h 0 = 3 (  lgn  ) h 1 = 2 h 2 = 1 h 3 = 0 i = 0 i = 1 i = 2 i = 3 (  lgn  ) No. of nodes h i = h – i height of the heap rooted at level i n i = 2 i number of nodes at level i

CS 477/677 - Lecture 9 Running Time of BUILD MAX HEAP Cost of HEAPIFY at level i  number of nodes at that level Replace the values of n i and h i computed before Multiply by 2 h both at the nominator and denominator and write 2 i as Change variables: k = h - i The sum above is smaller than the sum of all elements to  and h = lgn The sum above is smaller than 2 Running time of BUILD-MAX-HEAP: T(n) = O(n)

CS 477/677 - Lecture 9 Binary Search Trees Tree representation: –A linked data structure in which each node is an object Node representation: –Key field –Satellite data –Left: pointer to left child –Right: pointer to right child –p: pointer to parent ( p [root [T]] = NIL ) Satisfies the binary search tree property Left child Right child LR parent keydata

CS 477/677 - Lecture 9 Binary Search Tree Example Binary search tree property: –If y is in left subtree of x, then key [y] ≤ key [x] –If y is in right subtree of x, then key [y] ≥ key [x]

CS 477/677 - Lecture 9 Binary Search Trees Support many dynamic set operations –SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT, DELETE Running time of basic operations on binary search trees –On average:  (lgn) The expected height of the tree is lgn –In the worst case:  (n) The tree is a linear chain of n nodes

CS 477/677 - Lecture 9 Red-Black Trees “Balanced” binary trees guarantee an O(lgn) running time on the basic dynamic-set operations Red-black tree –Binary tree with an additional attribute for its nodes: color which can be red or black –Constrains the way nodes can be colored on any path from the root to a leaf Ensures that no path is more than twice as long as another  the tree is balanced –The nodes inherit all the other attributes from the binary-search trees: key, left, right, p

CS 477/677 - Lecture 9 Red-Black Trees Properties 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black No two red nodes in a row on a simple path from the root to a leaf 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes

CS 477/677 - Lecture 9 Example: RED-BLACK TREE For convenience we use a sentinel NIL[T] to represent all the NIL nodes at the leafs –NIL[T] has the same fields as an ordinary node –Color[NIL[T]] = BLACK –The other fields may be set to arbitrary values NIL

CS 477/677 - Lecture 9 Black-Height of a Node Height of a node: the number of edges in a longest path to a leaf Black-height of a node x: bh(x) is the number of black nodes (including NIL) on a path from x to leaf, not counting x NIL h = 4 bh = 2 h = 3 bh = 2 h = 2 bh = 1 h = 1 bh = 1 h = 1 bh = 1 h = 2 bh = 1 h = 1 bh = 1

CS 477/677 - Lecture 9 Properties of Red-Black Trees Claim –Any node with height h has black-height ≥ h/2 Proof –By property 4, there are at most h/2 red nodes on the path from the node to a leaf –Hence at least h/2 are black Property 4: if a node is red then both its children are black

CS 477/677 - Lecture 9 Properties of Red-Black Trees Claim The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Proof: By induction on height of x Basis: height[x] = 0  x is a leaf ( NIL[T] )  bh(x) = 0  # of internal nodes: = 0 NIL x

CS 477/677 - Lecture 9 Properties of Red-Black Trees Inductive step: Let height ( x) = h and bh(x) = b Any child y of x has: –bh (y) = b (if the child is red), or b - 1 (if the child is black)

CS 477/677 - Lecture 9 Properties of Red-Black Trees Want to prove: –The subtree rooted at any node x contains at least 2 bh(x) - 1 internal nodes Assume true for children of x : –Their subtrees contain at least 2 bh(x) - 1 – 1 internal nodes The subtree rooted at x contains at least: (2 bh(x) - 1 – 1) + (2 bh(x) - 1 – 1) + 1 = 2 · (2 bh(x) ) + 1 = 2 bh(x) - 1 internal nodes x l r

CS 477/677 - Lecture 9 Properties of Red-Black Trees Lemma: A red-black tree with n internal nodes has height at most 2lg(n + 1). Proof: n Add 1 to all sides and then take logs: n + 1 ≥ 2 b ≥ 2 h/2 lg(n + 1) ≥ h/2  h ≤ 2 lg(n + 1) root l r height(root) = h bh(root) = b number n of internal nodes ≥ 2 b - 1 ≥ 2 h/2 - 1 since b  h/2

CS 477/677 - Lecture 9 Operations on Red-Black Trees The non-modifying binary-search tree operations MINIMUM, MAXIMUM, SUCCESSOR, PREDECESSOR, and SEARCH run in O(h) time –They take O(lgn) time on red-black trees What about TREE-INSERT and TREE-DELETE? –They will still run in O(lgn) –We have to guarantee that the modified tree will still be a red-black tree

INSERT INSERT: what color to make the new node? Red? Let’s insert 35! –Property 4: if a node is red, then both its children are black Black? Let’s insert 14! –Property 5: all paths from a node to its leaves contain the same number of black nodes CS 477/677 - Lecture 9

DELETE DELETE: what color was the node that was removed? Red? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! OK! Does not create two red nodes in a row OK! Does not change any black heights CS 477/677 - Lecture 9

DELETE DELETE: what color was the node that was removed? Black? 1.Every node is either red or black 2.The root is black 3.Every leaf ( NIL ) is black 4.If a node is red, then both its children are black 5.For each node, all paths from the node to descendant leaves contain the same number of black nodes OK! Not OK! Could create two red nodes in a row Not OK! Could change the black heights of some nodes Not OK! If removing the root and the child that replaces it is red CS 477/677 - Lecture 9

Rotations Operations for restructuring the tree after insert and delete operations on red-black trees Rotations take a red-black tree and a node within the tree and: –Together with some node re-coloring they help restore the red-black tree property –Change some of the pointer structure –Do not change the binary-search tree property Two types of rotations: –Left & right rotations CS 477/677 - Lecture 9

Left Rotations Assumption for a left rotation on a node x : –The right child of x (y) is not NIL Idea: –Pivots around the link from x to y –Makes y the new root of the subtree –x becomes y ’s left child –y ’s left child becomes x ’s right child CS 477/677 - Lecture 9

Readings Chapter 8, Chapter 6