1 CSC 427: Data Structures and Algorithm Analysis Fall 2010 Divide & conquer  divide-and-conquer approach  familiar examples: binary search, merge sort,

Slides:



Advertisements
Similar presentations
CSC 421: Algorithm Design & Analysis
Advertisements

Comp 122, Spring 2004 Binary Search Trees. btrees - 2 Comp 122, Spring 2004 Binary Trees  Recursive definition 1.An empty tree is a binary tree 2.A node.
Binary Trees, Binary Search Trees CMPS 2133 Spring 2008.
Binary Trees, Binary Search Trees COMP171 Fall 2006.
Trees, Binary Trees, and Binary Search Trees COMP171.
4a-Searching-More1 More Searching Dan Barrish-Flood.
Lec 15 April 9 Topics: l binary Trees l expression trees Binary Search Trees (Chapter 5 of text)
Chapter 12 Trees. Copyright © 2005 Pearson Addison-Wesley. All rights reserved Chapter Objectives Define trees as data structures Define the terms.
Module #1 - Logic 1 Based on Rosen, Discrete Mathematics & Its Applications. Prepared by (c) , Michael P. Frank and Modified By Mingwu Chen Trees.
Marc Smith and Jim Ten Eyck
1 Joe Meehean.  Important and common problem  Given a collection, determine whether value v is a member  Common variation given a collection of unique.
1 CSC 427: Data Structures and Algorithm Analysis Fall 2010 transform & conquer  transform-and-conquer approach  balanced search trees o AVL, 2-3 trees,
Version TCSS 342, Winter 2006 Lecture Notes Trees Binary Trees Binary Search Trees.
1 CSC 321: Data Structures Fall 2012 Proofs & trees  proof techniques –direct proof, proof by contradiction, proof by induction  trees  tree recursion.
Recursion Bryce Boe 2013/11/18 CS24, Fall Outline Wednesday Recap Lab 7 Iterative Solution Recursion Binary Tree Traversals Lab 7 Recursive Solution.
1 CSC 321: Data Structures Fall 2012 Binary Search Trees  BST property  override binary tree methods: add, contains  search efficiency  balanced trees:
24/3/00SEM107 - © Kamin & ReddyClass 16 - Searching - 1 Class 16 - Searching r Linear search r Binary search r Binary search trees.
Lecture 10 Trees –Definiton of trees –Uses of trees –Operations on a tree.
S EARCHING AND T REES COMP1927 Computing 15s1 Sedgewick Chapters 5, 12.
1 Trees A tree is a data structure used to represent different kinds of data and help solve a number of algorithmic problems Game trees (i.e., chess ),
CMSC 341 Introduction to Trees. 8/3/2007 UMBC CMSC 341 TreeIntro 2 Tree ADT Tree definition  A tree is a set of nodes which may be empty  If not empty,
Binary Trees, Binary Search Trees RIZWAN REHMAN CENTRE FOR COMPUTER STUDIES DIBRUGARH UNIVERSITY.
CSC 211 Data Structures Lecture 13
Trees, Binary Trees, and Binary Search Trees COMP171.
Topic 15 The Binary Search Tree ADT Binary Search Tree A binary search tree (BST) is a binary tree with an ordering property of its elements, such.
Trees  Linear access time of linked lists is prohibitive Does there exist any simple data structure for which the running time of most operations (search,
Outline Binary Trees Binary Search Tree Treaps. Binary Trees The empty set (null) is a binary tree A single node is a binary tree A node has a left child.
1 CSC 421: Algorithm Design & Analysis Spring 2013 Divide & conquer  divide-and-conquer approach  familiar examples: merge sort, quick sort  other examples:
Lec 15 Oct 18 Binary Search Trees (Chapter 5 of text)
1 CSC 427: Data Structures and Algorithm Analysis Fall 2011 Divide & conquer (part 1)  divide-and-conquer approach  familiar examples: binary search,
+ David Kauchak cs312 Review. + Midterm Will be posted online this afternoon You will have 2 hours to take it watch your time! if you get stuck on a problem,
1 CSC 427: Data Structures and Algorithm Analysis Fall 2008 TreeSets and TreeMaps  tree structure, root, leaves  recursive tree algorithms: counting,
CMSC 341 Introduction to Trees. 2/21/20062 Tree ADT Tree definition –A tree is a set of nodes which may be empty –If not empty, then there is a distinguished.
Binary Search Trees (BST)
Tree Data Structures. Heaps for searching Search in a heap? Search in a heap? Would have to look at root Would have to look at root If search item smaller.
1 CSC 427: Data Structures and Algorithm Analysis Fall 2004 Trees and nonlinear structures  tree structure, root, leaves  recursive tree algorithms:
1 Joe Meehean. A A B B D D I I C C E E X X A A B B D D I I C C E E X X  Terminology each circle is a node pointers are edges topmost node is the root.
BINARY TREES Objectives Define trees as data structures Define the terms associated with trees Discuss tree traversal algorithms Discuss a binary.
Trees CSIT 402 Data Structures II 1. 2 Why Do We Need Trees? Lists, Stacks, and Queues are linear relationships Information often contains hierarchical.
(c) University of Washington20c-1 CSC 143 Binary Search Trees.
1 CMSC 341 Introduction to Trees Textbook sections:
CSC 321: Data Structures Fall 2016 Trees & recursion
Trees Chapter 11 (continued)
CSC 427: Data Structures and Algorithm Analysis
CSC 427: Data Structures and Algorithm Analysis
CSC 421: Algorithm Design & Analysis
CSC 421: Algorithm Design & Analysis
Trees Chapter 11 (continued)
Binary Search Trees A binary search tree is a binary tree
CSC 421: Algorithm Design & Analysis
CSC 421: Algorithm Design & Analysis
CSC 421: Algorithm Design & Analysis
CSC 421: Algorithm Design & Analysis
CSC 321: Data Structures Fall 2017 Trees & recursion
CMSC 341 Introduction to Trees.
Trees.
i206: Lecture 13: Recursion, continued Trees
Binary Trees, Binary Search Trees
Binary Tree and General Tree
Map interface Empty() - return true if the map is empty; else return false Size() - return the number of elements in the map Find(key) - if there is an.
Lec 12 March 9, 11 Mid-term # 1 (March 21?)
Binary Trees, Binary Search Trees
Trees.
CSC 321: Data Structures Fall 2018 Trees & recursion
CSC 421: Algorithm Design & Analysis
CSC 143 Java Trees.
CSC 143 Binary Search Trees.
Trees.
CSC 421: Algorithm Design & Analysis
Binary Trees, Binary Search Trees
Presentation transcript:

1 CSC 427: Data Structures and Algorithm Analysis Fall 2010 Divide & conquer  divide-and-conquer approach  familiar examples: binary search, merge sort, quick sort  graphics example: closest points  binary tree algorithms  binary search trees

2 X & Conquer algorithms many algorithms solve a problem by breaking it into subproblems, solving each subproblem (often recursively), then combining the results  if the subproblem is proportional to the list size (e.g., half), we call it divide-and-conquer  if the subproblem is reduced in size by a constant amount, we call it decrease-and-conquer  if the subproblem is a transformation into a simpler/related problem, we call it transform-and-conquer divide-and-conquer is probably the best known algorithmic approach e.g., binary search as divide-and-conquer 1.if list size = 0, then FAILURE 2.otherwise, check midpoint of list a)if middle element is the item being searched for, then SUCCESS b)if middle element is > item being searched for, then binary search the left half c)if middle element is < item being searched for, then binary search the right half

3 Iteration vs. divide & conquer many iterative algorithms can be characterized as divide-and-conquer  sum of list[0..N-1] = 0 if N == 0 list[N/2] + sum of list [0..N/2-1] + otherwise sum of list[N/2+1..N-1]  number of occurrences of X in list[0..N-1] = 0if N == num of occurrences of X in list[0..N/2-1] +if X == list[N/2] + num of occurrences of X in list[N/2+1..N-1] 0 + num of occurrences of X in list[0..N/2-1] +if X != list[N/2] + num of occurrences of X in list[N/2+1..N-1] recall: Cost(N) = 2Cost(N/2) + C  O(N), so no real advantage in these examples over brute force iteration

Merge sort we have seen divide-and-conquer algorithms that are more efficient than brute force e.g., merge sort list[0..N-1] 1.if list N <= 1, then DONE 2.otherwise, a)merge sort list[0..N/2] b)merge sort list[N/2+1..N-1] c)merge the two sorted halves recall: Cost(N) = 2Cost(N/2) +CN  O(N log N)  merging is O(N), but requires O(N) additional storage and copying 4

Quick sort Collections.sort implements quick sort, another O(N log ) sort which is faster in practice e.g., quick sort list[0..N-1] 1.if list N <= 1, then DONE 2.otherwise, a)select a pivot element (e.g., list[0], list[N/2], list[random], …) b)partition list into [items pivot] c)quick sort the partitions best case: pivot is median Cost(N) = 2Cost(N/2) +CN  O(N log N) worst case: pivot is smallest or largest value Cost(N) = Cost(N-1) +CN  O(N 2 ) 5

Quick sort (cont.) average case: O(N log N) there are variations that make the worst case even more unlikely  switch to selection sort when small (as in HW3)  median-of-three partitioning instead of just taking the first item (or a random item) as pivot, take the median of the first, middle, and last items in the list if the list is partially sorted, the middle element will be close to the overall median if the list is random, then the odds of selecting an item near the median is improved refinements like these can improve runtime by 20-25% however, O(N 2 ) degradation is possible 6

7 Closest pair given a set of N points, find the pair with minimum distance  brute force approach: consider every pair of points, compare distances & take minimum big Oh?  there exists an O(N log N) divide-and-conquer solution 1.sort the points by x-coordinate 2.partition the points into equal parts using a vertical line in the plane 3.recursively determine the closest pair on left side (Ldist) and the closest pair on the right side (Rdist) 4.find closest pair that straddles the line, each within min(Ldist,Rdist) of the line (can be done in O(N)) 5.answer = min(Ldist, Rdist, Cdist) O(N 2 )

8 Trees & divide-and-conquer a tree is a nonlinear data structure consisting of nodes (structures containing data) and edges (connections between nodes), such that:  one node, the root, has no parent (node connected from above)  every other node has exactly one parent node  there is a unique path from the root to each node (i.e., the tree is connected and there are no cycles) nodes that have no children (nodes connected below them) are known as leaves

9 Recursive definition of a tree trees are naturally recursive data structures:  the empty tree (with no nodes) is a tree  a node with subtrees connected below is a tree tree with 7 nodes a tree where each node has at most 2 subtrees (children) is a binary tree empty tree tree with 1 node (empty subtrees)

10 Trees in CS trees are fundamental data structures in computer science example: file structure  an OS will maintain a directory/file hierarchy as a tree structure  files are stored as leaves; directories are stored as internal (non-leaf) nodes descending down the hierarchy to a subdirectory  traversing an edge down to a child node DISCLAIMER: directories contain links back to their parent directories, so not strictly a tree

11 Recursively listing files to traverse an arbitrary directory structure, need recursion to list a file system object (either a directory or file): 1.print the name of the current object 2.if the object is a directory, then ─recursively list each file system object in the directory in pseudocode: public static void ListAll(FileSystemObject current) { System.out.println(current.getName()); if (current.isDirectory()) { for (FileSystemObject obj : current.getContents()) { ListAll(obj); }

12 Recursively listing files public static void ListAll(FileSystemObject current) { System.out.println(current.getName()); if (current.isDirectory()) { for (FileSystemObject obj : current.getContents()) { ListAll(obj); } this method performs a pre-order traversal : prints the root first, then the subtrees

13 UNIX du command in UNIX, the du command list the size of all files and directories from the ~davereed directory: unix> du –a 2./public_html/index.html 3./public_html/Images/reed.jpg 3./public_html/Images/logo.gif 7./public_html/Images 10./public_html 1./mail/dead.letter 2./mail 13. public static int du(FileSystemObject current) { int size = current.blockSize(); if (current.isDirectory()) { for (FileSystemObject obj : current.getContents()) { size += du(obj); } System.out.println(size + " " + current.getName()); return size; } this method performs a post-order traversal : prints the subtrees first, then the root

14 Dividing & conquering trees since trees are recursive structures, most tree traversal and manipulation operations can be classified as divide & conquer algorithms  can divide a tree into root + left subtree + right subtree  most tree operations handle the root as a special case, then recursively process the subtrees  e.g., to display all the values in a (nonempty) binary tree, divide into 1.displaying the root 2.(recursively) displaying all the values in the left subtree 3.(recursively) displaying all the values in the right subtree  e.g., to count number of nodes in a (nonempty) binary tree, divide into 1.(recursively) counting the nodes in the left subtree 2.(recursively) counting the nodes in the right subtree 3.adding the two counts + 1 for the root

BinaryTree class public class BinaryTree { protected TreeNode root; public BinaryTree() { this.root = null; } public void add(E value) { … } public boolean remove(E value) { … } public boolean contains(E value) { … } public int size() { … } public String toString() { … } } 15 to implement a binary tree, need to store the root node  the empty tree has a null root  then, must implement methods for basic operations on the collection

16 size method public int size() { return this.size(this.root); } private int size(TreeNode current) { if (current == null) { return 0; } else { return this.size(current.getLeft()) + this.size(current.getRight()) + 1; } } divide-and-conquer approach: BASE CASE: if the tree is empty, number of nodes is 0 RECURSIVE: otherwise, number of nodes is (# nodes in left subtree) + (# nodes in right subtree) + 1 for the root note: a recursive implementation requires passing the root as parameter  will have a public "front" method, which calls the recursive "worker" method

17 contains method public boolean contains(E value) { return this.contains(this.root, value); } private boolean contains(TreeNode current, E value) { if (current == null) { return false; } else { return value.equals(current.getData()) || this.contains(current.getLeft(), value) || this.contains(current.getRight(), value); } } divide-and-conquer approach: BASE CASE: if the tree is empty, the item is not found BASE CASE: otherwise, if the item is at the root, then found RECURSIVE: otherwise, search the left and then right subtrees

18 toString method public String toString() { if (this.root == null) { return "[]"; } String recStr = this.toString(this.root); return "[" + recStr.substring(0,recStr.length()-1) + "]"; } private String toString(TreeNode current) { if (current == null) { return ""; } return this.toString(current.getLeft()) + current.getData().toString() + "," + this.toString(current.getRight()); } must traverse the entire tree and build a string of the items  there are numerous patterns that can be used, e.g., in-order traversal BASE CASE: if the tree is empty, then nothing to traverse RECURSIVE: recursively traverse the left subtree, then access the root, then recursively traverse the right subtree

19 Alternative traversal algorithms private String toString(TreeNode current) { if (current == null) { return ""; } return current.getData().toString() + "," + this.toString(current.getLeft()) + this.toString(current.getRight()); } private String toString(TreeNode current) { if (current == null) { return ""; } return this.toString(current.getLeft()) + this.toString(current.getRight()) + current.getData().toString() + ","; } pre-order traversal: BASE CASE: if the tree is empty, then nothing to traverse RECURSIVE: access root, recursively traverse left subtree, then right subtree post-order traversal: BASE CASE: if the tree is empty, then nothing to traverse RECURSIVE: recursively traverse left subtree, then right subtree, then root

20 Exercises the number of times value occurs in the tree with specified root */ public int numOccur(TreeNode root, E value) { } the sum of all the values stored in the tree with specified root */ public int sum(TreeNode root) { } the # of nodes in the longest path from root to leaf in the tree */ public int height(TreeNode root) { }

21 add method public void add(E value) { this.root = this.add(this.root, value); } private TreeNode add(TreeNode current, E value) { if (current == null) { current = new TreeNode (value, null, null); } else if (current.getLeft() == null) { current.setLeft(new TreeNode (value, null, null)); } else if (current.getRight() == null) { current.setRight(new TreeNode (value, null, null)); } else if (Math.random() < 0.5) { current.setLeft(this.add(current.getLeft(), value)); } else { current.setRight(this.add(current.getRight(), value)); } return current; } how do you add to a binary tree?  ideally, would try to keep the tree balanced, but naïve approach is O(N 2 )  we will more consider efficient approaches for maintaining balance later  for now, we will add a new value at a random leaf location big-Oh?

22 remove method how do you remove from a binary tree?  tricky, since removing an internal node means rerouting pointers  must maintain binary tree structure simpler solution 1.find node (as in search) 2.if a leaf, simply remove it 3.if no left subtree, reroute parent pointer to right subtree 4.otherwise, replace current value with a leaf value from the left subtree (and remove the leaf node) you can see the implementation in BinaryTree.java

23 Searching linked lists recall: a (linear) linked list only provides sequential access  O(N) searches it is possible to obtain O(log N) searches using a tree structure in order to perform binary search efficiently, must be able to  access the middle element of the list in O(1)  divide the list into halves in O(1) and recurse HOW CAN WE GET THIS FUNCTIONALITY FROM A TREE?

24 Binary search trees a binary search tree is a binary tree in which, for every node:  the item stored at the node is ≥ all items stored in its left subtree  the item stored at the node is < all items stored in its right subtree in a (balanced) binary search tree: middle element = root 1 st half of list = left subtree 2 nd half of list = right subtree furthermore, these properties hold for each subtree

BinarySearchTree class public class BinarySearchTree > extends BinaryTree { public BinarySearchTree() { super(); } public void add(E value) { // OVERRIDE TO MAINTAIN BINARY SEARCH TREE PROPERTY } public void CONTAINS(E value) { // OVERRIDE TO TAKE ADVANTAGE OF BINARY SEARCH TREE PROPERTY } public void remove(E value) { // DOES THIS NEED TO BE OVERRIDDEN? } 25 can use inheritance to derive BinarySearchTree from BinaryTree

26 Binary search in BSTs to search a binary search tree: 1.if the tree is empty, NOT FOUND 2.if desired item is at root, FOUND 3.if desired item < item at root, then recursively search the left subtree 4.if desired item > item at root, then recursively search the right subtree public boolean contains(E value) { return this.contains(this.root, value); } private boolean contains(TreeNode current, E value) { if (current == null) { return false; } else if (value.equals(current.getData())) { return true; } else if (value.compareTo(current.getData()) < 0) { return this.contains(current.getLeft(), value); } else { return this.contains(current.getRight(), value); }

27 Search efficiency how efficient is search on a BST?  in the best case? O(1)if desired item is at the root  in the worst case? O(height of the tree) if item is leaf on the longest path from the root in order to optimize worst-case behavior, want a (relatively) balanced tree  otherwise, don't get binary reduction  e.g., consider two trees, each with 7 nodes

28 How deep is a balanced tree? Proof (by induction): BASE CASES: when H = 0, = 0 nodes when H = 1, = 1 node HYPOTHESIS: assume a tree with height H-1 can store up to 2 H-1 -1 nodes INDUCTIVE STEP: a tree with height H has a root and subtrees with height up to H-1 by our hypothesis, T1 and T2 can each store 2 H-1 -1 nodes, so tree with height H can store up to 1 + (2 H-1 -1) + (2 H-1 -1) = 2 H H-1 -1 = 2 H -1 nodes equivalently: N nodes can be stored in a binary tree of height  log 2 (N+1)  THEOREM: A binary tree with height H can store up to 2 H -1 nodes.

29 Search efficiency (cont.) so, in a balanced binary search tree, searching is O(log N) N nodes  height of  log2(N+1)   in worst case, have to traverse  log2(N+1)  nodes what about the average-case efficiency of searching a binary search tree?  assume that a search for each item in the tree is equally likely  take the cost of searching for each item and average those costs costs of search  17/7  2.42 define the weight of a tree to be the sum of all node depths (root = 1, …) average cost of searching a tree = weight of tree / number of nodes in tree

30 Search efficiency (cont.) costs of search  17/7  2.42 ~log N costs of search  28/7  4.00 ~N/2

31 Inserting an item inserting into a BST 1.traverse edges as in a search 2.when you reach a leaf, add the new node below it public void add(E value) { this.root = this.add(this.root, value); } private TreeNode add(TreeNode current, E value) { if (current == null) { return new TreeNode (value, null, null); } if (value.compareTo(current.getData()) <= 0) { current.setLeft(this.add(current.getLeft(), value)); } else { current.setRight(this.add(current.getRight(), value)); } return current; }

32 Removing an item recall BinaryTree remove 1.find node (as in search) 2.if a leaf, simply remove it 3.if no left subtree, reroute parent pointer to right subtree 4.otherwise, replace current value with a leaf value from the left subtree (and remove the leaf node) CLAIM: as long as you select the rightmost (i.e., maximum) value in the left subtree, this remove algorithm maintains the BST property WHY? so, no need to override remove

33 Maintaining balance PROBLEM: random insertions (and removals) do not guarantee balance  e.g., suppose you started with an empty tree & added words in alphabetical order braves, cubs, giants, phillies, pirates, reds, rockies, … with repeated insertions/removals, can degenerate so that height is O(N)  specialized algorithms exist to maintain balance & ensure O(log N) height  or take your chances braves cubs giants phillies

34 Balancing trees on average, N random insertions into a BST yields O(log N) height  however, degenerative cases exist (e.g., if data is close to ordered) we can ensure logarithmic depth by maintaining balance maintaining full balance can be costly  however, full balance is not needed to ensure O(log N) operations (LATER)