Design and Analysis of Algorithms – Chapter 41 Decrease and Conquer II Dr. Ying Lu RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms – Chapter 42 b Giving credit where credit is due: Most of the lecture notes are based on the slides from the Textbook’s companion websiteMost of the lecture notes are based on the slides from the Textbook’s companion website – Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope CollegeSome examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College I have modified many of their slides and added new slides.I have modified many of their slides and added new slides. RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms – Chapter 43 Examples of Decrease and Conquer b Decrease by one: Insertion sortInsertion sort Graph search algorithms:Graph search algorithms: –DFS –BFS –Topological sorting Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets b Decrease by a constant factor Binary searchBinary search Fake-coin problemsFake-coin problems multiplication à la russemultiplication à la russe Josephus problemJosephus problem b Variable-size decrease Euclid’s algorithmEuclid’s algorithm Selection by partitionSelection by partition BSTBST

Design and Analysis of Algorithms – Chapter 44 Fake coin problem b You have 8 coins and a two-pan balance. All the coins weigh the same, except for one, which is lighter than all the others. The coins are otherwise indistinguishable. You may make no assumption about how much lighter the fake coin is. What is the minimum number of weighings needed to be certain of identifying the fake coin? b How about identifying one light fake coin out of n coins?

Design and Analysis of Algorithms – Chapter 45 Examples of Decrease and Conquer b Decrease by one: Insertion sortInsertion sort Graph search algorithms:Graph search algorithms: –DFS –BFS –Topological sorting Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets b Decrease by a constant factor Binary searchBinary search Fake-coin problemsFake-coin problems multiplication à la russe (read textbook yourself)multiplication à la russe (read textbook yourself) Josephus problem (read textbook yourself)Josephus problem (read textbook yourself) b Variable-size decrease Euclid’s algorithmEuclid’s algorithm Selection by partitionSelection by partition BST (slides provided for reviewing yourself)BST (slides provided for reviewing yourself)

Design and Analysis of Algorithms – Chapter 46 The selection problem b Input: A set S of n elements b Output: The k th smallest element of S

Design and Analysis of Algorithms – Chapter 47 The selection problem b Input: A set S of n elements b Output: The k th smallest element of S b To find the median (the middle value) b To find the smallest element b To find the largest element K = K = 1 K = n

Design and Analysis of Algorithms – Chapter 48 The selection problem b Input: A set S of n elements b Output: The k th smallest element of S b The straightforward algorithm: step 1: Sort the n elementsstep 1: Sort the n elements step 2: Locate the k th element in the sorted list.step 2: Locate the k th element in the sorted list. Time complexity: O(nlogn)Time complexity: O(nlogn) b This algorithm is an overkill!

Design and Analysis of Algorithms – Chapter 49 Partition problem b Select a pivot (partitioning element), say the left most element b Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot p A[i]≤p A[i]  p

Design and Analysis of Algorithms – Chapter 410 Partition problem b Select a pivot (partitioning element), say the left most element b Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot b Design a linear O(n) partition algorithm b Work on problem instance: 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0 to help form the idea p A[i]≤p A[i]  p

Partition Example 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0

Partition algorithm (I) Design and Analysis of Algorithms – Chapter 412

Design and Analysis of Algorithms – Chapter 413 Partition algorithm (II) 

Design and Analysis of Algorithms – Chapter 414 Partition algorithm (II)  A “sentinel” at A[n] to prevent i advances beyond position n.

Design and Analysis of Algorithms – Chapter 415 Partition algorithm b Select a pivot (partitioning element) b Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot b How to solve the selection problem based on the partition algorithm? p A[i]≤p A[i]  p

Design and Analysis of Algorithms – Chapter 416 Selection by partition b Select a pivot (partitioning element) b Partition the list using the pivot b If pivot is in the k th position, the pivot is the element, done! b If pivot’s position is > k, repeat the above process with the first sublist b If pivot’s position is < k, repeat the above process with the second sublist p A[i]≤p A[i]>p

Example: b Find the median of the following list of nine elements: 4, 1, 10, 9, 7, 12, 8, 2, 15. Design and Analysis of Algorithms – Chapter 417

Design and Analysis of Algorithms – Chapter 418 Selection by partition b Select a pivot (partitioning element) b Partition the list using the pivot b If pivot is in the k th position, the pivot is the element, done! b If pivot’s position is > k, repeat the above process with the first sublist b If pivot’s position is < k, repeat the above process with the second sublist b Time efficiency p A[i]≤p A[i]>p

Design and Analysis of Algorithms – Chapter 419 Binary Search Trees (BSTs)

Design and Analysis of Algorithms – Chapter 420 Binary Search Trees (BSTs) b Binary Search Tree property A binary tree in which the key of an internal node is greater than the keys in its left subtree and less than or equal to the keys in its right subtree.A binary tree in which the key of an internal node is greater than the keys in its left subtree and less than or equal to the keys in its right subtree. b An inorder traversal of a binary search tree produces a sorted list of keys.

Design and Analysis of Algorithms – Chapter 421 Variable-size-decrease: Binary search trees b Keys are arranged in a binary tree with the binary search tree property: k <k kk

Design and Analysis of Algorithms – Chapter 422 BST Examples b Binary Search trees with different degrees of balance b Black dots denote empty trees b Size of a search tree

Design and Analysis of Algorithms – Chapter 423 Variable-size-decrease: Binary search trees b Arrange keys in a binary tree with the binary search tree property: k <k  k k Example 1: Example 1: 5, 10, 3, 1, 7, 12, 9 Example 2: Example 2: 4, 5, 7, 2, 1, 3, 6

Design and Analysis of Algorithms – Chapter 424 BST Operations b Find the min/max element (  ) b Search for an element b Find the successor/predecessor of an element b Delete an element

Design and Analysis of Algorithms – Chapter 425 BST: Min/Max b The minimum element is the left-most node x is a non-empty BST b The maximum element is the right-most node

Design and Analysis of Algorithms – Chapter 426 BST: Min/Max b The minimum element is the left-most node x is a non-empty BST b The maximum element is the right-most node (Time efficiency )

Design and Analysis of Algorithms – Chapter 427 BST Operations b Find the min/max element b Search for an element (  ) b Find the successor/predecessor of an element b Delete an element

Design and Analysis of Algorithms – Chapter 428 BST Search (Retrieval) Element bstSearch(BinTree bst, Key K) 1.Element found 2.if (bst == nil) 3. found = null; 4.else 5. Element root = root(bst); 6. if (K == root.key) 7. found = root; 8. else if (K < root.key) 9. found = bstSearch (leftSubtree(bst), K); 10. else 11. found = bstSearch(rightSubtree(bst), K); 12.return found; (Time efficiency )

Design and Analysis of Algorithms – Chapter 429 BST Operations b Find the min/max element b Search for an element b Find the successor/predecessor of an element (  ) b Delete an element

Design and Analysis of Algorithms – Chapter 430 BST: Successor/Predecessor b Finding the successor of a node x (if it exists): If x has a nonempty right subtree, then successor(x) is the smallest element in the tree root at rightSubtree(x)If x has a nonempty right subtree, then successor(x) is the smallest element in the tree root at rightSubtree(x)

Design and Analysis of Algorithms – Chapter 431 BST: Successor/Predecessor b Finding the successor of a node x (if it exists): If rightSubtree(x) is empty, then successor(x) is the lowest ancestor of x whose left child is also an ancestor of x.If rightSubtree(x) is empty, then successor(x) is the lowest ancestor of x whose left child is also an ancestor of x.

Design and Analysis of Algorithms – Chapter 432 BST: Successor/Predecessor b The predecessor operation is symmetric to successor.

Design and Analysis of Algorithms – Chapter 433 Why binary search tree? Array: BST:

Design and Analysis of Algorithms – Chapter 434 BST: advantage b The advantage to the binary search tree approach is that it combines the advantage of an array--the ability to do a binary search with the advantage of a linked list--its dynamic size.

Design and Analysis of Algorithms – Chapter 435 BST Operations b Find the min/max element b Search for an element b Find the successor/predecessor of an element b Delete an element (  )

Design and Analysis of Algorithms – Chapter 436 BST: Delete b Deleting a node z is by far the most difficult BST operation. b There are three cases to consider If z has no children, just delete it.If z has no children, just delete it. If z has one child, splice out z, That is, link z’s parent and childIf z has one child, splice out z, That is, link z’s parent and child If z has two children, splice out z’s successor y, and replace the contents of z with the contents of yIf z has two children, splice out z’s successor y, and replace the contents of z with the contents of y b The last case works because if z has two children, then its successor has no left child

Design and Analysis of Algorithms – Chapter 437 BST: splice out examples

Design and Analysis of Algorithms – Chapter 438 BST: splice out algorithm Only works when a node has at most one child

Design and Analysis of Algorithms – Chapter 439 BST: Deletion Algorithm b Delete is now simple!

Design and Analysis of Algorithms – Chapter 440 BST: one more delete example

Design and Analysis of Algorithms – Chapter 441 Analysis of BST Operations b All of the BST operations have time complexity  (h), where h is the height of the tree b However, in the worst-case, the height may be  (n) where n is the number of internal nodes For example, a long chain tree structureFor example, a long chain tree structure b For a tree as balanced as possible,  (log 2 n) b The objective is to make the tree as balanced as possible Technique: Binary Tree RotationsTechnique: Binary Tree Rotations b Balanced search tree: AVL tree and 2-3 tree