Chapter 5 Divide & conquer Master method Mergesort and Quicksort Binary tree traversals Closest pair and Convex hull revisited 7/2/2018 Analysis of Algorithms

Analysis of Algorithms 7/2/2018 Diagram a problem of size n subproblem 2 of size n/2 subproblem 1 of size n/2 a solution to subproblem 1 a solution to subproblem 2 Figure 5.1, p. 170 a solution to the original problem 7/2/2018 Analysis of Algorithms Chapter 5

Technique and Complexity Divide the problem into two or more pieces. Solve those pieces as sub-problems recursively. Combine the results to find a solution to the original problem. Time (example): T(n) = D(n) + 2T(n/2) + C(n) where: D(n) is the time it takes to divide the problem into two parts C(n) is the time it takes to combine the solutions from those parts 7/2/2018 Analysis of Algorithms

Analysis of Algorithms The Master Method T(n) = aT(n/b) + f(n) where f(n) ∈ Θ(nᵏ) [and of course T(1) is O(1)] a < bᵏ T(n) ∈ Θ(nᵏ) the terms are decreasing a = bᵏ T(n) ∈ Θ(nᵏ log n) the terms are all equal a > bᵏ T(n) ∈ Θ(nlogb a) the terms are increasing Examples: 9T(n/3) + 1 Θ(n²) T(2n/3) + 1 Θ(log n) T(n/2) + n Θ(n) 7/2/2018 Analysis of Algorithms

Proof of Master Method: T(n) = aT(n/b) + nᵏ Analysis of Algorithms 7/2/2018 Proof of Master Method: T(n) = aT(n/b) + nᵏ Top level: nᵏ [once] = nᵏ Next level: (n/b)ᵏ … [a times] = nᵏ(a/bᵏ) : ith level: (n/bⁱ)ᵏ … [aⁱ times] = nᵏ(a/bᵏ)ⁱ Last level: O(1) … [alogb n] = nlogb a [let a = bᶜ] The total is a geometric series whose highest term dominates the sum, or else a logarithmic number of equal terms. logb n levels 7/2/2018 Analysis of Algorithms Chapter 5

Mergesort algorithm (outline) Split the array A in two and copy the halves into the arrays B and C. Sort the arrays B and C recursively. Merge the sorted arrays B and C back into array A as follows: Repeat the following until no elements remain in B or C: Compare the first elements remaining in B and C. Move the smaller of the two into A (how do we break ties?) Once one of the arrays is exhausted, move the remaining elements from the other array into A. Not ‘in-place’ 7/2/2018 Analysis of Algorithms

Analysis of Algorithms 7/2/2018 Example 1 2 2 3 4 5 6 6 2 4 5 6 1 2 3 6 2 5 4 6 1 3 2 6 The diagram in the book is probably better. 5 2 4 6 1 3 2 6 7/2/2018 Analysis of Algorithms Chapter 5

Analysis of Algorithms Code Sort(A[0..n−1]) Sort an array of size n > 0 if n > 1 then there are two or more elements let q ← n/2 find the ‘middle’ of the array let B[0..⌊q⌋−1] ← A[0..⌊q⌋−1] Copy the first half of the array let C[0..⌈q⌉−1] ← A[⌊q⌋..n−1] Copy the second half of the array Sort(B[0..⌊q⌋−1]) recursively sort the first half Sort(C[0..⌈q⌉−1]) recursively sort the second half Merge(B, C, A) combine them back into A 7/2/2018 Analysis of Algorithms

Analysis of Algorithms 7/2/2018 Diagram Induction Basis A: · · · · · · · · · divide · · · · · sort · · · · · · conquer combine · · · · · · · · · · · 7/2/2018 Analysis of Algorithms Chapter 5

Analysis of Algorithms 7/2/2018 Correctness Show: Sort(A[0..n−1]) orders the elements in A[0..n−1] for n > 0. Basis: If n = 1, then A[0..0] has one element, and is hence sorted. Induction: If n > 1, then 0 < q = n/2 < n − 1. If n is even, then ⌊q⌋ = ⌈q⌉ = q and hence B and C are of equal size. Otherwise, if n is odd, then ⌊q⌋ + 1 = ⌈q⌉ and B is one element smaller than C in size. By induction hypothesis, lines 5 and 6 correctly order B[0..⌊q⌋−1] and C[0..⌈q⌉−1]. It remains to show that the merge process works correctly. This is true because the element that is moved back into A at each step is smaller than or equal to all the remaining elements that have not yet been moved. This uses the fact that B and C are already sorted. Verify that all the indices are correct. 7/2/2018 Analysis of Algorithms Chapter 5

Analysis of Algorithms 7/2/2018 Complexity Solve the recurrence relation: Is O(n log n) by recursion tree or master method log n levels O(n) O(1) for n = 1 T(n) = 2T(n/2) + O(n) otherwise O(n/2) O(n/2) · 7/2/2018 Analysis of Algorithms Chapter 5

Analysis of Algorithms 7/2/2018 Quicksort Given an array of keyed records, sort the records in order of their keys. keys → Outline Pick a pivot point x (try to choose a pivot near median) Split the array recursively around a pivot (an ‘in-place’ algorithm) Sort recursively around the pivot (combining is trivial) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - < x x ≥ x 7/2/2018 Analysis of Algorithms Chapter 5

Analysis of Algorithms Pivoting Select a pivot point (or use the linear-time median algorithm). Partition the list so that all the elements in the positions before the pivot are smaller than the pivot and those after the pivot are larger than or equal to the pivot (see the next slide). Exchange the pivot with the last element in the left (<) half. The pivot is now in its final position. Sort the positions below the pivot and above the pivot. p < p ≥ p 7/2/2018 Analysis of Algorithms

Analysis of Algorithms Partitioning Move all keys < pivot to the left of it; all keys ≥ pivot end up to the right of it. Note: this uses A[i] as the pivot key SPLIT(A, i, j) ► returns pivot point k within A[i..j] k ← i ► pivot point chosen initially left for index = i + 1 to j ► scan all remaining keys if A[index] < A[i] then ► A[index] belongs below the pivot k ← k + 1 ► move pivot point right A[k] ↔ A[index] ► swap with leftmost key ≥ x A[i] ↔ A[k] ► swap pivot with rightmost key < x 7/2/2018 Analysis of Algorithms

Analysis of Algorithms Diagrammatic proof Everything is trivial except for showing that SPLIT partitions correctly. Basis: i = j (just check that it returns i) Induction: Loop invariant right after line 3 (careful of empty regions) It starts with k = i and index = i + 1, and ends with index = j + 1. k pivot x < x ≥ x ? … i index j 7/2/2018 Analysis of Algorithms

Review of binary tree traversals Analysis of Algorithms 7/2/2018 Review of binary tree traversals Trick: trace around the outside of the tree, starting at the root CCW Preorder: when nodes are first encountered (going down the left side) Inorder: when going under a node Postorder: when nodes are last encountered (going up the right side) Computing height: h(T) = max{h(TL), h(TR)} + 1 if T ≠ ∅ and otherwise h(∅) = -1. Efficiency: Θ(n) Redraw example correctly so nodes are in correct horizontal order. Explain non-recursive algorithm for binary tree reversal. 7/2/2018 Analysis of Algorithms Chapter 5

Extremal elements in a binary search tree TREE-MINIMUM(x) while left[x] ≠ nil do x ← left[x] return x TREE-MAXIMUM(x) while right[x] ≠ nil do x ← right[x] return x 7/2/2018 Analysis of Algorithms

Successor (predecessor is just the reverse) TREE-SUCCESSOR(x) ► returns nil if x is maximal if right[x] ≠ nil then ► x has a right child return TREE-MINIMUM(right[x]) ► go to the right, otherwise … repeat ► go up until you go right y ← x ► save value of x x ← parent[x] ► go up until x = nil or left[x] = y ► reach root or successor return x ► the successor

In-order traversal of a binary search tree Start at the minimal element (in the left ‘corner’) Repeatedly take successor (until no longer possible) What is the result of this procedure in a binary search tree? The nodes come out in order from smallest to largest. Is there a more efficient procedure? 7/2/2018 Analysis of Algorithms

Analysis of Algorithms Threads Goal: Compute these without comparisons in O(h) time. Strategy: The left and right pointers are partial injective functions. Using threads, complete them into total injective functions l and r. Let l(x) = left[x] if it exists. Otherwise, repeatedly take its “right” parent. Let r(x) = right[x] if it exists. Otherwise, repeatedly take its “left” parent. Now, the successor of x is just r∙l⁻¹(x), and its predecessor is l∙r⁻¹(x). 7/2/2018 Analysis of Algorithms

Closest pair problem revisited Analysis of Algorithms 7/2/2018 Closest pair problem revisited Problem: Given a set of n points in the plane, find the closest pair. Sort the points by their horizontal coordinates. Divide them into two subsets Pl and Pr by a vertical line drawn about the median x = m so that half the points lie to the left or on the line and half the points lie to the right or on the line. What if they all lie on that median line? 7/2/2018 Analysis of Algorithms Chapter 5

Closest pair continued Find recursively the closest pairs for the left and right subsets. Set d = min{dl, dr}. We can limit our attention to the points in the symmetric vertical strip S of width 2d as possible closest pair. (The points are stored and processed in increasing order of their y coordinates.) Scan the points in the vertical strip S from the lowest up. For every point p(x, y) in the strip, inspect points in the strip that may be closer to p than d. There can be no more than 5 such points following p on the strip list (a sort of pigeon-hole principle)! Complexity: T(n) = 2T(n/2) + O(n) 7/2/2018 Analysis of Algorithms

Convex hull problem revisited Problem: Find the smallest convex set which includes a set of points. Sort points lexicographically by their (x, y) coordinate values. Identify (lower) leftmost point p1 and (upper) rightmost point pn. Find point pmax that is farthest away from the line p1pn. Show there are no points to the left of both lines p1pmax and pmaxpn. 7/2/2018 Analysis of Algorithms

Analysis of Algorithms 7/2/2018 Convex hull continued Concatenate the upper hull of the points to the left of the line p1pmax with the upper hull of the points to the left of the line pmaxpn. Compute the lower hull in a similar manner. Combine the upper and lower hulls to get the convex hull of the whole. Actually linear-time on average! 7/2/2018 Analysis of Algorithms Chapter 5