1. Theory of Algorithms: Transform and Conquer James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004

2. Objectives lTo introduce the transform-and-conquer mind set lTo show a variety of transform-and-conquer solutions:  Presorting  Horner’s Rule and Binary Exponentiation  Problem Reduction lTo discuss the strengths and weaknesses of a transform-and-conquer strategy l“The Secret of Life … to replace one worry with another” - Charles M. Schultz

3. Transform and Conquer 1.In a transformation stage, modify the problem to make it more amenable to solution 2.In a conquering stage, solve it lOften employed in mathematical problem solving and complexity theory Problem’s Instance Simpler Instance OR Another Representation OR Another Problem’s Instance Problem’s Solution

4. Flavours of Transform and Conquer 1.Instance Simplification = a more convenient instance of the same problem  Presorting  Gaussian elimination 2.Representation Change = a different representation of the same instance  Balanced search trees  Heaps and heapsort  Polynomial evaluation by Horner’s rule 3.Problem Reduction = a different problem altogether  Reductions to graph problems

5. Presorting: Instance Simplification lSolve instance of problem by preprocessing the problem to transform it into another simpler/easier instance of the same problem lMany problems involving lists are easier when list is sorted:  Searching  Computing the median (selection problem)  Computing the mode  Finding repeated elements  Convex Hull and Closest Pair lEfficiency:  Introduce the overhead of an  (n log n) preprocess  But the sorted problem often improves by at least one base efficiency class over the unsorted problem (e.g.,  (n 2 )   (n))

6. Example: Presorted Selection lFind the k th smallest element in A[1],…, A[n] lSpecial cases:  Minimum: k = 1  Maximum: k = n  Median: k =  n/2  lPresorting-based algorithm: Sort list Return A[k] lPartition-based algorithm (Variable Decrease & Conquer): Pivot/split at A[s] using Partitioning algorithm from Quicksort IF s=k RETURN A[s] ELSE IF s<k repeat with sublist A[s+1],…A[n] ELSE IF s>k repeat with sublist A[1],…A[s-1]

7. Notes on the Selection Problem lPresorting-based algorithm:  (n lgn) +  (1) =  (n lgn) lPartition-based algorithm (Variable decrease & conquer):  Worst case: T(n) =T(n-1) + (n+1)   (n 2 )  Best case:  (n)  Average case: T(n) =T(n/2) + (n+1)   (n)  Also identifies the k smallest elements (not just the k th ) lSimpler linear (brute force) algorithm is better in the case of max & min lConclusion: Presorting does not help in this case

8. Finding Repeated Elements lPresorting algorithm for finding duplicated elements in a list:  use mergesort:  (n lgn)  scan to find repeated adjacent elements:  (n) lBrute force algorithm:  Compare each element to every other:  (n 2 ) lConclusion: presorting yields significant improvement lSimilar improvement for mode lWhat about searching? What about amortised searching?  (n lgn)

9. Balancing Trees lSearching, insertion and deletion in a Binary Search Tree:  Balanced =  (log n)  Unbalanced =  (n) lMust somehow enforce balance lInstance Simplification:  AVL and Red-black trees constrain imbalances by restructuring trees using rotations lRepresentation Change:  2-3 Trees and B-Trees attain perfect balance by allowing more than one element in a node

10. Heapsort: Representation Change lA heap is a binary tree with the following conditions:  It is essentially complete  The key at each node is ≥ keys at its children  The root has the largest key  The subtree rooted at any node of a heap is also a heap lHeapsort Algorithm: 1.Build heap 2.Remove root – exchange with last (rightmost) leaf 3.Fix up heap (excluding last leaf) 4.Repeat 2, 3 until heap contains just one node lEfficiency:  (n) +  (n log n) =  (n log n) in both worst and average cases  Unlike Mergesort it is in place

11. Horner’s Rule: Representation Change Horner’s Rule: Representation Change lAddresses the problem of evaluating a polynomial p(x) = a n x n + a n-1 x n-1 + … + a 1 x + a 0 at a given point x = x 0 lRe-invented by W. Horner in early 19th Century lApproach: Convert to p(x) = (… (a n x + a n-1 ) x + …) x + a 0 lAlgorithm: lExample:  Q(x) = 2x 3 - x 2 - 6x + 5 at x = 3  P[ ]:2-1-65  p: 23*2 + (-1) = 53*5 + (-6) = 93*9 + 5 = 32 p  P[n] FOR i  n -1 DOWNTO 0 DO p  x  p + P[i] RETURN p

12. Notes on Horner’s Rule lEfficiency:  Brute Force =  (n 2 )  Transform and Conquer =  (n) lHas useful side effects:  Intermediate results are coefficients of the quotient of p(x) divided by x - x0 lAn optimal algorithm (if no preprocessing of coefficients allowed) lBinary exponentiation:  Also uses ideas of representation change to calculate a n by considering the binary representation of n

13. Problem Reduction lIf you need to solve a problem reduce it to another problem that you know how to solve lUsed in Complexity Theory to classify problems lComputing the Least Common Multiple:  The LCM of two positive integers m and n is the smallest integer divisible by both m and n  Problem Reduction: LCM(m, n) = m * n / GCD(m, n)  Example: LCM(24, 60) = 1440 / 12 = 120 lReduction of Optimization Problems:  Maximization problems seek to find a function’s maximum. Conversely, minimization seeks to find the minimum  Can reduce between: min f(x) = - max [ - f(x) ]

14. Reduction to Graph Problems lAlgorithms such as Breadth First and Depth First Traversal available after reduction to a graph rep lState-Space Graphs:  Vertices represent states and edges represent valid transitions between states  Often with an initial and goal vertex  Widely used in AI lExample: The River Crossing Puzzle [(P)easant, (w)olf, (g)oat, (c)abbage] Pwgc | |Pwc | | gPgc | | wPwg | | cPg | | wc wc | | Pgc | | Pwgw | | Pgcg | | Pwc | | Pwgc

15. Strengths and Weaknesses of Transform-and-Conquer Strengths:  Allows powerful data structures to be applied  Effective in Complexity Theory ûWeaknesses:  Can be difficult to derive (especially reduction)