1. Design and Analysis of Algorithms - Chapter 61 Transform and Conquer Solve problem by transforming into: b a more convenient instance of the same problem (instance simplification) Presorting, Gaussian elimination, matrix inversion, determinant computationPresorting, Gaussian elimination, matrix inversion, determinant computation b a different representation of the same instance (representation change) balanced search trees, heaps and heapsort, polynomial evaluation by Horner’s rule, Fast Fourier Transformbalanced search trees, heaps and heapsort, polynomial evaluation by Horner’s rule, Fast Fourier Transform b a different problem altogether (problem reduction) reductions to graph problems, linear programmingreductions to graph problems, linear programming

2. Design and Analysis of Algorithms - Chapter 62 Instance simplification - Presorting Solve instance of problem by transforming into another simpler/easier instance of the same problem Presorting: Many problems involving lists are easier when list is sorted. b element uniqueness b computing the mode b finding repeated elements b searching b computing the median (selection problem)

3. Design and Analysis of Algorithms - Chapter 63 Selection Problem Find the k-th smallest element in A[1],…A[n]. minimum: k = 1minimum: k = 1 maximum: k = nmaximum: k = n median: k = n/2median: k = n/2 b Presorting-based algorithm sort listsort list return A[k]return A[k] b Partition-based algorithm (decrease & conquer): pivot/split at A[s] using partitioning algorithmpivot/split at A[s] using partitioning algorithm if s=k return A[s]if s=k return A[s] else if s<k repeat with sublist A[s+1],…A[n].else if s<k repeat with sublist A[s+1],…A[n]. else if s>k repeat with sublist A[1],…A[s-1].else if s>k repeat with sublist A[1],…A[s-1].

4. Design and Analysis of Algorithms - Chapter 64 Notes on Selection Problem b Presorting-based algorithm: Ω(nlgn) + Θ(1) = Ω(nlgn) b Partition-based algorithm (decrease & conquer): worst case: T(n) =T(n-1) + (n+1) -> Θ(n 2 )worst case: T(n) =T(n-1) + (n+1) -> Θ(n 2 ) best case: Θ(n)best case: Θ(n) average case: T(n) =T(n/2) + (n+1) -> Θ(n)average case: T(n) =T(n/2) + (n+1) -> Θ(n) Bonus: also identifies the k smallest elementsBonus: also identifies the k smallest elements b Special cases max, min: better, simpler linear algorithm (brute force) b Conclusion: Presorting does not help in this case.

5. Design and Analysis of Algorithms - Chapter 65 Finding repeated elements b Presorting-based algorithm: use mergesort (optimal): Θ(nlgn)use mergesort (optimal): Θ(nlgn) scan array to find repeated adjacent elements: Θ(n)scan array to find repeated adjacent elements: Θ(n) in total it makes: Θ(nlgn)in total it makes: Θ(nlgn) b Brute force algorithm: Θ(n 2 ) b Conclusion: Presorting yields significant improvement

6. Design and Analysis of Algorithms - Chapter 66 Checking element uniqueness b Brute force algorithm: Θ(n 2 ) b Algorithm PresortedElementUniqueness Sort the array A for i  0 to n-2 do if A[i]=A[i+1] return false else return true b Conclusion: Presorting again improves b Similar improvement for mode

7. Design and Analysis of Algorithms - Chapter 67 Checking mode b Mode is the most often met element b Brute force: scan the list and compute the frequencies. Then find the largest frequency b Algorithm PresortedMode Sort the array A i  1; modefrequency  0; while i  n-1 do runlength  1; runvalue  A[i]; while i+runlength≤n-1 and A[i+runlength]=runvalue runlength  runlength+1 if runlength> modefrequency modefrequency  runlength, modevalue  runvalue i  i+runlength return modevalue b Conclusion: Presorting again improves

8. Design and Analysis of Algorithms - Chapter 68 Gaussian Elimination b Given a system of two linear equations with two unknowns a 11 x + a 12 y = b 1 a 21 x + a 22 y = b 2 b It has a unique solution unless the coefficients are proportional b Express one variable as function of the other and substitute to solve one equation. b What if the system has n equations and n unknowns ?

9. Design and Analysis of Algorithms - Chapter 69 Gaussian Elimination (2) b Transform Ax=b to A’x=b’, where A’ upper triangular b Then, solution is possible with backward substitution b Elementary operations Exchange equationsExchange equations Replace an equation with a nonzero multipleReplace an equation with a nonzero multiple Replace an equation with a sum or difference of this equation and some multiple of another equationReplace an equation with a sum or difference of this equation and some multiple of another equation b Example 2x 1 - x 2 + x 3 = 1 4x 1 + x 2 - x 3 = 5 x 1 + x 2 + x 3 = 0

10. Design and Analysis of Algorithms - Chapter 610 Gaussian Elimination (3) b Algorithm GaussElimination for i  1 to n do A[i,n+1]  b[i] for i  1 to n-1 do for j  i+1 to n do for k  i to n+1 do A[j,k]  A[j,k]-A[i,k]*A[j,i]/A[i,i] b Potential problems if A[i,i] is zero or very small

11. Design and Analysis of Algorithms - Chapter 611 Gaussian Elimination (partial pivoting) b Algorithm GaussElimination2 for i  1 to n do A[i,n+1]  b[i] for i  1 to n-1 do pivotrow  i for j  i+1 to n do if |A[j,i]|>|A[pivot,i]| pivotrow  j for k  i to n+1 do swap(A[i,k],A[pivotrow,k]) for j  i+1 to n do temp  A[j,i]/A[i,i] for k  I to n+1 do A[j,k]  A[j,k]-A[I,k]*temp b Efficiency

12. Design and Analysis of Algorithms - Chapter 612 LU decomposition b Byproduct of Gaussian Elimination b Example A=LU b LUx=b. Denote y=Ux  Ly=b b Solve Ly=b, then solve Ux=y b Solve with as many times with different b’s. b No extra space 100 210 1/21/212103-3 002

13. Design and Analysis of Algorithms - Chapter 613 Computing a matrix inverse b AA -1 =I b A singular matrix does not have an inverse b A matrix is singular if and only if one of the rows is a linear combination of the other rows. b Apply Gaussian elimination. If it yields an upper-triangular with no zeros on the diagonal, then the matrix is not singular b Ax j =e j

14. Design and Analysis of Algorithms - Chapter 614 Computing the determinant b A well-known recursive formula b What if n is large ? Efficiency ? b Apply Gaussian elimination. b The determinant of an upper-triangular matrix is the product of elements on its diagonal. b Efficiency ? b Cramer’s rule

15. Design and Analysis of Algorithms - Chapter 615 Taxonomy of Searching Algorithms b Elementary searching algorithms sequential searchsequential search binary searchbinary search binary tree searchbinary tree search b Balanced tree searching AVL treesAVL trees red-black treesred-black trees multiway balanced trees (2-3 trees, 2-3-4 trees, B trees)multiway balanced trees (2-3 trees, 2-3-4 trees, B trees) b Hashing separate chainingseparate chaining open addressingopen addressing

16. Design and Analysis of Algorithms - Chapter 616 Balanced trees: AVL trees b For every node, difference in height between left and right subtree is at most 1 b AVL property is maintained through rotations, each time the tree becomes unbalanced b lg n ≤ h ≤ 1.4404 lg (n + 2) - 1.3277 average: 1.01 lg n + 0.1 for large n b Disadvantage: needs extra storage for maintaining node balance b A similar idea: red-black trees (height of subtrees is allowed to differ by up to a factor of 2)

17. Design and Analysis of Algorithms - Chapter 617 AVL tree rotations b Small examples: 1, 2, 31, 2, 3 3, 2, 13, 2, 1 1, 3, 21, 3, 2 3, 1, 23, 1, 2 b Larger example: 4, 5, 7, 2, 1, 3, 6 b See figures 6.4, 6.5 for general cases of rotations;

18. Design and Analysis of Algorithms - Chapter 618 General case: single R-rotation

19. Design and Analysis of Algorithms - Chapter 619 Double LR-rotation

20. Design and Analysis of Algorithms - Chapter 620 Balance factor b Algorithm maintains balance factor for each node. For example:

21. Design and Analysis of Algorithms - Chapter 621 Heapsort Definition: A heap is a binary tree with the following conditions: b it is essentially complete: b The key at each node is ≥ keys at its children

22. Design and Analysis of Algorithms - Chapter 622 Definition implies: b Given n, there exists a unique binary tree with n nodes that is essentially complete, with h= lg n b The root has the largest key b The subtree rooted at any node of a heap is also a heap

23. Design and Analysis of Algorithms - Chapter 623 Heapsort Algorithm: 1. Build heap 2. Remove root –exchange with last (rightmost) leaf 3. Fix up heap (excluding last leaf) Repeat 2, 3 until heap contains just one node.

24. Design and Analysis of Algorithms - Chapter 624 Heap construction b Insert elements in the order given breadth-first in a binary tree b Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does not satisfy the heap condition: 1.exchange it with its largest child 2.fix the subtree rooted at it (now in the child’s position) Example: 2 3 6 7 5 9

25. Design and Analysis of Algorithms - Chapter 625 Root deletion The root of a heap can be deleted and the heap fixed up as follows: b exchange the root with the last leaf b compare the new root (formerly the leaf) with each of its children and, if one of them is larger than the root, exchange it with the larger of the two. b continue the comparison/exchange with the children of the new root until it reaches a level of the tree where it is larger than both its children

26. Design and Analysis of Algorithms - Chapter 626 Representation b Use an array to store breadth-first traversal of heap tree: b Example: b Left child of node j is at 2j b Right child of node j is at 2j+1 b Parent of node j is at j /2 b Parental nodes are represented in the first n /2 locations 9 1 53 42 1 2 3 4 5 6 9 5 3 1 4 2

27. Design and Analysis of Algorithms - Chapter 627 Bottom-up heap construction algorithm

28. Design and Analysis of Algorithms - Chapter 628 Analysis of Heapsort See algorithm HeapBottomUp in section 6.4 b Fix heap with “problem” at height j: 2j comparisons b For subtree rooted at level i it does 2(h-i) comparisons b Total for heap construction phase: Σ 2(h-i) 2 i = 2 ( n – lg (n + 1)) = Θ(n) i=0 h-1 # nodes at level i

29. Design and Analysis of Algorithms - Chapter 629 Analysis of Heapsort (continued) Recall algorithm: 1.Build heap 2.Remove root –exchange with last (rightmost) leaf 3.Fix up heap (excluding last leaf) Repeat 2, 3 until heap contains just one node. Θ(n)Θ(n) Θ(log n) n – 1 times Total: Total: Θ(n) + Θ( n log n) = Θ(n log n) Note: Note: this is the worst case. Average case also Θ(n log n).

30. Design and Analysis of Algorithms - Chapter 630 Priority queues b A priority queue is the ADT of an ordered set with the operations: find element with highest priorityfind element with highest priority delete element with highest prioritydelete element with highest priority insert element with assigned priorityinsert element with assigned priority b Heaps are very good for implementing priority queues

31. Design and Analysis of Algorithms - Chapter 631 Insertion of a new element b Insert element at last position in heap. b Compare with its parent and if it violates heap condition exchange them b Continue comparing the new element with nodes up the tree until the heap condition is satisfied Example:Efficiency:

32. Design and Analysis of Algorithms - Chapter 632 Bottom-up vs. Top-down heap construction b Top down: Heaps can be constructed by successively inserting elements into an (initially) empty heap b Bottom-up: Put everything in and then fix it b Which one is better?

33. Design and Analysis of Algorithms - Chapter 633 Horner’s rule b Horner published in early 19 th century b According to Knuth, the method was used by Newton b Evaluate a polynomial at a point x p(x) = a n x n + a n-1 x n-1 + … + a 1 x + a 0 p(x) = ( … (a n x + a n-1 ) x + … )x + a 0 b Example: evaluate p(x)=2x 4 -x 3 +3x 2 +x-5 at x=3 p(x) = x (x (x (2x-1) + 3) + 1) - 5 b Visualization by a table

34. Design and Analysis of Algorithms - Chapter 634 Horner’s rule [2] b Algorithm Horner(P[0..n],x) // Evaluate polynomial at a given point // Input: an array P[0..n] of coefficients and a number x //Output: the value of polynomial at point x p  P[n] for i  n-1 down to 0 do p  x*p + P[i] return p b Efficiency ? b Byproduct: coefficients of the quotient of the division of p(x) by (x-x 0 )

35. Design and Analysis of Algorithms - Chapter 635 Binary exponentiation b Horner is not efficient to compute p(x)=x n at x=a b Degenerate to brute force b Let the binary representation n=b l b l-1 … b i … b 1 b 0 b p(x) = b l x l + b l-1 x l-1 + … + b 1 x + b 0 and x=2 b Algorithm LeftRightBinaryExponentiation product  a for i  l-1 down to 0 do product  product * product if b i  1 then product  product*a return product b Example: compute a 13, n=13=1101 b Efficiency

36. Design and Analysis of Algorithms - Chapter 636 Binary exponentiation (2) b Compute a n b Consider n = b l 2 l + b l-1 2 l-1 + … + b 1 2 + b 0 and multiply independent powers terms b Algorithm RightLeftBinaryExponentiation term  a if b 0 =1 then product  a else product  1 for i  1 to l do term  term * term if b i = 1 then product  product * term return product b Example: compute a 13, n=13=1101 b Efficiency

37. Design and Analysis of Algorithms - Chapter 637 Least common multiple b lcm(24,60)=120, lcm(11,5)=55 b Example: 24 = 2 x 2 x 2 x 3 60 = 2 x 2 x 3 x 5 lcm(24,60) = (2x2x3) x 2 x 5 b Efficiency (a list of primes is required) b lcm(m,n) = mn / gcd(m,n)

38. Design and Analysis of Algorithms - Chapter 638 Counting paths in a graph b The number of different paths of length k>0 from node i to node j equals the (i,j) element of the A k, where A the adjacency matrix b Example b Efficiency

39. Design and Analysis of Algorithms - Chapter 639 Reduction to graph problems b Applies for a variety of games and puzzles b Build the state-space graph b Example: peasant, wolf, goat, cabbage b Traverse the graph by applying what?