Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Towers of Hanoi (from Wikipedia) If n>1, then somewhere along the sequence of moves, the largest disk must be moved from peg f to another peg, preferably to peg t. If n>1, then somewhere along the sequence of moves, the largest disk must be moved from peg f to another peg, preferably to peg t. The only situation that allows this move is when all smaller n-1 disks are on peg r. The only situation that allows this move is when all smaller n-1 disks are on peg r. First all h-1 smaller disks must go from f to r.First all h-1 smaller disks must go from f to r. Move the largest diskMove the largest disk Finally move the h-1 smaller disks from peg r to peg t.Finally move the h-1 smaller disks from peg r to peg t. Now the problem is reduced to moving h-1 disks from one peg to another one, first from f to r and subsequently from r to t Now the problem is reduced to moving h-1 disks from one peg to another one, first from f to r and subsequently from r to t The same method can be used both times by renaming the pegs.The same method can be used both times by renaming the pegs. The same strategy can be used to reduce the h-1 problem to h-2, h-3, and so on until only one disk is left. This is called recursion.The same strategy can be used to reduce the h-1 problem to h-2, h-3, and so on until only one disk is left. This is called recursion.recursion 1-2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Towers of Hanoi The following is a procedure for moving a tower of h disks from a peg f onto a peg t, with r being the remaining third peg: The following is a procedure for moving a tower of h disks from a peg f onto a peg t, with r being the remaining third peg: Step 1: If h>1 then first move the h-1 smaller disks from peg f to peg r.Step 1: If h>1 then first move the h-1 smaller disks from peg f to peg r. Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f to peg t.Step 2: Now the largest disk, i.e. disk h-1 can be moved from peg f to peg t. Step 3: If h>1 then again use this procedure to move the h-1 smaller disks from peg r to peg t.Step 3: If h>1 then again use this procedure to move the h-1 smaller disks from peg r to peg t. 1-3

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Example 2: The Tower of Hanoi Puzzle Recursive Solution (Move n>1 disks from P1 to P3 Move recursively n-1 disks from P1 to P2 (P3 is Aux) Then move the largest disk from P1 to P3 Move recursively n-1 disks from P2 to P3 (P1 is Aux) Recursive Algorithm Analysis M(n) = M(n-1) M(n-1) for n>1, M(1) = 1 M(n) = 2M(n-1)+1, M(1) = 1 1-4

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Example 2: The Tower of Hanoi Puzzle Recurrence for number of moves:

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Solving recurrence for number of moves M(n) = 2M(n-1) + 1, M(1) = 1 Use Backward Substitution M(n) = 2M(n-1)+1 =2[2M(n-2)+1]+1 = 2 2 M(n-2)+2+1 =2[2M(n-2)+1]+1 = 2 2 M(n-2)+2+1 =2 2 [2M(n-3)+1]+2+1 = 2 3 M(n-3) =2 2 [2M(n-3)+1]+2+1 = 2 3 M(n-3) Pattern M(n) = 2 i M(n-i)+2 i-1 +2 i-2 +…+2+1 M(n) = 2 i M(n-i)+2 i-1 +2 i-2 +…+2+1 = 2 i M(n-i)+2 i -1 = 2 i M(n-i)+2 i -1 Initial condition: n=1 or i = n-1 Initial condition: n=1 or i = n-1 = 2 n -1 exponential algorithm : still optimal = 2 n -1 exponential algorithm : still optimal

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Tree of calls for the Tower of Hanoi Puzzle

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Example 3: Counting #bits Basic operation: additions Recurrence relation: A(n) = A(floor(n/2)) + 1 for n>1, A(1) = 0 Problem: floor(n/2) makes backward substitution not work when n is not a power of 2 Estimate: for only cases where n is a power of 2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Slightly Updated Recurrence A(2 k ) = A(2 k-1 ) + 1 for k>0, A(2 0 ) = 0 A(2 k ) = A(2 k-1 ) + 1 for k>0, A(2 0 ) = 0 Use backward substitution Use backward substitution A(2 k ) = A(2 k-1 ) + 1 = [A(2 k-2 ) + 1]+1 = A(2 k-2 ) + 2 = [A(2 k-3 ) + 1]+2 = A(2 k-3 ) + 3 … = A(2 k-i ) + i … = A(2 k-k ) + k Thus, A(2 k ) = A(1) + k = k n=2 k and hence k = log 2 n n=2 k and hence k = log 2 n A(n) = log2n = Θ(log n) A(n) = log2n = Θ(log n) 1-9

Chapter 3 Brute Force Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Brute Force A straightforward approach, usually based directly on the problems statement and definitions of the concepts involved Just do it, and dont be fancy Examples: Computing a n (a > 0, n a nonnegative integer) Computing a n (a > 0, n a nonnegative integer) Computing n! Computing n! Multiplying two matrices Multiplying two matrices Searching for a key of a given value in a list Searching for a key of a given value in a list

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 So, When and Why Brute Force Why not brute force? Why not brute force? Youll never get an elegant algorithmYoull never get an elegant algorithm Youll never give yourself a chance to see the hidden patterns necessary for getting an optimal solutionYoull never give yourself a chance to see the hidden patterns necessary for getting an optimal solution For problems where solutions that are better than all pairs exist, brute force will not be optimalFor problems where solutions that are better than all pairs exist, brute force will not be optimal Why brute force? Why brute force? Exceedingly easy to conceptualizeExceedingly easy to conceptualize The method can be applied to a large range of problems (not true for a lot of the other strategies well see)The method can be applied to a large range of problems (not true for a lot of the other strategies well see) Works well for small values (input sizes) and its easy to implementWorks well for small values (input sizes) and its easy to implement Expense of a more advanced algorithm may not be justifiedExpense of a more advanced algorithm may not be justified –Only a few difficult instances to solve Makes a good yard stickMakes a good yard stick –i.e. We can do at least this well very easily

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Brute-Force Sorting Algorithm Selection Sort Scan the array to find its smallest element and swap it with the first element. Then, starting with the second element, scan the elements to the right of it to find the smallest among them and swap it with the second elements. Generally, on pass i (0 i n-2), find the smallest element in A[i..n-1] and swap it with A[i]: A[0]... A[i-1] | A[i],..., A[min],..., A[n-1] in their final positions in their final positions Example: Example:

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Analysis of Selection Sort

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Analysis Time efficiency: C (n) = i=0 to n-2 j=i+1 to n-1 1C (n) = i=0 to n-2 j=i+1 to n-1 1 = i=0 to n-2 [(n-1)-(i+1)+1] = i=0 to n-2 [(n-1)-(i+1)+1] = i=0 to n-2 (n-1-i) = i=0 to n-2 (n-1-i) = (n-1) + (n-2) + (n-3) + … + 1 = (n-1) + (n-2) + (n-3) + … + 1 by (S2) by (S2) = ((n-1)n)/2 = (n 2 – n) / 2 = ((n-1)n)/2 = (n 2 – n) / 2 approx. = n 2 /2 O(n 2 ) approx. = n 2 /2 O(n 2 ) Space efficiency: Stability: Note: # swaps is only Θ(n), or exactly n-1 -Is this good or bad?

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Brute-Force Sorting Algorithm #2 Bubble Sort Scan the array looking at each consecutive pair of elements. Swap the two elements if they are out of order. Repeatedly scan the array in an all-pairs manner. Bubble Sort Scan the array looking at each consecutive pair of elements. Swap the two elements if they are out of order. Repeatedly scan the array in an all-pairs manner. See pg 100 for the pseudocode See pg 100 for the pseudocode Example: Example: Note: we can do better by stopping if no swaps are needed Note: we can do better by stopping if no swaps are needed i.e. – the list is sortedi.e. – the list is sorted

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Analysis Time efficiency: C (n) = i=0 to n-2 j=0 to n-2-i 1C (n) = i=0 to n-2 j=0 to n-2-i 1 = i=0 to n-2 [(n-2-i)-0+1] = i=0 to n-2 [(n-2-i)-0+1] = i=0 to n-2 (n-1-i) = i=0 to n-2 (n-1-i) = (n-1) + (n-2) + (n-3) + … + 1 = (n-1) + (n-2) + (n-3) + … + 1 by (S2) by (S2) = ((n-1)n)/2 = (n 2 – n) / 2 = ((n-1)n)/2 = (n 2 – n) / 2 approx. = n 2 /2 O(n 2 ) approx. = n 2 /2 O(n 2 ) Space efficiency: Stability: Note: How many swaps must take place in the worst case? That is, look at the code for Selection Sort and Bubble Sort and note where the swap commands are.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Improving Brute Force With brute force, its often possible to do small things that decrease the run-time With brute force, its often possible to do small things that decrease the run-time These dont generally change the asymptotic boundsThese dont generally change the asymptotic bounds Example: Sequential Search Example: Sequential Search Make sure that you have the value you are looking for appended to the end of the listMake sure that you have the value you are looking for appended to the end of the list –That way, the item will have to be found and we wont have to constantly check for the end of the list Obviously, we could first sort our listObviously, we could first sort our list

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Brute-Force String Matching pattern: a string of m characters to search for pattern: a string of m characters to search for text: a (longer) string of n characters to search in text: a (longer) string of n characters to search in problem: find a substring in the text that matches the pattern problem: find a substring in the text that matches the pattern Brute-force algorithm Step 1 Align pattern at beginning of text Step 2 Moving from left to right, compare each character of pattern to the corresponding character in text until –all characters are found to match (successful search); or –a mismatch is detected Step 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat Step 2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Examples of Brute-Force String Matching Pattern: Text: Pattern: Text: Pattern: happy Text: It is never too late to have a happy childhood. Pattern: happy Text: It is never too late to have a happy childhood. b Pattern: happy Text: happhapphapphappy b Pattern: aaab Text: aaaaaaaaaaaaaaaaaaaaaaaaaaaab Text: aaaaaaaaaaaaaaaaaaaaaaaaaaaab

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Pseudocode and Efficiency Efficiency: Worst Case? Expected Average? Expected Average?

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Brute-Force Polynomial Evaluation Problem: Find the value of polynomial p(x) = a n x n + a n-1 x n-1 +… + a 1 x 1 + a 0 p(x) = a n x n + a n-1 x n-1 +… + a 1 x 1 + a 0 at a point x = x 0 at a point x = x 0 Brute-force algorithm Efficiency: p 0.0 for i n downto 0 do power 1 power 1 for j 1 to i do//compute x i for j 1 to i do//compute x i power power x power power x p p + a[i] power p p + a[i] power returnp return p

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2 nd ed., Ch. 1 Polynomial Evaluation: Improvement We can do better by evaluating from right to left: p(x) = a n x n + a n-1 x n-1 +… + a 1 x 1 + a 0 at a point x = x 0 at a point x = x 0 Better brute-force algorithm Efficiency: p a[0] power 1 for i 1 to n do power power x power power x p p + a[i] power p p + a[i] power return p