Discrete Mathematics 7th edition, 2009 Chapter 4 Algorithms Introduction Examples of Algorithms Analysis of Algorithms Recursive Algorithms

4.1 Introduction An algorithm has the following characteristics: Input: the algorithm receives input Output: the algorithm produces output Precision: the steps are precisely stated Determinism: the intermediate results of each step of execution are unique and are determined only by the inputs and the results of the preceding steps Finiteness: the algorithm terminates; i.e., it stops after finitely many instructions have been executed Correctness: the output produced by the algorithm is correct; i.e., the algorithm correctly solves the problem Generality: the algorithm applies to a sets of inputs

Example: Finding the Maximum of Three Numbers Algorithm to find the largest of three numbers a, b, c. Input : a, b, c Output : large (the largest of a, b, and c) max3(a,b,c) { large = a if (b>large) // if b is larger than large, update large large = b If (c > large) // if c is larger than large, update large large = c return large }

4.2 Examples of Algorithms Searching – Text Search Algorithm searches for an occurrence of the pattern p in text t. It returns the smallest index i such that p occurs in t starting at index i. If p does not occur in t, it returns 0. Input: p (indexed from 1 to m), m, t (indexed 1 to n), n Output: i text_search(p,m,t,n) { for i=1 to n-m+1 { /* p=“hello”, t=“hello world !”, m = 5, n = 13 */ j=1 // i is the index in t of the first character of the substring // to compare with p, and j is the index in p // the while loop compares ti…ti+m-1 and p1…pm while (ti+j-1 == pj) { j = j+1 if (j>m) return I /* found something */ } return 0 /* did not find anything */

Examples of Algorithms Sorting – Insertion Sort This algorithm sorts the sequence s1, …, sn in nondecreading order Input: s, n Output: s (sorted) insertion_sort(s,n) { for i=2 to n { val = si // save si so it can be inserted into the corrected place j = i-1 // if val < sj, move sj right to make room for si while (j1  val<sj) { sj+1 = sj j = j-1 } sj+1 = val // insert val

Insertion Sort Given: 4 2 6 1 2 Initialize: 4 / 2 6 1 2 val = 2 (insert 2 into the left) 2 4 / 6 1 2 val = 6 (insert 6 into the left) 2 4 6 / 1 2 val = 1 (insert 1 into the left) 1 2 4 6 / 2 1 2 2 4 6

Randomized Algorithms Relaxation the requirements of an algorithm Finiteness An operating system never terminates Determinism Algorithms for multiprocessor machine or distributed environment are rarely deterministic Generality or Correctness Many practical problems are too difficult to be solved efficiently

Randomized Algorithms does not require that the intermediate results of each step of execution be uniquely defined depend only on the inputs and results of the proceeding steps at some points it makes random choice

Randomized Algorithms major bridge tournaments use computer programs to shuffle the cards shuffle algorithm assume that a function rand(i,j) exists returns a random integer x, i  x  j Input: a, n Output: a (shuffled) shuffle(a,n) { for i=1 to n-1 swap(ai, arand(i,n)) }

4.3 Analysis of Algorithms Useless program for certain type of input even though derived from a correct algorithm the time needed to run the program is too great or the space needed to hold the data is too great Example X = a set of n elements some elements are labeled ‘red’, some labeled ‘black’. Find the number of subsets of X that contain at least one red item. Need to examine 2n subsets. What if n is very big?

Complexity of algorithms The time needed to execute an algorithm is a function of the input is difficult to obtain an explicit formula instead of dealing directly with the input, we use parameters that characterize the size of the input we estimate the time of an algorithm rather than computing its exact time

Complexity of algorithms Complexity: the amount of time and/or space needed to execute the algorithm. Performance parameters of a computer program Computer that is being used to run the program The way the data are represented How the program is translated to machine instructions What kind of computer language is used

Types of complexity Best-case time Worst-case time Average-case time minimum time needed to execute the algorithm for inputs of size n Worst-case time maximum time needed to execute the algorithm for inputs of size n Average-case time average time needed

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} f(n) = O(g(n)) f(n) is of order at most g(n) if there exists a positive constant C1 such that |f(n)|  C1|g(n)| for all but finitely many n f(n) is of order at most g(n) or f(n) is big oh of g(n) asymptotic upper bound for f Example 1 60n2 + 5n + 1  60n2 +5n2 + n2 = 66n2 for all n1, C1 = 66  60n2 + 5n + 1 = O(n2) Example 2 2n + 3 lg n  2n +3n = 5n for all n1, C1 = 5  2n + 3 lg n = O(n)

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} f(n) = (g(n)) f(n) is of order at least g(n) if there exists a positive constant C2 such that |f(n)| > C2|g(n)| for all but finitely many n f(n) is of order at least g(n) or f(n) is omega of g(n) asymptotic lower bound for f Example 1 60n2 + 5n + 1  60n2 for all n1, C1 = 60  60n2 + 5n + 1 = (n2) Example 2 2n + 3 lg n  2n for all n1, C1 = 2  2n + 3 lg n = (n)

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} f(n) = (g(n)) f(n) is or order g(n) if it is O(g(n)) and (g(n)). f(n) is of order g(n) or f(n) is theta of g(n) asymptotic tight bound for f Example 1 60n2 + 5n + 1 = O(n2) and 60n2 + 5n + 1 = (n2)  60n2 + 5n + 1 = (n2) Example 2 2n + 3 lg n = O(n) and 2n + 3 lg n = (n)  2n + 3 lg n = (n)

Order of an algorithm Theorem 4.3.4 Let p(n) = aknk + ak-1nk-1 + … + a1n + a0 be a polynomial in n of degree k, where each ai is nonnegative. Then p(n) = (nk) Proof 1. p(n) = O(nk) Let C1 = ak + ak-1 + … + a1 + a0. Then, for all n, p(n) = aknk + ak-1nk-1 + … + a1n + a0  (ak + ak-1 + … + a1 + a0) nk = C1 nk Therefore, p(n) = O(nk) 2. p(n) =  (nk) For all n, p(n) = aknk + ak-1nk-1 + … + a1n + a0  aknk = C2 nk Therefore, p(n) = (nk) 3. Since p(n) = O(nk) and p(n) = (nk), p(n) = (nk)

Growth of some common functions (1) (lg lg n) (lg n) (n) (n lg n) (n2) (n3) (nk), k1 (cn), c1 (n!) Theta Form Name Constant Log log Log Linear n log n Quadratic Cubic Polynomial Exponential Factorial y 256 128 64 32 16 8 4 2 1 1 2 3 4 5 6 7 8 9 10 11 12 n y=2n y=n2 y=nlgn y=n y=lgn y=1

Examples 1 + 2 + … + n 1 + 2 + … + n  n + n + … + n = nn = n2 for all n1 1 + 2 + … + n = O(n2) 1 + 2 + … + n  n/2 + … + (n-1) + n  n/2 + … + n/2 + n/2 = (n+1)/2 n/2  (n/2)(n/2) = n2/4 for all n1 1 + 2 + … + n = (n2) 1 + 2 + … + n = O(n2) and 1 + 2 + … + n = (n2) 1 + 2 + … + n = (n2)

Examples lg n! lg n! = lg n + lg (n-1) + … + lg 2 + lg 1 for all n1  lg n + lg n + … + lg n + lg n = n lg n for all n1 lg n! = O(n lg n) lg n + lg (n-1) + … + lg 1  lg n + lg (n-1) + … + lg n/2  lg n/2 + … + lg n/2 = (n+1)/2 lg n/2  (n/2) lg (n/2) = (n/2) [lg n – lg 2] = (n/2) [(lg n)/2 + ((lg n)/2 -1)]  (n/2)(lg n)/2 = (n lg n)/4 for all n4 lg n! = (n lg n) lg n! = O(n lg n) and n! = (n lg n) lg n! = (n lg n)

4.4 Recursive algorithms A recursive procedure is a procedure that invokes itself A recursive algorithm is an algorithm that contains a recursive procedure

Factorial of n Definition given a positive integer n, factorial of n is defined as the product of n by all numbers less than n and greater than 0. Notation n! = n(n-1)(n-2)…321 Notes n! = n(n-1)! = n(n-1)(n-2)!, etc.

Factorial of n Algorithm 4.4.2 Input: n, an integer greater than or equal to 0 Output: n! factorial (n) { if (n == 0) return 1 return n * factorial(n-1) }

Factorial of n Theorem 4.4.3 Algorithm 4.4.2 returns the value of n!, n0 Proof 1. Basis Step (n=0) If n=0, Algorithm 4.4.2 correctly returns the value of 0! (1) 2. Inductive Step Assume that Algorithm 4.4.2 correctly returns the value of (n-1)!, n>0. Suppose that n is input to Algorithm 4.4.2. Since n0, we proceed to line 4 of the algorithm. By the inductive assumption, the function computes the value of (n-1)! At line 4, the function correctly computes the value (n-1)!n = n! 3. conclusion Therefore, Algorithm 4.4.2 correctly returns the value of n!, for every integer n0.

Fibonacci sequence Leonardo Fibonacci (Pisa, Italy, ca. 1170-1250) Fibonacci sequence f1, f2,… defined recursively as follows: f1 = 1 f2 = 2 fn = fn-1 + fn-2 for n > 3 First terms of the sequence 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597,…

Robot Walking a2 an-1 a1 a1 = 1 a2 = 1+1 an-2 an = an-1 + an-2 How many ways to reach n steps: Constraint: 1 or 2 step possible at once a2 an-1 a1 a1 = 1 a2 = 1+1 an-2 N step 을 가기위해 마지막에 1 혹은 2 step 이용 방법을 도합 an = an-1 + an-2