1. Computer Science/Ch. Algorithmic Foundation of CS 4-1 Chapter 4 Chapter 4 Algorithmic Foundation of Computer Science

2. Computer Science/Ch. Algorithmic Foundation of CS 4-2 What is an algorithm? Example: recipe oven baker ingredients cake software hardware algorithm Software : algorithm Baking a cake

3. Computer Science/Ch. Algorithmic Foundation of CS 4-3 What is an algorithm? Formal definitions: –A sequence of operations that can solve a specified problem –A sequence of computational steps that transform the input into the output –A tool for solving a well-specified computational problem –A well-ordered collection of unambiguous and effectively computable operations that when executed produces a result and halts in a finite amount of time Algorithmics: –The area of human study, knowledge, and expertise that concerns algorithms.

4. Computer Science/Ch. Algorithmic Foundation of CS 4-4 History 400 and 300 B.C. : –Greek mathematician: Euclid. –GCD (greatest common divisor) –The first non-trivial algorithm Algorithms came from: –Persian mathematician: Mohammed al-Khowarizmi –When written in Latin: Algorismus Key person : Alan Turing –English mathematician –Turing Machine: Finite Automata »Develop some results of the theory of algorithms, that concern the capabilities and limitations of machine- executable algorithms –Turing Award: » Nobel Prize of Computer Science

5. Computer Science/Ch. Algorithmic Foundation of CS 4-5 Problem and Instance Problem: –sorting problem: »sorting a sequence of numbers into nondecreasing order –input : A sequence of n number –Output: A permutation (reordering) of input sequence such that a 1 '<a 2 '<... < a n '. –Example: => Instance : –a particular input sequence of a problem Correct algorithm : –for every input sequence, the algorithm halts with the correct output.

6. Computer Science/Ch. Algorithmic Foundation of CS 4-6 Why study algorithms? More needs of high speed computations How to accomplish that? –High speed computers –Highly efficient algorithms Note: High speed computers High speed computations T T : time N : # of data elements Quicksort on PC/XT Insertion sort on VAX 8800 N

7. Computer Science/Ch. Algorithmic Foundation of CS 4-7 How to solve problems? Algorithmic methods: –Getting it done methodically Correctness of algorithms: –Getting it done right –Prove the correctness of algorithms –How? »By induction... Efficiency of algorithms: –Getting it done cheaply or quickly Inefficiency and Intractability: –You can't always get it done cheaply Noncomputability and Undecidability –Sometimes you can't get it done at all

8. Computer Science/Ch. Algorithmic Foundation of CS 4-8 Performance of algorithms How to determine the performance of an algorithm ? –depends on the size of input. –use special notations to denote the growth rate. How to distinguish the easy problems and the difficult problems? –the complexity of problems –easy problem »a problem which can be solved by a polynomial time algorithm. –difficult problem »a problem which can only be solved by some exponential time algorithms. »NP-Complete problems.

9. Computer Science/Ch. Algorithmic Foundation of CS 4-9 Difficult problems Partition problem: –Input : S = {1, 7, 10, 9, 5, 8, 3, 13} –Output : S 1 and S 2 s.t. sum of S 1 = sum of S 2. –Example: S 1 = { 1, 10, 9, 8 } S 2 = { 7, 5, 3, 13 } Traveling salesperson problem: –Find a minimum length tour 3 5 6 1 2 4 8 3 3

10. Computer Science/Ch. Algorithmic Foundation of CS 4-10 Easy problems Minimum spanning tree: –Find a tree with minimum length One-center problem: –Find a smallest circle which can cover all points

11. Computer Science/Ch. Algorithmic Foundation of CS 4-11 Analysis of algorithms How to analyze? –Empirical approach »actually running time. –Theoretical approach »determining mathematically the quantity of resources needed by each algorithm as a function of the size of the instances considered. How to measure? –Use a particular step or an elementary operation. –Examples: What measurement? –Use O-notation,  -notation and  -notation. Problem Find x in a list of names Multiply two matrices Sort n integers Operation Comparison Multiplication Comparison/ Data Movement

12. Computer Science/Ch. Algorithmic Foundation of CS 4-12 Complexity Question: –Let A 1 and A 2 be two algorithms that solve the same problem. Let the time complexity of A 1 and A 2 be O(n 2 ) and O(n) respectively. –Would the program for A 2 run faster than that of A 1 ? Answer: –Not exactly! Example: –A 1 : n 2 –A 2 : 100n 100n > n 2, for n < 100. Note:

13. Computer Science/Ch. Algorithmic Foundation of CS 4-13 Complexity How Important Is Order ? –Time Complexity Functions 10 10 2 10 3 10 4 3.3 6.6 10 13.3 10 10 2 10 3 10 4 0.33*10 2 0.7*10 3 10 4 1.3*10 5 1024 1.3*10 30 >10 100 >10 100 3*10 6 >10 100 >10 100 >10 100 lg n n nlg n 2 n n! functions problem size

14. Computer Science/Ch. Algorithmic Foundation of CS 4-14 What complexity? What complexity do we need? –Best-case complexity –Worst-case complexity –Average-case complexity Definition: –Let D n be the set of inputs of size n for the problem under consideration, and let I be an element of D n. –Let t(I) be the number of basic operations performed by the algorithm on input I. –Worst-Case Complexity : W(n) »W(n) = max { t(I) | I  D n } –Best-Case Complexity : B(n) »B(n) = min { t(I) | I  D n } –Average-Case Complexity: A(n) »p(I) : probability that input I occurs. A(n) =  p(I) t(I) IDnIDn

15. Computer Science/Ch. Algorithmic Foundation of CS 4-15 Example of analysis Sequential search: –Problem: Find x in list L. –Basic operation: Comparison of x with a list entry. Analysis: –Best case: »B(n) = 1 –Worst case: »W(n) = n –Average case: Assume all elements are distinct. –Case 1: Suppose x is in L »I i : represent the case where x appears in the i-th position in L. »t(I) : the # of comparisons. »p(I i ) : prob. that I i occurred. »t(I i ) = i, for 1  i  n. => A(n) =  p(I i )t(I i ) =  (1/n) i n i=1 n = (1/n)  i = (1/n)(n(n+1)/2)=(n+1)/2 i=1 n

16. Computer Science/Ch. Algorithmic Foundation of CS 4-16 Example of analysis –Case 2: x si not in L »(n+1) inputs should be considered. »I n+1 : represent the case where x is not in L. »q : prob. that x is in L. »p(I i ) = q/n, for 1 <= i <= n »p(I n+1 ) = 1- q. A(n) =  p(I i )t(I i ) n+1 i=1 n = q((n+1)/2) + (1-q)n When q=1, A(n) = (n+1)/2 When q=1/2, A(n) = (n+1)/4 + n/2 i=1  (q/n)i + (1-q)n = Note: = (q/n)(n(n+1))/ 2+ (1-q)n

17. Computer Science/Ch. Algorithmic Foundation of CS 4-17 Sorting problem Sorting problem: –Input: a list of numbers –Output: increasing order –Example:5, 7, 2, 8, 3 => 2, 3, 5, 7, 8 Selection sort: 1.Get values for n and the n items 2.Set the marker for the unsorted section at the end of the list 3.Repeat steps 4 thru 6 until the unsorted section of the list is empty 4.Select the largest number in the unsorted section of the list 5.Exchange this number with the last number in the unsorted section 6. Move the marker for the unsorted section forward one position 7.Stop 5, 7, 2, 8, 3 | 5, 7, 2, 3 | 8 5, 3, 2 | 7, 8 2, 3 | 5, 7, 8 2 | 3, 5, 7, 8 | 2, 3, 5, 7, 8 # of comparisons: (n-1)+(n-2)+... + 2 + 1 = n*(n-1)/2

18. Computer Science/Ch. Algorithmic Foundation of CS 4-18 Binary search X < X > X Time: # of comparisons: lg n

19. Computer Science/Ch. Algorithmic Foundation of CS 4-19 Pattern matching Pattern matching problem: –Input: a pattern P and a text T –Output: All occurrences of pattern P –Example: –Text: KLMNPAQKKAQJDS –Pattern:AQ Time:# of comparisons: –Length of T : n –Length of P : m –Best case : O(n) –Worst case: O(m*n)

20. Computer Science/Ch. Algorithmic Foundation of CS 4-20 When things get out of hand Exponential algorithm: –An algorithm which needs O(2 n ) steps to solve the problem Intractable problems: –those problems which have no polynomially bounded algorithm to solve the problem exists Example: –Bin-packing problem: »Given an unlimited number of bins of volume 1 unit, and n objects, all of volume between 0.0 and 1.0, find the minimum number of bins needed to store the n objects. Approximation algorithms: –provide a close approximation to the solution of the problem. –Example: Bin-packing problem »First-Fit