Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Chapter 1 Introduction

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Why Study Algorithms  All Computer Programs Depend on Algorithms All businesses and many facets of daily life depend on computer programsAll businesses and many facets of daily life depend on computer programs Computers and TelescopesComputers and Telescopes  Learning to Write Excellent Algorithms Means Learning to Use Excellent Problem Solving Techniques 1-2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Quote from Donald Knuth A person well-trained in computer science knows how to deal with algorithms: how to construct them, manipulate them, understand them, analyze them. This knowledge is preparation for much more than writing good computer programs; it is a general-purpose mental tool that will be a definite aid to the understanding of other subjects, whether they be chemistry, linguistics, or music, etc. The reason for this may be understood in the following way: It has often been said that a person does not truly understand something until after teaching it to someone else. Actually, a person does not really understand something until after teaching it to a computer, i.e, expressing it as an algorithm… An attempt to formalize things as algorithms leads to a much deeper understanding than if we simply try to comprehend things in the traditional wayA person well-trained in computer science knows how to deal with algorithms: how to construct them, manipulate them, understand them, analyze them. This knowledge is preparation for much more than writing good computer programs; it is a general-purpose mental tool that will be a definite aid to the understanding of other subjects, whether they be chemistry, linguistics, or music, etc. The reason for this may be understood in the following way: It has often been said that a person does not truly understand something until after teaching it to someone else. Actually, a person does not really understand something until after teaching it to a computer, i.e, expressing it as an algorithm… An attempt to formalize things as algorithms leads to a much deeper understanding than if we simply try to comprehend things in the traditional way 1-3

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch What is an algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. “computer” problem algorithm inputoutput Note: “Computer” – one who computes. Which came first, the chicken or the egg?

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Historical Perspective  Euclid’s algorithm for finding the greatest common divisor Later, we will see three methods (including this one) for how to solve this problemLater, we will see three methods (including this one) for how to solve this problem  Muhammad ibn Musa al-Khwarizmi – 9 th century mathematician

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Example of computational problem: sorting  Statement of problem: Input: A sequence of n numbers Input: A sequence of n numbers Output: A reordering of the input sequence so that a ´ i ≤ a ´ j whenever i so that a ´ i ≤ a ´ j whenever i < j  Instance: The sequence  Instance: The sequence  Algorithms: Selection sortSelection sort Insertion sortInsertion sort Merge sortMerge sort (many others)‏(many others)‏

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Selection Sort  Input: array a[1],…,a[n]  Output: array a sorted in non-decreasing order  Algorithm: for i =1 to n swap a[i] with smallest of a[i],…a[n] see also pseudocode, section 3.1

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Some Well-known Computational Problems  Sorting  Searching  Shortest paths in a graph  Minimum spanning tree  Primality testing  Traveling salesman problem  Knapsack problem  Chess  Towers of Hanoi  Program termination

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Important Points to Consider with Algorithms  Each step of an algorithm must be non-ambiguous. Note: this does not necessarily mean – “must be discrete”Note: this does not necessarily mean – “must be discrete”  The range of inputs for which an algorithm works has to be specified carefully.  The same algorithm can be represented in multiple ways.  Several algorithms for solving the same problem may exist. Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds (also, which dramatically different resource requirements – usually these are opposing considerations)‏Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds (also, which dramatically different resource requirements – usually these are opposing considerations)‏ –Raises the notion of which algorithm is “better” (assuming both algorithms are “correct” 1-9

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Basic Issues Related to Algorithms  How to design algorithms Your understanding of the problem space (which is a prerequisite for solving that kind of problem in general), and your understanding of the algorithm to represent these solutions will necessarily grow and evolve in parallel.Your understanding of the problem space (which is a prerequisite for solving that kind of problem in general), and your understanding of the algorithm to represent these solutions will necessarily grow and evolve in parallel.  How to express algorithms Psuedocode, yesPsuedocode, yes Also, what level of complexity and expressiveness will be necessaryAlso, what level of complexity and expressiveness will be necessary  Proving correctness Mathematical strong proofs of correctness are sometimes possible.Mathematical strong proofs of correctness are sometimes possible.  Efficiency Theoretical analysisTheoretical analysis Empirical analysisEmpirical analysis  Optimality Mathematically, what is the absolute best performance of any algorithm that we can come up with.Mathematically, what is the absolute best performance of any algorithm that we can come up with. If our algorithm has that performance, then it must be optimal.If our algorithm has that performance, then it must be optimal.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Algorithm design strategies  Brute force  Divide and conquer  Decrease and conquer  Transform and conquer  Greedy approach  Dynamic programming  Backtracking and Branch and bound  Space and time tradeoffs

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Analysis of Algorithms  How good is the algorithm? CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace efficiency  Does there exist a better algorithm? Lower boundsLower bounds OptimalityOptimality

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch What is an algorithm?  Recipe, process, method, technique, procedure, routine,… with following requirements: 1. Finiteness  terminates after a finite number of steps 2. Definiteness  rigorously and unambiguously specified 3. Input  valid inputs are clearly specified 4. Output  can be proved to produce the correct output given a valid input 5. Effectiveness  steps are sufficiently simple and basic

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Why study algorithms (part 2)?  Theoretical importance Question: what is computer science?Question: what is computer science?  Practical importance A practitioner’s toolkit of known algorithmsA practitioner’s toolkit of known algorithms Framework for designing and analyzing algorithms for new problemsFramework for designing and analyzing algorithms for new problems

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Euclid’s Algorithm Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ? Euclid’s algorithm is based on repeated application of equality gcd(m,n) = gcd(n, m mod n)‏ until the second number becomes 0, which makes the problem trivial. Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Not impressed?  What is the greatest common divisor of 3456 and 2133? gcd(3885, 1736)‏gcd(3885, 1736)‏ gcd(1736,413)‏gcd(1736,413)‏ gcd(413,84)‏gcd(413,84)‏ gcd(84,77)‏gcd(84,77)‏ gcd(77,7)gcd(77,7) gcd(7,0) = 7gcd(7,0) =

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Two descriptions of Euclid’s algorithm -Euclid of Alexandria – 3 rd Century B.C. Step 1 If n = 0, return m and stop; otherwise go to Step 2 Step 2 Divide m by n and assign the value fo the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1. while n ≠ 0 do r ← m mod n m← n m← n n ← r n ← r return m

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Other methods for computing gcd(m,n)‏ Consecutive integer checking algorithm Step 1 Assign the value of min{m,n} to t Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4 Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4 Step 4 Decrease t by 1 and go to Step 2 Note: This does not work when one of the input numbers (m or n) is zero.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Other methods for gcd(m,n) [cont.] Middle-school procedure (Paraphrased from your middle school teacher, 20 th century A.D.)‏ Step 1 Find the prime factorization of m Step 2 Find the prime factorization of n Step 3 Find all the common prime factors Step 4 Compute the product of all the common prime factors and return it as gcd(m,n)‏ Is this an algorithm?

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Was that an algorithm?  Can you use the algorithm to find gcd( , )?  How did you find the lists of prime factors?  How did you sort these lists to find the factors in common?  The procedure before is not an algorithm because it is definitely not unambiguous. Aside: finding the list of prime factors for large numbers turns out to be hardAside: finding the list of prime factors for large numbers turns out to be hard 1-20

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Sieve of Eratosthenes (ca. 200BC)‏ Input: Integer n ≥ 2 Output: List of primes less than or equal to n for p ← 2 to n do A[p] ← p for p ← 2 to  n  do if A[p]  0 //p hasn’t been previously eliminated from the list j ← p * p if A[p]  0 //p hasn’t been previously eliminated from the list j ← p * p while j ≤ n do while j ≤ n do A[j] ← 0 //mark element as eliminated A[j] ← 0 //mark element as eliminated j ← j + p j ← j + p Example:

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Two main issues related to algorithms  How to design algorithms  How to analyze algorithm efficiency

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Algorithm design techniques/strategies  Brute force  Divide and conquer  Decrease and conquer  Transform and conquer  Space and time tradeoffs  Greedy approach  Dynamic programming  Iterative improvement  Backtracking  Branch and bound

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Analysis of algorithms  How good is the algorithm? time efficiencytime efficiency space efficiencyspace efficiency  Does there exist a better algorithm? lower boundslower bounds optimalityoptimality

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Important problem types  sorting  searching  string processing  graph problems  combinatorial problems  geometric problems  numerical problems

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Fundamental data structures  list arrayarray linked listlinked list stringstring  stack  queue  priority queue  graph  tree  set and dictionary