1. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

2. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 1-2 Two main issues related to algorithms  How to design algorithms  How to analyze algorithm efficiency

3. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 1-3 1.i. How to Design: Basic Patterns  Brute force  Divide and conquer  Decrease and conquer  Transform and conquer  Space and time tradeoffs  Greedy approach  Dynamic programming  Iterative improvement  Backtracking  Branch and bound

4. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 1-4 1. ii. How to Design: Basic Problem Types  sorting  searching  string processing  graph problems  combinatorial problems  geometric problems  numerical problems

5. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 1-5 1. iii. How to Design: Basic Tools  list arrayarray linked listlinked list stringstring  stack  queue  priority queue  graph  tree  set and dictionary

6. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 1-6 2. i. Analysis of algorithms: Questions  How good is the algorithm? time efficiencytime efficiency space efficiencyspace efficiency  Does there exist a better algorithm? lower boundslower bounds optimalityoptimality

7. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 2. ii. Analysis of algorithms  Issues: correctnesscorrectness time efficiencytime efficiency space efficiencyspace efficiency optimalityoptimality  Approaches: empirical analysisempirical analysis theoretical analysistheoretical analysis

8. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Empirical analysis of time efficiency  Select a specific (typical) sample of inputs  Use physical unit of time (e.g., milliseconds)‏ or or Count actual number of basic operation’s executions Count actual number of basic operation’s executions  Analyze the empirical data

9. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size  Basic operation: the operation that contributes most towards the running time of the algorithm T(n) ≈ c op C(n)‏ T(n) ≈ c op C(n)‏ running time execution time for basic operation Number of times basic operation is executed input size

10. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Input size and basic operation examples Problem Input size measure Basic operation Searching for key in a list of n items ?? Multiplication of two matrices ?? Checking primality of a given integer n ?? Typical graph problem ??

11. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1Problem Input size measure Basic operation Searching for key in a list of n items Number of list’s items, i.e. n Key comparison Multiplication of two matrices Matrix dimensions or total number of elements Multiplication of two numbers Checking primality of a given integer n n’size = number of digits (in binary representation)‏ Division Typical graph problem #vertices and/or edges Visiting a vertex or traversing an edge

12. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Best-case, average-case, worst-case For some algorithms efficiency depends on form of input:  Worst case: C worst (n) – maximum over inputs of size n  Best case: C best (n) – minimum over inputs of size n  Average case: C avg (n) – “average” over inputs of size n Number of times the basic operation will be executed on typical inputNumber of times the basic operation will be executed on typical input NOT the average of worst and best caseNOT the average of worst and best case Expected number of basic operations considered as a random variable under some assumption about the probability distribution of all possible inputsExpected number of basic operations considered as a random variable under some assumption about the probability distribution of all possible inputs

13. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Example: Sequential search  Worst case  Best case  Average case

14. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Types of formulas for basic operation’s count  Exact formula e.g., C(n) = n(n-1)/2 e.g., C(n) = n(n-1)/2  Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n 2 e.g., C(n) ≈ 0.5 n 2  Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn 2 e.g., C(n) ≈ cn 2

15. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Order of growth  Most important: Order of growth within a constant multiple as n→∞  Example: How much faster will algorithm run on computer that is twice as fast?How much faster will algorithm run on computer that is twice as fast? How much longer does it take to solve problem of double input size?How much longer does it take to solve problem of double input size?

16. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Values of some important functions as n  

17. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Growth Matters  Magic Computer – One Trillion Operations per Second It would take approx 4*10 10 years to execute 2 100 operationsIt would take approx 4*10 10 years to execute 2 100 operations This is more than 4.5 billion yearsThis is more than 4.5 billion years  Imagine how long it would take to execute 100! Operations  These are both fine examples of a somewhat small (100) factor with 2 n and n! complexity 1-17