1 Chapter 1 Analysis Basics

2 Chapter Outline What is analysis? What to count and consider Mathematical background Rates of growth Tournament method Analyzing programs

3 Prerequisites Before beginning this chapter, you should be able to: –Read and create algorithms –Identify comparison and arithmetic operations –Use basic algebra

4 Goals At the end of this chapter you should be able to: –Describe how to analyze an algorithm –Explain how to choose the operations that are counted and why others are not –Explain how to do a best-case, worst-case, and average case analysis

5 Goals (continued) –Work with logarithms, probabilities, and summations –Describe  (f), Ω(f), O(f), growth rate, and algorithm order –Use a decision tree to determine a lower bound on complexity

6 Analyzing an Algorithm Show that it properly solves the problem Determine how efficient it solves the problem –Approximate the number of operation

7 Algorithm Classification Iterative –Based on loop and conditional statements –Determine the work in the loop and the number of times the loop executes Recursive –Based on calls to the same subprogram with a smaller problem –Determine the number of smaller problems and the amount of work done to produce them and then put their solutions together

8 What is Analysis? An approximation of how long an algorithm will take or how much extra space it needs Comparison of two or more algorithms that solve the same problem Independent of the computer because a faster computer does not make an algorithm more efficient Independent of initializations because those may be done faster

9 Input Classes Groups of input sets that cause the same amount of work to be done For example, finding the largest value –Input classes could be based on the location of the largest element

10 Space Complexity The amount of space needed for an algorithm to complete its task Algorithms are classified as either in place or needing extra space –Depends on whether the algorithm moves data around within its current storage or copies information to a new space while it’s working Use of extra space can be critical in software designed for small embedded systems

11 What to Count Comparisons –Equal, greater, not equal, … Arithmetic –Additions add, subtract, increment, decrement –Multiplications multiply, divide, modulus, remainder

12 Cases to Consider Best Case –The least amount of work done for any input set Worst Case –The most amount of work done for any input set Average Case –The amount of work done averaged over all of the possible input sets

13 Average Case Determining the average case: –Find the number of input set classes –Find the probability that the input will be from each of these classes –Find the amount of work done for each class The average case is given by:

14 Mathematical Background The floor of X is the largest integer  to X The ceiling of X is the smallest integer  to X The logarithm (base Y) of X is the value Z that satisfies the equation X = Y Z

15 Binary Trees A binary tree with N nodes has at least  lg N  +1 levels There are 2 K-1 nodes on level K In a complete binary tree all of the nodes on levels 1 to J-1 have two children, so all of the leaves are on level J A complete binary tree with J levels has 2 J -1 nodes

16 Probabilities A probability is a number between 0 and 1 If something will never occur, it has a probability of 0 If something always occurs, it has a probability of 1 If there are N equally likely events, each one has a probability of 1/N

17 Summations A summation is a compact way to express the sum of a series of values The sum of the numbers from 1 to 7 would be written: There are closed forms for many summations given in the text

18 Rate of Growth The rate of growth of a function determines how fast the function value increases when the input increases

19 Rate of Growth If algorithm A does x 3 operations on an input of size x and algorithm B does x 2 operations, algorithm B is more efficient Because of the relative growth rate, we will consider x 3 + x 2 + x equivalent to x 3

20 Classification of Growth Big Omega Ω(f) –The class of functions that grow at least as fast as the function f, and maybe faster Big Oh O(f) –The class of functions that grow no faster than f, and maybe slower Big Theta Θ (f) –The class of functions that grow at the same rate as the function f

21 Tournament Method Finding the second largest element –Build a binary tree with the values in the leaves –The larger of a pair is copied into the parent node –The root has the largest value –The second largest value must have lost to the largest

22 Decision Trees Each choice is represented by a different child Path from the root to a leaf indicates how that solution is found Different algorithms that solve a problem will produce different decision trees

23 Decision Tree Example

24 Lower Bounds The absolute smallest number of operations needed to solve a problem Based on the problem not a particular algorithm Any algorithm that claims to solve the problem faster must fail

25 Lower Bounds Example For example, sorting a list –Each leaf holds one of the N! possible initial orderings of elements –Each internal node indicates an element comparison –If the tree is packed as tightly as possible, the number of levels indicate the lower bound –Thus, the lower bound for sorting is N lg N

26 Analyzing Programs Program profilers are available that will indicate where the program is spending its execution time Can be done by hand by: –Incrementing a unique counter every time each subprogram is called –Incrementing a unique counter every time a particular code segment is reached