1. Analyzing Programs: Order of Growth CMSC 11500 Introduction to Computer Programming October 11, 2002

2. Roadmap Motivation: Max Order of Growth –Analyzing resource requirements –Example 1: Tree Recursion: Fibonacci Recursive and Iterative solutions –Example 2: Exponentiation Recursive Iterative Fast

3. Orders of Growth How much resource does computation use? –Time: how many steps? –Space: how much memory? How do resource needs change with input size? –If input doubles in size, do requirements Stay the same? - constant - e.g. fact-iter space Double? - linear - e.g. fact-iter time Quadruple? - quadratic (n^2) - naïve sort Go up exponentially? - Fibonacci tree

4. Defining Order of Growth n: Measure of size of input –e.g count, elts in list, nodes/edges in graph, R(n):amount of resource used solve problem of size n

5. Order of Growth “Constant”: Theta(1): Fact-iter space “Linear”: Theta(n): Factorial time “Exponential”: Theta(phi^n): Fibonacci tree “Logarithmic”: Theta(log n): Fast exponent

6. Tree Recursive Processes Fibonacci: –Originally developed to model rabbit reproduction rates –Fib(n)= 0 if n=0; Fib(n) = 1 if n=1 –O.w. Fib(n) = Fib(n-1)+Fib(n-2) –0,1,1,2,3,5,8,13,21,34,55,89,... (define (fib n) (cond ((= n 0) 0) ((= n 1) 1) (else (+ (fib (- n 1)) (fib (- n 2))))))

7. Recursive Pattern Fib 5 Fib 3Fib 4 Fib 1Fib2 1Fib 1Fib 0 10 Fib 2Fib 3 Fib 0Fib1 Fib 2 011Fib 0Fib1 01

8. Fibonacci Analysis Note: Lots of computation –Increases exponentially in size of n Number of leaves is Fib(n+1) –Integer closest to where Note: Lots of wasted computation –Entire computation of Fib(3) is repeated Iterative implementation –Successive additions –Increases only linearly in n

9. Iterative Fibonacci (define (fib n) (fib-iter 1 0 n)) (define (fib-iter a b count) (if (= count 0) b (fib-iter (+ a b) a (- count 1)))))

10. Example:Exponents Linear recursive (define (power number exponent) (cond ((= exponent 0) 1) ((= exponent 1) number) (else (* number (power number (- exponent 1))))))

11. Example: Exponents: Iterative (define (power number exponent) (power-iter number 1 exponent)) (define (power-iter value product counter) (if (= counter 0) product (power-iter value (* value product) (- counter 1))))

12. Fast Exponentiation Key insight: To compute b^8, –b^2=b*b; b^4=b^2*b^2; b^8=b^4*b^4 Only 3 multiplications to compute b^8 –Logarithmic growth: log 8 = 3 –Detail: Odd exponent: b^7=b*b^6 (define (fast-exp b n) (cond ((= n 0) 1) ((even? n) (square (fast-exp b (/ n 2)))) (else (* b (fast-exp b (- n 1))))))

13. Summary Recursive and Iterative Processes –Tree recursion Computation and Resource Consumption –Time & Space –Increase in resource needs with size of input Characteristic of implementation, not of problem –Fibonacci: exponential; Fib-iter: linear

14. Next Time Procedures as first class objects –Procedures as input and output of procedures Scope with let and lambda