מבוא מורחב - שיעור 4 1 Lecture 4 Order of Growth Fun with recursion  Fast exponentiation  Hanoi towers

מבוא מורחב - שיעור 4 2 מבוא מורחב 2 Recursive Process (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1)))))) (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81

מבוא מורחב - שיעור 4 3 מבוא מורחב 3 Iterative Process (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter ) (exp-iter ) 81 (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1))

מבוא מורחב - שיעור 4 4 מבוא מורחב 4 The Difference (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter ) (exp-iter ) 81 (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Growing amount of space Constant amount of space

מבוא מורחב - שיעור 4 55 Why More Space? Recursive exponentiation: (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1))))) operation pending Iterative exponentiation: (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product)))) (exp-iter a b 1)) no pending operations

מבוא מורחב - שיעור 4 6 מבוא מורחב 6 Summary Recursive process num of deferred operations “grows proportional to b” Iterative process num of deferred operations stays “constant” (actually it’s zero) Can we better quantify these observations? Orders of growth…

מבוא מורחב - שיעור 4 77 Order of Growth: Recursive Process (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81  4 (exp-1 3 5) (* 3 (exp-1 3 4)) (* 3 (* 3 (exp-1 3 3))) (* 3 (* 3 (* 3 (exp-1 3 2)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 1))))) (* 3 (* 3 (* 3 (* 3 3)))) (* 3 (* 3 (* 3 9))) (* 3 (* 3 27)) (* 3 81) 243  5 (* 3 (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 (* 3 1)))) Dependent on b

מבוא מורחב - שיעור 4 88 Iterative Process (define (exp-2 a b) (define (exp-iter a b product) (if (= b 0) product (exp-iter a (- b 1) (* a product)))) (exp-iter a b 1) (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter ) (exp-iter ) 81 (exp-2 3 5) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter ) 243 (exp-iter ) (exp-iter ) Some constant, independent of b

מבוא מורחב - שיעור 4 9 מבוא מורחב 9 Orders of Growth Suppose n is a parameter that measures the size of a problem (the size of its input) R(n) measures the amount of resources needed to compute a solution procedure of size n. Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps. The worst-case over all inputs of size n

מבוא מורחב - שיעור 4 10 מבוא מורחב 10 Orders of Growth Want to estimate the “order of growth” of R(n): R 1 (n)=100n 2 R 2 (n)=2n 2 +10n+2 R 3 (n) = n 2 Are all the same in the sense that if we multiply the input by a factor of 2, the resource consumption increases by a factor of 4 Order of growth is proportional to n 2

מבוא מורחב - שיעור 4 11 Order of Growth

מבוא מורחב - שיעור 4 12 מבוא מורחב 12 Orders of Growth We say R(n) has order of growth  (f(n))  if there are constants c 1  0 and c 2  0 such that for all n c 1 f(n) <= R(n) <= c 2 f(n) R(n)  (f(n)) if there is a constant c  0 such that for all n c f(n) <= R(n) R(n)  O(f(n)) if there is a constant c  0 such that for all n R(n) <= c f(n)

מבוא מורחב - שיעור 4 13 f(n) = O(g(n))

מבוא מורחב - שיעור 4 14 f(n) = Θ (g(n))

מבוא מורחב - שיעור 4 15 The intuition of this notation … R(n)  O(f(n)) R(n) does not grow faster than f(n) R(n)   (f(n)) R(n) does not grow slower than f(n) R(n)   (f(n)) R(n) and f(n) grow at the same rate R(n)  O(f(n))  f(n)   (R(n)) R(n)   (f(n))  f(n)   (R(n))

מבוא מורחב - שיעור 4 16 מבוא מורחב 16 Resources Consumed by EXP-1 (exp-1 3 4) “n”=b=4 (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Space b <= R( b ) <= b which is  b  Time b <= R( b ) <= 2 b which is  b  Linear Recursive Process

מבוא מורחב - שיעור 4 17 מבוא מורחב 17 Resources Consumed by EXP-2 (exp-2 3 4) “n”=b=4 (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter ) (exp-iter ) 81 Space  1  Time  b  Linear Iterative Process

מבוא מורחב - שיעור 4 18 מבוא מורחב 18 Orders of Growth t t f f 100n 2  O(n) 100n 2   (n) 100n 2   (n) 100n 2   (n 2 ) True or False? t t f f   (n) n  O(n 2 ) 2 n   (n) 2 10   (1) True or False?

מבוא מורחב - שיעור 4 19 The conditional form (cond ( ) ( ) …. ( ) (else )) (define (abs x) (cond ((> x 0) x) ((= x 0) 0) ((< x 0) (- x)))) (else (- x))))

מבוא מורחב - שיעור 4 20 Another algorithm for computing a b If b is even, then a b = (a 2 ) (b/2) If b is odd, then a b = a*a (b-1) Problem size reduced to half in at most two steps. (define (exp-fast1 a b) ; computes a b (cond ((= b 0) 1) ((even? b) (exp-fast1 (* a a) (/ b 2))) (else (* a (exp-fast1 a (- b 1)))))))

מבוא מורחב - שיעור 4 21 (exp-fast 3 56) ; compute 3^56 (exp-fast1 3 56) ; b= (exp-fast1 9 28) ; b= (exp-fast ) ; b= (exp-fast ) ; b= * (exp-fast ) ; b= * (exp-fast ) ; b= * * (exp-fast ) ; b= * * (exp-fast ) ; b= * * * (exp-fast1.. 0) ; b= * * Note: scheme allows the use of very large numbers (define (exp-fast1 a b) (cond ((= b 0) 1) ((even? b) (exp-fast1 (* a a) (/ b 2))) (else (* a (exp-fast1 a (- b 1)))))))

מבוא מורחב - שיעור 4 22 How much time does exp-fast take? If b is even: T(b) = T(b/2)+c and if b is odd then: T(b) = T(b-1)+c = T((b-1)/2)+2c OK, but what is it? In theta notation, that is!

מבוא מורחב - שיעור 4 23 Let ’ s start with a simpler case Say that T(b) = T(b/2)+c always For what values of b does this happen? T(b) = T(b/2)+c = T(b/4)+c+c = T(b/8)+c+c+c = …. = T(b/(2^k))+ k*c When does b/(2^k) = 1? K = log(b) => T(b) = T(1)+ log(b) =  (1)+log(b) =  log(b)) Note: log base1 x = k log base2 x for some constant k > 0

מבוא מורחב - שיעור 4 24 Counting “ odd ” steps We also make “odd” steps (where b becomes odd) But how much? Claim: At most one “odd” step per each “even” step So we make no more than twice the number of steps! Still  log(b)) Space complexity is  log(b)) as well

מבוא מורחב - שיעור 4 25 Fast exponentiation (Iterative Approach) Even? b: base  base 2 exp  exp/2 Odd? b:product  product*a exp  exp-1 product  1, base = a, exp = b Init: Loop:

מבוא מורחב - שיעור 4 26 Scheme code (Iterative Approach) (define (exp-fast2 a b) (define (exp-fast-iter a b product) (cond ((= b 0) product) ((even? b) (exp-fast-iter (* a a) (/ b 2) product)) (else (exp-fast-iter a (- b 1) (* a product))))) (exp-fast-iter a b 1))

מבוא מורחב - שיעור 4 27 Comparing the three exponentiation procedures exp-fast1and exp-fast2 are exponentially faster than exp-1 and exp-2 TimeSpace exp-1 (recursive)  b  exp-2 (iterative)  b  1  exp-fast1 (recursive)  log  b  log  b  exp-fast2 (iterative)  log  b  1 

מבוא מורחב - שיעור 4 28 Towers of Hanoi Three posts, and a set of disks of different sizes. A disk can be placed only on a larger disk (or on bottom). At the beginning all the disks are on the left post. The goal is to move the disks one at a time, while preserving these conditions, until the entire stack has moved from the left post to another You are allowed to move only the topmost disk at a step

מבוא מורחב - שיעור 4 29 Use our paradigm Wishful thinking: Smaller problem: A problem with one disk less How do we use it ? To move n disks from post A to post C (using B as aux): Move top n-1 disks from post A to post B (using C as aux) Move the largest disk from post A to post C Move n-1 disks from post B to post C (using A as aux) We solve 2 smaller problems !

מבוא מורחב - שיעור 4 30 Towers of Hanoi (define (one-move from to) (display "Move top disk from ") (display from) (display " To ") (display to) (newline)) (define (move-tower size from to aux) (cond ((= size 1) (one-move from to)) (else (move-tower (- size 1) from aux to) (one-move from to) (move-tower (- size 1) aux to from))))

מבוא מורחב - שיעור 4 31 Tree Recursion (mt ) (mt ) (mt ) (one-move 1 2) (mt ) (one-move 3 2) (mt ) (one-move 1 3) (mt ) (one-move 2 3)(one-move 3 1)(one-move 1 2)

מבוא מורחב - שיעור 4 32 Towers of Hanoi -- trace (move-tower ) Move top disk from 1 to 2 Move top disk from 1 to 3 Move top disk from 2 to 3 Move top disk from 1 to 2 Move top disk from 3 to 1 Move top disk from 3 to 2 Move top disk from 1 to a3a (move-tower ) (move-tower )

מבוא מורחב - שיעור 4 33 Orders of growth for towers of Hanoi Denote by T(n) the number of steps that we need to take to solve the case for n disks. T(n) = 2T(n-1) + 1 T(1) = 1 This solves to: T(n) = 2 n -1=  (2 n ) What does that mean ?

מבוא מורחב - שיעור 4 34 Hanoi Towers Say we want to solve the problem for 400 disks. Say it takes a second to move a disk. We need about seconds. That’s about years. That’s about millenniums. Might be longer then the age of the universe …. Infeasible !!!!

מבוא מורחב - שיעור 4 35 Let ’ s buy a fast computer and make it feasible. Our new computer can move giga billion (2 60 ) disks a second. Absolutely the last word in the field of computing. We need about seconds. That’s about years. That’s about millenniums. Does not help much. Infeasible !!!! An algorithm with exponential time complexity is not scalable

מבוא מורחב - שיעור 4 36 What about Space complexity?

מבוא מורחב - שיעור 4 37 Towers of Hanoi (define (one-move from to) (display "Move top disk from ") (display from) (display " To ") (display to) (newline)) (define (move-tower size from to aux) (cond ((= size 1) (one-move from to)) (else (move-tower (- size 1) from aux to) (one-move from to) (move-tower (- size 1) aux to from)))) Pending operations

מבוא מורחב - שיעור 4 38 Towers of Hanoi Space complexity The number of pending operations is the height of the recursion tree. So the space complexity is S(n) =  (n) Note that the second recursive call is treated as tail recursion, and forms an iteration (no pending operation for these calls).

מבוא מורחב - שיעור 4 39 Tree Recursion (mt ) (mt ) (mt ) (one-move 2 1) (mt ) (one-move 3 1) (mt ) (one-move 2 3) (mt ) (one-move 1 3)(one-move 3 2)(one-move 2 1)