Material in the textbook Sections 1.1.5 to 1.2.1 Lecture 3 Material in the textbook Sections 1.1.5 to 1.2.1 מבוא מורחב
Review Procedures capture common patterns Naming abstraction allows us to use them as primitive objects עוץ-לי-גוץ-לי Principle: “If you capture a process (in a procedure), and give it a name, you have power over that process” The Substitution Model In order to control complex systems, we need a model, we have the substitution model. The models we are using are approximations, we can use them but not always. This is similar to Newtonian Physics, which is great for our everyday world, but not for sub-atomic particles. In a similar fashion, we now work with the substitution model. מבוא מורחב
Evaluation of An Expression To Evaluate a combination: (other than special form) Evaluate all of the sub-expressions in any order Apply the procedure that is the value of the leftmost sub-expression to the arguments (the values of the other sub-expressions) The value of a numeral: number The value of a built-in operator: machine instructions to execute The value of any name: the associated object in the environment To Apply a compound procedure: (to a list of arguments) Evaluate the body of the procedure with the formal parameters replaced by the corresponding actual values. מבוא מורחב
Review: Block structure Lets write a procedure that given x, y, and z computes f(x,y,z) = (x+y)2 + (x+z)2 (define (sum-and-square x y) (square (+ x y))) (define (f x y z) (+ (sum-and-square x y) (sum-and-square x z))) מבוא מורחב
Block structure (cont.) We could also make sum-and-square private to f: (define (f x y z) (define (sum-and-square x y) (square (+ x y))) (+ (sum-and-square x y) (sum-and-square x z))) Syntactically: (define (<name> <params>) <list of defs> <body>) מבוא מורחב
Need to clarify the substitution model.. (define (f x y z) (define (sum-and-square x y) (square (+ x y))) (+ (sum-and-square x y) (sum-and-square x z))) ==> (f 1 2 3) (define (sum-and-square 1 2) (square (+ 1 2))) (+ (sum-and-square 1 2) (sum-and-square 1 3))) מבוא מורחב
Bounded variables and scope A procedure definition binds its formal parameters The scope of the formal parameter is the body of the procedure. (define (f x y z) (define (sum-and-square x y) (square (+ x y))) (+ (sum-and-square x y) (sum-and-square x z))) x,y,z x,y מבוא מורחב
Evaluation of An Expression (refined) To Evaluate a combination: (other than special form) Evaluate all of the sub-expressions in any order Apply the procedure that is the value of the leftmost sub-expression to the arguments (the values of the other sub-expressions) The value of a numeral: number The value of a built-in operator: machine instructions to execute The value of any name: the associated object in the environment To Apply a compound procedure: (to a list of arguments) Evaluate the body of the procedure with the formal parameters replaced by the corresponding actual values. Do not substitute for occurrences in an internal definition. מבוא מורחב
The refined substitution model (define (f x y z) (define (sum-and-square x y) (square (+ x y))) (+ (sum-and-square x y) (sum-and-square x z))) ==> (f 1 2 3) (define (sum-and-square x y) (square (+ x y))) (+ (sum-and-square 1 2) (sum-and-square 1 3))) מבוא מורחב
SQRT To find an approximation of square root of x: Make a guess G Improve the guess by averaging G and x/G Keep improving the guess until it is good enough G = 1 X = 2 X/G = 2 G = ½ (1+ 2) = 1.5 X/G = 4/3 G = ½ (3/2 + 4/3) = 17/12 = 1.416666 X/G = 24/17 G = ½ (17/12 + 24/17) = 577/408 = 1.4142156 מבוא מורחב
Procedure SQRT (define initial-guess 1.0) (define precision 0.0001) (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define (sqrt x) (sqrt-iter initial-guess x)) מבוא מורחב
Procedural decomposition of SQRT sqrt-iter good-enough? improve abs square average מבוא מורחב
Procedural Abstraction It is better to: Export only what is needed Hide internal details. The procedure sqrt is of interest for the user. The procedure improve-guess is an internal detail. Exporting only what is needed leads to: A clear interface Avoids confusion מבוא מורחב
Rewriting SQRT (Block structure) (define (sqrt x) (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define initial-guess 1.0) (define precision 0.00001) The same x (sqrt-iter initial-guess x) ) מבוא מורחב
SQRT (Cleaner) (define (sqrt x) (define (good-enough? guess) Drop X as a parameter since it has the same value as the global X (define (sqrt x) (define (good-enough? guess) (< (abs (- (square guess) x)) precision)) (define (improve guess) (average guess (/ x guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess) ) The order of the definitions does not matter מבוא מורחב
SQRT ( Cont.) ==>(sqrt 2) (define (good-enough? guess) (< (abs (- (square guess) 2)) precision)) (define (improve guess) (average guess (/ 2 guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess))
Today Refine our model Applicative vs. Normal Order Understand how it captures the nature of processes Recursive vs. Iterative processes Orders of growth מבוא מורחב
Applicative order vs. Normal Order Evaluation To Evaluate a combination: (other than special form) Evaluate all of the sub-expressions in any order Apply the procedure that is the value of the leftmost sub-expression to the arguments (the values of the other sub-expressions) מבוא מורחב
Applicative order evaluation rules Combination... (<operator> <operand1> …… <operand n>) Evaluate <operator> to get the procedure and evaluate <operands> to get the arguments If <operator> is primitive: do whatever magic it does If <operator> is compound: evaluate body with formal parameters replaced by arguments מבוא מורחב
Example: Factorial n! = n * (n-1)! wishful thinking : base case: n = 1 (define fact (lambda (n) (if (= n 1) 1 (* n (fact (- n 1)))))) מבוא מורחב
Example: Factorial ==>(fact 3) ([[n](if(= n 1) 1 (* n (fact (- n 1))))] 3) (if (= 3 1) 1 (* 3 (fact (- 3 1)))) (if #f 1 (* 3 (fact (- 3 1)))) (* 3 (fact (- 3 1))) (* 3 ([[n](if(= n 1) 1 (* n (fact (- n 1))))] (- 3 1))) (* 3 (if (= 2 1) 1 (* 2 (fact (- 2 1))))) (* 3 (if #f 1 (* 2 (fact (- 2 1))))) (* 3 (* 2 (fact (-2 1)))) (* 3 (* 2 (([[n](if(= n 1) 1 (* n (fact (- n 1))))] (- 2 1)))) (* 3 (* 2 (if (= 1 1) 1 (* (fact (- 1 1))))))) (* 3 (* 2 (if #t 1 (* 1 (fact (- 1 1)))))) (* 3 (* 2 1)) (* 3 2) 6 מבוא מורחב
Normal order evaluation Combination … (<operator> <operand1> …… <operand n>) Evaluate <operator> to get the procedure and evaluate <operands> to get the arguments If <operator> is primitive: do whatever magic it does If <operator> is compound: evaluate body with formal parameters replaced by arguments מבוא מורחב
The Difference This may matter in some cases: Normal ((lambda (x) (+ x x)) (* 3 4)) (+ (* 3 4) (* 3 4)) (+ 12 12) 24 Applicative ((lambda (x) (+ x x)) (* 3 4)) (+ 12 12) 24 This may matter in some cases: ((lambda (x y) (+ x 2)) 3 (/ 1 0)) Scheme is an Applicative Order Language! מבוא מורחב
Compute ab (Recursive Approach) wishful thinking : base case: ab = a * a(b-1) a0 = 1 (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1)))))) מבוא מורחב
Compute ab (Iterative Approach) Another approach: ab = a * a * a*…*a b ab = a2 *a*…*a= a3 *…*a Which is: Operationally: Halting condition: product product * a counter counter - 1 counter = 0 מבוא מורחב
Compute ab (Iterative Approach) (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) Syntactic Recursion Notice the use of design rules, the modular structure of the procedures, and the fact that there is syntactic recursion in the code, just like the “recursive” solution. So obviously when we mean recursive, we are not talking about the syntax of the procedures. Lets How then, do the two procedures differ? They give rise to different processes – lets use our model to understand how. מבוא מורחב
Recursive Process (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1)))))) (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) For simplicity we skip application steps and remember how the if statement works. (* 3 (* 3 9)) (* 3 27) 81 מבוא מורחב
Iterative Process (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) For simplicity we skip application steps and remember how the if statement works. (exp-iter 3 1 27) (exp-iter 3 0 81) 81 מבוא מורחב
The Difference Growing amount of space Constant amount of space (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Growing amount of space (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 Constant amount of space Space copmpelxity is diffrent מבוא מורחב
Why More Space? operation pending no pending operations Recursive exponentiation: (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1))))) operation pending Iterative exponentiation: (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product)))) (exp-iter a b 1)) no pending operations
Summary Recursive process num of deferred operations “grows proportional to b” Iterative process num of deferred operations stays “constant” (actually it’s zero) Can we better quantify these observations? Orders of growth… מבוא מורחב
Order of Growth: Recursive Process (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 4 (exp-1 3 5) (* 3 (exp-1 3 4)) (* 3 (* 3 (exp-1 3 3))) (* 3 (* 3 (* 3 (exp-1 3 2)))) Dependent on b (* 3 (* 3 (* 3 (* 3 (exp-1 3 1))))) (* 3 (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 (* 3 3)))) (* 3 (* 3 (* 3 9))) (* 3 (* 3 27)) (* 3 81) 5 243
Iterative Process Some constant, independent of b (define (exp-2 a b) (define (exp-iter a b product) (if (= b 0) product (exp-iter a (- b 1) (* a product)))) (exp-iter a b 1) (exp-2 3 4) (exp-2 3 5) (exp-iter 3 4 1) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 2 27) (exp-iter 3 0 81) (exp-iter 3 0 81) 81 (exp-iter 3 0 243) 243 Some constant, independent of b
Orders of Growth Suppose n is a parameter that measures the size of a problem (the size of its input) R(n)measures the amount of resources needed to compute a solution procedure of size n. Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps. The worst-case over all inputs of size n מבוא מורחב
Orders of Growth R1(n)=100n2 R2(n)=2n2+10n+2 R3(n) = n2 Want to estimate the “order of growth” of R(n): R1(n)=100n2 R2(n)=2n2+10n+2 R3(n) = n2 Are all the same in the sense that if we multiply the input by a factor of 2, the resource consumption increases by a factor of 4 Order of growth is proportional to n2 מבוא מורחב
c1f(n)<= R(n)<= c2f(n) Orders of Growth We say R(n)has order of growth Q(f(n)) if there are constants c1 0 and c2 0 such that for all n c1f(n)<= R(n)<= c2f(n) R(n)O(f(n)) if there is a constant c 0 such that for all n R(n) <= c f(n) R(n)(f(n)) if there is a constant c 0 such that for all n c f(n) <= R(n) מבוא מורחב
Orders of Growth True or False? 100n2 O(n) f 100n2 (n) t 100n2 Q(n) f 100n2 Q(n2) t True or False? 2100 (n) f 2100n O(n2) t 2n Q(n) f 210 Q(1) t מבוא מורחב
Resources Consumed by EXP-1 (exp-1 3 4) “n”=b=4 (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Space b <= R(b) <= b which is Q(b) Time b <= R(b) <= 2b which is Q(b) Linear Recursive Process מבוא מורחב
Resources Consumed by EXP-2 (exp-2 3 4) “n”=b=4 (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 Space Q(1) Time Q(b) Linear Iterative Process מבוא מורחב
Summary Trying to capture the nature of processes Quantify various properties of processes: Number of steps a process takes (Time Complexity) Amount of Space a process uses (Space Complexity) מבוא מורחב