The Metacircular Evaluator (Continued) Chapter 4 Section 4.1 Reminder: We will not cover every little detail in the lectures, so you MUST read Section 4.1 in full. http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-26.html

Last Lecture: The Metacircular Evaluator Inputs Scheme Program ME CPU It might seem circular to think about evaluating Scheme programs in Scheme, but the idea is since we have throughout the course used Scheme as our way of describing processes, and the evaluator is a process, then its only natural that we describe it too using Scheme. Outputs The Metacircular Evaluator is an Interpreter for Scheme on a machine whose machine language is Scheme.

The Eval/Apply Cycle

The Meta-Circular Evaluator: Eval (define (eval exp env) (cond ((self-evaluating? exp) exp) ((variable? exp) (lookup-variable-value exp env)) ((quoted? exp) (text-of-quotation exp)) ((assignment? exp) (eval-assignment exp env)) ((definition? exp) (eval-definition exp env)) ((if? exp) (eval-if exp env)) ((lambda? exp) (make-procedure (lambda-parameters exp) (lambda-body exp) env)) ((begin? exp) (eval-sequence (begin-actions exp) env)) ((cond? exp) (eval (cond->if exp) env)) ((application? exp) (apply (eval (operator exp) env) (list-of-values (operands exp) env))) (else (error "Unknown expression type -- EVAL" exp)))) Special forms This is a top down, modular piece of software, built correctly with many layers of abstraction. However, since you havn’t built an interpreter before, we will not start directly a top down review of the process but rather first give a simplified example. Note that the use of a high level description in the main eval loop allows easy modification For possible changes to the language, but also makes it harder to add new constructs since they must Be added by changing this main eveal loop.

The Meta-Circular Evaluator: Apply (define (apply procedure arguments) (cond ((primitive-procedure? procedure) (apply-primitive-procedure procedure arguments)) ((compound-procedure? procedure) (eval-sequence (procedure-body procedure) (extend-environment (procedure-parameters procedure) arguments (procedure-environment procedure)))) (else (error "Unknown procedure type" procedure))))

Read-Eval-Print Loop (define input-prompt ";;; M-Eval input:") (define output-prompt ";;; M-Eval value:") (define (prompt-for-input string) (newline) (newline) (display string) (newline)) (define (announce-output string) (newline) (display string) (newline)) (define (driver-loop) (prompt-for-input input-prompt) (let ((input (read))) (let ((output (eval input the-global-env))) (announce-output output-prompt) (user-print output))) (driver-loop))

Sample Execution (define the-global-environment (setup-environment)) (driver-loop) ;;; M-Eval input: (define (append x y) (if (null? x) y (cons (car x) (append (cdr x) y)))) ;;; M-Eval value: ok (append '(a b c) '(d e f)) (a b c d e f)

The Environment Representation Last lecture we assumed an environment consisting of a single frame frame x y 4 5 list of variables list of values

(define (greater? exp) (tag-check exp ’>))   (define (greater? exp) (tag-check exp ’>)) (define (if? exp) (tag-check exp 'if)) (define (eval exp) (cond ((number? exp) exp) ((symbol? exp) (lookup exp)) ((sum? exp) (eval-sum exp)) ((greater? exp) (eval-greater exp)) ((definition? exp) (eval-definition exp)) ((if? exp) (eval-if exp)) (else (error "unknown expression " exp)))) (define (eval-greater exp) (> (eval (cadr exp)) (eval (caddr exp))))

(if (true? (eval (if-predicate exp))) (eval (if-consequent exp)) (define (eval-if exp) (if (true? (eval (if-predicate exp))) (eval (if-consequent exp)) (eval (if-alternative exp)))) (define (if-predicate exp) (cadr exp)) (define (if-consequent exp) (caddr exp)) (define (if-alternative exp) (if (not (null? (cdddr exp))) (cadddr exp) 'false)) (define (true? x) (not (eq? x #f))) (define (false? x) (eq? x #f))   (eval '(define y 9)) (eval '(if (> y 6) (+ y 2) 15))

We are just walking through a tree … if 6 y > 2 + 15 (eval ) 6 y > Then (eval ) or (eval 15 ) 2 y >

(define (application? exp) (pair? exp)) (define (eval exp) (cond   (define (application? exp) (pair? exp)) (define (eval exp) (cond ((number? exp) exp) ((symbol? exp) (lookup exp)) ((definition? exp) (eval-definition exp)) ((if? exp) (eval-if exp)) ((application? exp) (apply (eval (car exp)) (map eval (cdr exp)))) (else (error "unknown expression " exp)))) ;; rename scheme’s apply so we can reuse the name (define scheme-apply apply) (define (apply procedure arguments) (if (primitive-procedure? procedure) (apply-primitive-procedure procedure arguments) (error "operator not a procedure: " procedure))) (define (apply-primitive-procedure proc args) (scheme-apply (primitive-implementation proc) args))

  (define prim-tag 'primitive) (define (make-primitive scheme-proc) (list prim-tag scheme-proc)) (define (primitive-procedure? exp) (tag-check exp prim-tag)) (define (primitive-implementation prim) (cadr prim)) (define environment (make-frame nil nil)) (add-binding-to-frame! environment ’+ (make-primitive +)) (add-binding-to-frame! environment ’> (make-primitive >)) (add-binding-to-frame! environment 'true #t) (eval '(define z 9)) (eval '(+ 9 6)) (eval '(if true 10 15))

The Environment after first “define” Name Value z 9 true #t > + symbol primitive scheme procedure > symbol primitive scheme procedure +

Evaluation of (+ 9 6) (eval '(+ 9 6)) (apply (eval +) (map eval '(9 6))) (apply '(primitive #[add]) (list (eval 9) (eval 6)) (apply '(primitive #[add]) '(9 6)) (scheme-apply (primitive-implementation '(primitive #[add])) '(9 6)) (scheme-apply #[add] '(9 6)) 15

Evaluation of (if true 10 15) (eval '(if true 10 15)) (eval-if '(if true 10 15)) (if (true? (eval 'true)) ...) (if (true? (lookup 'true))) ...) (if (true? #t) ...) (eval 10) 10 Apply is never called!

Things to observe Applications must be last the case in eval No tag check Applications evaluate all subexpressions Expressions that need special handling, like if, get their own case in eval

The Environment as an Explicit Parameter Change from (eval '(+ 6 4)) to (eval '(+ 6 4) environment) All procedures that call eval have extra argument lookup and define use the environment from the argument No other change from earlier evaluator

The Environment as an Explicit Parameter (define (eval exp env) (cond   ;This change is boring! Exactly the same functionality as #4. (define (eval exp env) (cond ((number? exp) exp) ((symbol? exp) (lookup exp env)) ((definition? exp) (eval-defininition exp env)) ((if? exp) (eval-if exp env)) ((application? exp) (apply (eval (car exp) env) (map (lambda (exp) (eval exp env)) (cdr exp)))) (else (error "unknown expression " exp)))) (define (lookup name env) (lookup-variable-value name env)) (define (eval-define exp env) (let ((name (cadr exp)) (defined-to-be (eval (caddr exp) env))) (add-binding-to-frame! name defined-to-be env) ‘undefined))

(define (eval-if exp env) (if (true? (eval (if-predicate exp) env))   (define (eval-if exp env) (if (true? (eval (if-predicate exp) env)) (eval (if-consequent exp) env) (eval (if-alternative exp) env)))

Defining New Procedures Want to add new procedures For example, a scheme program: (define twice (lambda (x) (+ x x))) (twice 4) Strategy: Add a case for lambda to eval the value of lambda is a compound procedure Extend apply to handle compound procedures Implement environment model

(define (make-procedure parameters body env)   (define (lambda? exp) (tag-check exp 'lambda)) (define (eval exp env) (cond ((number? exp) exp) ((symbol? exp) (lookup exp env)) ((define? exp) (eval-define exp env)) ((if? exp) (eval-if exp env)) ((lambda? exp) (eval-lambda exp env)) ((application? exp) (apply (eval (car exp) env) (map (lambda (e) (eval e env)) (cdr exp)))) (else (error "unknown expression " exp)))) (define (eval-lambda exp env) (make-procedure (lambda-parameters exp) (lambda-body exp) env)) (define (lambda-parameters exp) (cadr exp)) (define (lambda-body exp) (cddr exp)) (define (make-procedure parameters body env) (list 'procedure parameters body env))

Implementation of lambda (eval '(define twice (lambda (x) (+ x x))) GE) (eval '(lambda (x) (+ x x)) GE) (eval-lambda '(lambda (x) (+ x x)) GE) (make-procedure '(x) ’((+ x x)) GE) (list ’procedure '(x) ’((+ x x)) GE) GE This data structure is a procedure! symbol procedure symbol + symbol x

Naming the procedure (eval '(define twice (lambda (x) (+ x x))) GE) z 9 true #t + twice symbol primitive scheme procedure + symbol procedure symbol + symbol x

(define (apply procedure arguments)   (define (apply procedure arguments) (cond ((primitive-procedure? procedure) (apply-primitive-procedure procedure arguments)) ((compound-procedure? procedure) (eval-sequence (procedure-body procedure) (extend-environment (procedure-parameters procedure) arguments (procedure-env procedure)))) (else (error "Unknown procedure type -- APPLY" procedure)))) (define (compound-procedure? exp) (tag-check exp ‘procedure)) (define (procedure-parameters compound) (cadr compound)) (define (procedure-body compound) (caddr compound)) (define (procedure-env compound) (cadddr compound))

(define (eval-sequence exps env) (cond ((last-exp? exps) (eval (first-exp exps) env)) (else (eval (first-exp exps) env) (eval-sequence (rest-exps exps) env)))) (define (last-exp? seq) (null? (cdr seq))) (define (first-exp seq) (car seq)) (define (rest-exps seq) (cdr seq))

How the Environment Works manipulation 3. How the Environment Works E2 x: 10 plus: (procedure ...) E1 Abstractly – in our environment diagrams: Concretely – our implementation (as in SICP) list of values enclosing- environment list of variables frame x plus 10 E2 procedure

Extending the Environment plus: (procedure ...) E3 x: 4 y: 5 Abstractly (extend-environment '(x y) (list 4 5) E2) Concretely E2 list of values list of variables frame x y 4 E3 5 E1

(define (extend-environment vars vals base-env) (if (= (length vars) (length vals)) (cons (make-frame vars vals) base-env) (if (< (length vars) (length vals)) (error "Too many arguments supplied" vars vals) (error "Too few arguments supplied" vars vals))))

Implementation of apply (1) (eval '(twice 4) GE) (apply (eval 'twice GE) (map (lambda (e) (eval e GE)) '(4))) (apply (list 'procedure '(x) ’((+ x x)) GE) '(4)) (eval-seq ’((+ x x)) (extend-environment '(x) '(4) GE)) (eval '(+ x x) E1) GE E1 A names values 4 x

Implementation of apply (2) (eval '(+ x x) E1) (apply (eval + E1) (map (lambda (e) (eval e E1)) '(x x))) (apply '(primitive #[add]) (list (eval 'x E1) (eval 'x E1))) (apply '(primitive #[add]) '(4 4)) (scheme-apply #[add] '(4 4)) 8 GE E1 A names values 4 x

"Scanning" the environment Look for a variable in the environment... Look for a variable in a frame... loop through the list of vars and list of vals in parallel detect if the variable is found in the frame If not found in frame (out of variables in the frame), look in enclosing environment

(define (lookup-variable-value var env) (define (env-loop env) (define (scan vars vals) (cond ((null? vars) (env-loop (enclosing-environment env))) ((eq? var (car vars)) (car vals)) (else (scan (cdr vars) (cdr vals))))) (if (eq? env the-empty-environment) (error "Unbound variable" var) (let ((frame (first-frame env))) (scan (frame-variables frame) (frame-values frame))))) (env-loop env)) (define (enclosing-environment env) (cdr env)) (define (frame-variables frame) (car frame)) (define (frame-values frame) (cdr frame))

(define (define-variable! var val env) (let ((frame (first-frame env))) (define (scan vars vals) (cond ((null? vars) (add-binding-to-frame! var val frame)) ((eq? var (car vars)) (set-car! vals val)) (else (scan (cdr vars) (cdr vals))))) (scan (frame-variables frame) (frame-values frame)))) (define (eval-definition exp env) (let ((name (cadr exp)) (defined-to-be (eval (caddr exp) env))) (define-variable! name defined-to-be env) ‘undefined))

The Initial (Global) Environment primitives and initial env. 4. The Initial (Global) Environment (define (setup-environment) (let ((initial-env (extend-environment (primitive-procedure-names) (primitive-procedure-objects) the-empty-environment))) (define-variable! 'true #t initial-env) (define-variable! 'false #f initial-env) initial-env)) (define the-empty-environment '()) (define the-global-environment (setup-environment))

Is eval an iterative or recursive algorithm? It depends on the scheme program being evaluated An iterative algorithm in scheme (eval '(define odd (lambda (n) (odd (- n 2)))) GE) A recursive algorithm in scheme (eval '(define sum (lambda (n) (+ n (sum (- n 1))))) GE) Base case and if check omitted from both algorithms to simplify the example

(eval '(define odd (lambda (n) (odd (- n 2)))) GE) ==> undefined (eval '(odd 4) GE) [call apply, which creates E1: n = 4, then eval body of odd] (eval '(odd (- n 2)) E1) (apply (eval 'odd E1) (list (eval '(- n 2) E1))) [skip some steps in which (- n 2) ==> 2] (apply (list ’procedure '(n) '(odd (- n 2)) GE) '(2)) [apply creates E2: n = 2, then eval body of odd] (eval '(odd (- n 2)) E2)) No pending operations on the recursive call to eval

(eval '(define sum (lambda (n) (+ n (sum (- n 1))))) GE) (eval '(sum 4) GE) [call apply, which creates E1: n = 4, then eval body of sum] (eval '(+ n (sum (- n 1))) E1) (apply '(primitive #[add]) (list (eval 'n E1) (eval '(sum (- n 1)) E1))) [skip some steps in which (- n 1) ==> 3] (apply '(primitive #[add]) (list 4 (apply (eval 'sum E1) '(3)))) [apply creates E2: n = 3, then eval body of sum] (apply '(primitive #[add]) (list 4 (eval '(+ n (sum (- n 1))) E2)) There are pending operations on the recursive call to eval

Summary Cycle between eval and apply is the core of the evaluator eval calls apply with operator and argument values apply calls eval with expression and environment What is still missing from scheme ? Some special forms data types other than numbers and booleans

SICP’s version of eval (define (eval exp env) (cond ((self-evaluating? exp) exp) ((variable? exp) (lookup-variable-value exp env)) ((quoted? exp) (text-of-quotation exp)) ((assignment? exp) (eval-assignment exp env)) ((definition? exp) (eval-definition exp env)) ((if? exp) (eval-if exp env)) ((lambda? exp) (make-procedure (lambda-parameters exp) (lambda-body exp) env)) ((begin? exp) (eval-sequence (begin-actions exp) env)) ((cond? exp) (eval (cond->if exp) env)) ((application? exp) (apply (eval (operator exp) env) (list-of-values (operands exp) env))) (else (error "Unknown expression type -- EVAL" exp))))

Note some minor differences like (define (self-evaluating? exp) (cond ((number? exp) true) ((string? exp) true) (else false))) (define (variable? exp) (symbol? exp)) (define (list-of-values exps env) (if (no-operands? exps) '() (cons (eval (first-operand exps) env) (list-of-values (rest-operands exps) env))))

Note: Syntactic Abstraction Semantics What the language means Model of computation Syntax Particulars of writing expressions E.g. how to signal different expressions Separation of syntax and semantics: allows one to easily alter syntax syntax procedures eval/apply

Basic Syntax Routines to detect expressions (define (if? exp) (tagged-list? exp 'if)) (define (lambda? exp) (tagged-list? exp 'lambda)) (define (application? exp) (pair? exp)) Routines to get information out of expressions (define (operator app) (car app)) (define (operands app) (cdr app)) (define (first-operand args) (car args)) (define (rest-operands args) (cdr args)) Routines to build expressions (define (make-if predicate consequent alternative) (list 'if predicate consequent alternative))

Example – Changing Syntax Suppose you wanted a "verbose" application syntax: (CALL <proc> ARGS <arg1> <arg2> ...) Changes – only in the syntax routines! (define (application? exp) (tagged-list? 'CALL)) (define (operator app) (cadr app)) (define (operands app) (cdddr app))

Implementing "Syntactic Sugar" Idea: Implement a simple fundamental "core" in the evaluator Easy way to add alternative/convenient syntax? "let" as sugared procedure application: (let ((<name1> <val1>) (<name2> <val2>)) <body>) ((lambda (<name1> <name2>) <body>) <val1> <val2>)

Detect and Transform the Alternative Syntax (define (eval exp env) (cond ((self-evaluating? exp) exp) ((variable? exp) (lookup-variable-value exp env)) ((quoted? exp) (text-of-quotation exp)) . . . ((cond? exp) (eval (cond->if exp) env)) ((let? exp) (eval (let->combination exp) env)) ((application? exp) (apply (eval (operator exp) env) (list-of-values (operands exp) env))) (else (error "Unknown expression" exp))))

Implementing cond: Syntax procedures (define (cond-clauses exp) (cdr exp)) (define (cond-else-clause? clause) (eq? (cond-predicate clause) 'else)) (define (cond-predicate clause) (car clause)) (define (cond-actions clause) (cdr clause))

Cond syntax (cond ((= x 23) (+ x 1)) (else (- x 1))) cond else - x 1 =

Transforming sequence of expression to an expression (define (sequence->exp seq) (cond ((null? seq) seq) ((last-exp? seq) (first-exp seq)) (else (make-begin seq)))) (define (make-begin seq) (cons 'begin seq)) begin * x 2 car y

Implementing cond (Cont.) (cond ((= x 23) (+ x 1)) (else (- x 1))) (if (= x 23) (+ x 1) (- x 1))

Implementing cond (define (cond->if exp) (expand-clauses (cond-clauses exp))) (define (expand-clauses clauses) (if (null? clauses) 'false ; no else clause (let ((first (car clauses)) (rest (cdr clauses))) (if (cond-else-clause? first) (if (null? rest) (sequence->exp (cond-actions first)) (error "ELSE clause isn't last -- COND->IF" clauses)) (make-if (cond-predicate first) (expand-clauses rest))))))

Details of cond syntax transformation (cond ((= x 23) (+ x 1)) (else (- x 1))) cond else - x 1 = x 23 + x 1

Details of cond syntax transformation (expand-clauses else - x 1 = x 23 ) + x 1

Details of cond syntax transformation rest first else - x 1 = x 23 + x 1

Details of cond syntax transformation (make-if = x 23 + x 1 )) (expand-clauses else - x 1

Details of cond syntax transformation (make-if = x 23 + x 1 - x 1 )

Details of cond syntax transformation if = x 23 + x 1 - x 1

Named Procedures – Syntax vs. Semantics (define (foo <parm>) <body>) Semantic implementation – just another define: (define (eval-definition exp env) (define-variable! (definition-variable exp) (eval (definition-value exp) env) env)) (define (definition-variable exp) (if (symbol? (cadr exp)) (cadr exp) (caadr exp))) (define (definition-value exp) (caddr exp) (make-lambda (cdadr exp) ;formal params (cddr exp)))) ;body