Macros and general code walkers in Lisp: how useful! or, how useful? Ernst van Waning

Benelux-LispersMacros in Common Lisp2 Overview Short overview of macros in Lisp Macros transform macro calls Macros are not code transformers In frustration I have sometimes thought macros have a certain smell Lisp has many specific code walkers Q: can we make general code walkers?

Benelux-LispersMacros in Common Lisp3 Macros are replaced by their expansion (expansion time) The original call evaporates After expansion, their code may be evaluated (evaluation time) Macros in Lisp

Benelux-LispersMacros in Common Lisp4 Expansion and evaluation Arguments of macro calls are not evaluated At expansion time, appropriate error messages do make sense Evaluation is done after the macro expanded

Benelux-LispersMacros in Common Lisp5 Debugging macros Macroexpand-1 expands your macro once (i.e., applies the expander once) Macroexpand expands your macro call until it is no longer a macro call With a non-macro they just return (values form ’nil) Expanding all macros in all subforms is a different game: code walker

Benelux-LispersMacros in Common Lisp6 Tracing expansions Useful for debugging local macros Useful for studying system macros (defmacro trexp (form &environment env) (let ((*print-pretty* t)) (multiple-value-bind (expansion expanded-p) (macroexpand-1 form env) (cond (expanded-p (format *trace-output* "~&~S~%-m1->~%“ form) `(trexp,expansion)) (t (format *trace-output* "~&~S~%~%" form) expansion)))))

Benelux-LispersMacros in Common Lisp7 A model for macros Lisp is explained using Lisp, so we explain Lisp’s macros with Lisp (defmacro our-expander (name) `(get,name 'expander)) Our-expander is an accessor, i.e., it can be set with setf

Benelux-LispersMacros in Common Lisp8 A model for macros (defun our-macro-call? (xpr) (and (consp xpr) (our-expander (car xpr)))) (defun our-macroexpand-1 (xpr) (if (our-macro-call? xpr) (funcall (our-expander (car xpr)) xpr) xpr))

Benelux-LispersMacros in Common Lisp9 A model for macros (defmacro our-defmacro (name args &body body) (let ((g (gensym)));unique symbol `(progn (setf (our-expander ',name) ;store (lambda (,g) (block,name;for return-from (destructuring-bind,args ',name)))

Benelux-LispersMacros in Common Lisp10 A model for macros The actual argument to an expander is the entire macro call Macros do not evaluate the arguments of the macro call The expander is really a function retaining its lexical environment Macros are not first-class values

What are macros good for?

Benelux-LispersMacros in Common Lisp12 Macros control evaluation Special syntax (dotimes (i 10 i) (princ i)) Implicit quoting `(setf (our-expander ‘,name) …) Design your own control structures (aif xpr pos zero neg) Multiple evaluations (dotimes (i 10 i) (princ i))

Benelux-LispersMacros in Common Lisp13 Macros compute during expansion (defmacro avg (&rest numbers) `(/ numbers))) Definers also compute during expansion Defun, defmacro, &c. do a lot of bookkeeping Think of writing your own definers!

Benelux-LispersMacros in Common Lisp14 Macros as accessors (setf (our-expander name) new) ~m~> (something like) (let ((#:g452 name) (#:g453 'expander) (#:g455 new)) (inverse-get #:g452 #:g453 #:g455)) Setf looks “inside” its first argument There are many accessors with their own setf-expanders

Benelux-LispersMacros in Common Lisp15 Macros set up lexical bindings (dotimes (i 10 i) (princ i)) With-macros with-open-file, with-output-to-string &c. Do-macros do, dolist, dotimes, do-symbols &c.

What more do we want?

Benelux-LispersMacros in Common Lisp17 Macros as code transformers Suppose we want to have type-strict expressions but without pain A macro scalar is fine: (scalar :e) –m-> … But what if I want this? (scalar (exp 1)) -m-> (exp (scalar 1)) -m-> (exp 1d0) -m-> …;at expansion time

Benelux-LispersMacros in Common Lisp18 Code transformation Sums of squares of real numbers are non-negative (at least, in theory ): (scalar (sqrt ssq)) -m*-> (the (scalar 0) (sqrt (the (scalar 0) ssq))) Can we prove that ssq is non-negative by expanding a macro? Macros do not allow arbitrary code transformations

Benelux-LispersMacros in Common Lisp19 Domain specific extensions Lisp is an extensible programming language Build applications as extensions to Lisp Telescoping: a strategy to automatically generate highly optimized domain specific libraries But how easy is it to write in Lisp domain-specific semantic checks? domain-specific optimizations?

Benelux-LispersMacros in Common Lisp20 Term Rewriting Systems (TRS) aka Code Walkers Useful in automatic theorem proving Lisp has many code walkers: Read, eval, print, compile Cross-referencers, and more… Lisp makes it easy to write specific ones

Benelux-LispersMacros in Common Lisp21 A general code walker in Lisp? What is a general code walker? Minimizes effort to write specific ones Traverses all forms, a fortiori special ones …more…? Integration with macros is possible Change macroexpand-1 Surrounding code can be made visible by means of a stack

Benelux-LispersMacros in Common Lisp22 A general code walker in Lisp? The infamous 24 special forms of Lisp: Special forms implemented as macro Macros implemented as special form Traversal of implementation-specific special forms All special forms retrievable in any Lisp We can check if they have a walker

Benelux-LispersMacros in Common Lisp23 A general code walker in Lisp? What about macros? What about system-macros? What about the portability of your walkers?

Benelux-LispersMacros in Common Lisp24 A general code walker in Lisp? Looks like a clear idea, but is it? Even if the idea would be clear, there are real problems to solve Would it really help you write your code walkers? Specific code walkers, however, remain very useful