1. Macros You might recall from earlier that some of the statements we use are not actually functions but macros –In CL, a function evaluates all of its parameters and applies them to the function if is a macro, not a function –only one of the two clauses gets evaluated based on the condition let is a macro –the variables are not evaluated, only allocated and bound –the initialization code is evaluated but not the variables dolist and dotimes are not macros –the (var lis/value) is not a function call etc –Macros are way of writing code that can be used to generate code rather than be invoked by code –Macros are essential in order to “grow” the language many programming languages have macro facilities, but few languages use them as extensively as in Lisp

2. What is a Macro A macro is a piece of code, much like a function –but in the case of a macro, the code is not evaluated when entered into the REPL environment –instead, it is evaluated on demand The macro returns a form (code) rather than a value –so that the code returned can be evaluated by someone else at a later time The macro can be defined so that it builds code –you can also pass it code to be used as it builds code –for instance, you might want to build a function by appending pieces of code to a list and then returning the list when done So the macro allows us to postpone evaluation –defining a macro is somewhat like writing a function, you use defmacro, include params and follow it by the code to be executed

3. Controlling Evaluation The ’ is normally used to control evaluation –We have used ’ to say “take literally, do not evaluate” In a macro, we can continue to use the ’ but there is a superior operator to use: ` (back quote) –The quote and back quote both say: “do not evaluate what follows” but the back quote permits the use of the, (comma) which means “unsuppress the back quote” the, will allow us to send a name to a macro definition and use that name in place of the parameter’s name –Deciding when to use ` and, will be challenging if you try to use, outside of a ` you will get an error you do not have to use, in your ` code but if not, then it is the same as if you were using ’ –Note that we could avoid using ` and, by using a combination of list and ’ as shown on the next slide but this would get tedious

4. Back Quote and Comma Examples `(the sum of 5 and 10 is (+ 5 10)) –since this is all in the scope of `, none of it is evaluated, instead it is returned verbatim: (the sum of 5 and 10 is (+ 5 10)) we get the same result if we use the normal quote –if we place a, prior to (+ 5 10) we get (the sum of 5 and 10 is 15) ` (like ’) says “do not evaluate, just return” whereas ` …,x... says “evaluate x right now” `(dolist (a lis) (print a)) –returns (dolist (a lis) (print a)) –assume that lis is the list ’(a b c d), then `(dolist (a,lis) (print a)) returns (dolist (a (a b c d)) (print a)) `(,x +,y =,(+ x y)) –if x = 3 and y = 4 then this returns (3 + 4 = 7) –Notice that we could achieve the same results by placing any of these lists into a (list) function call and quoting each item in the list except those that have a, (list ’the ’sum ’of ’5 ’and ’10 ’is (+ 5 10)) (list x ’+ y ’= (+ x y))

5. A More Useful Example Consider that we want a function that will evaluate an expression and return one of three values –First value if expression is negative –Second value if expression is zero –Third value if expression is positive This is equivalent to FORTRAN’s first if statement –if(expression) a, b, c –We can’t use defun because all 3 expressions would be evaluated We could use an if or cond statement though, but using a macro is easier (defmacro if3 (expr val1 val2 val3) `(cond ((<,expr 0),val1) ((=,expr 0),val2) (t,val3))) If we call if3 as (if3 (- a b) ’+ ’0 ’-) this becomes (or unfolds into): (cond ((< (- a b) 0) ’+) ((= (- a b) 0) ‘0) (t ’-) You can see that (- a b) is substituted for expr, this is called a macro-substitution

6. Multiple Assignment Example Assume that you want to assign multiple variables to the same expression –We can do this by (setf a expr b expr c expr …) but this is inefficient (we are evaluated expr several times) –I could also do (let ((temp (expr))) (setf a temp b temp c temp) …) but this can be tedious –What I want to do is use code like this (setf a b c … expr) setf however does not work properly, so instead I will define a macro (defmacro assign2 (v1 v2 exp) `(let ((temp,exp)) (setf,v1 temp) (setf,v2 temp) temp)) (defmacro assign-many (exp v1 &optional v2 v3 v4) `(let ((temp,exp)) (setf,v1 temp) (if (not (null,v2)) (setf,v2 temp)) (if (not (null,v3)) (setf,v3 temp)) (if (not (null,v4)) (setf,v4 temp)) temp)) Returns the value assigned Up to 4 variables can be assigned, only one is required Notice the order, exp comes first

7. Variable Assignment Example Here I want to choose which of two variables to assign to the other –For instance, if a < b, then assign a to b otherwise assign b to a –Code: (if (< a b) (setf a b) (setf b a)) –As a macro: (defmacro assigntwoways (v1 v2 test) `(progn (if,test (setf,v1,v2) (setf,v2,v1)),v1)) Notice the use of progn here, it is needed because of the,v1 at the end without `(progn, CL would think that the ` was only in front of the if statement meaning that the,v1 has a, outside of a ` scope Since this code unfolds into an if statement which executes a setf, v1 or v2 will change – as opposed to writing a defun statement where we would have to pass back v1 and/or v2 to be stored again

8. Writing a Macro So far it doesn’t seem difficult to write a macro but –how do you know when to use ` and, –how do you know what code to use –how do you know when to use progn or let Start by writing the code that you want to have produced in a given setting –(cond ((< (- x y) 0) x) ((= (- x y) 0) y) (t (+ x y))) Now make it into a function with parameters to replace the various items (the statements x, y, (+ x y) and the expression (- x y)) with variables Since the variables are not to be evaluated by the macro, but substituted when the macro is called, insert a, before them Since we have, in our code, we have to ` -- find a location where ` will cover the scope of all parameters that have a, Replace defun with defmacro

9. Example I want to write a do loop that will iterate while the loop variable is positive –(do ((x init step)) ((<= x 0)) code) I want this code to be generated whenever anyone wants to use it by passing the variable, the init value, the step function, and the code (defun while-positive (x init step code) (do ((x init step)) ((<= x 0)) code)) The defun has a flaw, x in the loop is not the same as the x passed as a parameter, I need macro substitution: (defmacro while-positive (x init step code) `(do ((,x,init,step)) ((<=,x 0)),code)) Call this with (while-positive y 10 (- y 2) (print y))

10. Using &Body and,@ So far, the body of our construct (the executable code inside the loop or if statement) has been a single statement –this approach does not work if we have lists of executable statements for instance, (while-positive x (+ x 1) (setf a (+ a x)) (print a)) does not work because there are two items where the macro expects the single item code –to accommodate multiple statements as the body, we could use &rest (all of the following code is placed into a single list of functions) –instead, we use a variation called &body the only difference is that &body pretty prints better than &rest We supply a single parameter after this which is then the collection of all items passed –inside of the ` scope, we can apply the body by doing,@parameter

11. Examples Using `,@ vs ’ Backquote SyntaxEquivalent List-Building CodeResult `(a (+ 1 2) c)(list 'a '(+ 1 2) 'c)(a (+ 1 2) c) `(a,(+ 1 2) c)(list 'a (+ 1 2) 'c)(a 3 c) `(a (list 1 2) c)(list 'a '(list 1 2) 'c)(a (list 1 2) c) `(a,(list 1 2) c)(list 'a (list 1 2) 'c)(a (1 2) c) `(a,@(list 1 2) c)(append (list 'a) (list 1 2) (list 'c))(a 1 2 c)

12. Code Examples A number of the operations we have been using are macros by using,@body: –(defmacro when (condition &body body) `(if,condition (progn,@body))) –(defmacro unless (condition &body body) `(if (not,condition) (progn,@body))) –(defmacro dotimes ((var1 var2) &body body) `(do ((,var1 0 (+,var1 1))) ((=,var1,var2)),@body)) Notice the extra layer of ( ) in the parameters to fit the dotimes form (dotimes (a 10) …) –(defmacro dolist ((var1 lis) &body body) `(do ((,var1 (car,lis) (car,lis))) ((null,lis)) (setf,lis (cdr,lis)),@body)) the @ is used in conjunction with, to say “evaluate the following as a group” Many implements of CL allow you to use &rest instead of &body but use &body just to play safe, it also helps when pretty-printing

13. Destructuring Notice in some of the previous examples that we wrapped our params in an extra set of ( ) –When binding the formal parameters (those in the macro header) to the actual parameters (those in the macro call), CL does something called destructuring – it finds the matching parameters based on embeddedness (defmacro dotimes ((var1 var2) &body body) `(do ((,var1 0 (+,var1 1))) ((=,var1,var2)),@body)) (dotimes (x 10) …) – var1 gets x, var2 gets 10 Destructuring can be done to any arbitrary nestedness –consider some examples (defmacro m1 ((a b) (c d) (e f)) …) called with (m1 (1 2) (3 4) (5 6)) (defmacro m2 (a (b c) ((d) e)) …) called with (m2 1 (2 3) ((4) 5)) (defmacro m3 (a (b &optional c) (&optional d)) …) called with (m3 1 (2) ( )) or (m3 (1 (2 3) ( )) or (m3 (1 (2) (3))

14. More Macro Examples Consider that we want to write a counter controlled loop that iterates over the sequence of prime numbers –(do ((i 2 (+ i 1))) ((= i n)) (if (prime i) (b i))) b is the body of code to execute on each prime This becomes the macro –(defmacro doprimes ((var limit) &body b) `(do ((,var 2 (+,var 1))) ((=,var,limit)) (if (prime,var),@b)) Now consider the same thing over a sequence generated by some function f rather than prime –(defmacro dosequence ((var limit f) &body b) `(do ((,var 2 (+,var 1))) ((=,var,limit)) (if (funcall,f,var),@b))) could be called by (dosequence (a 100 #’evenp) (print a)) –(defmacro dosequence ((var limit f) &body b) `(do ((,var 2 (+,var 1))) ((=,var,limit)) (if (funcall #’,f,var),@b))) could be called by (dosequence (a 100 evenp) (print a))

15. Building Code From A List Let’s make a simple switch statement in CL –In Java/C, we do switch (var) { case val1 : statement; break; case val2 : statement; break; …} –Here, we want to be able to say (switch x val1 statement1 val2 statement2 …) we don’t need to add the extra syntax of “case” and :, ; to simplify matters, we will assume that each statement is a single operation, and to make it even easier, merely a return value (say a number) we will also simplify the syntax by permitting the values and return values to be stated outside of a list (switch x 1 1 2 20 3 300 4 -1) – x = 1 returns 1, x = 2 returns 20, x = 3 returns 300, x = 4 returns -1 Problem: we can’t just use,@body here because we need to divide the body up into separate statements that say ((= x 1) 1) ((= x 2) 20) ((= x 3) 300) etc –So rather than relying on,@body, we will build our body through a do loop

16. Building Our Macro It seems like we could write any function to build a cond statement for us: –(let ((temp ’(cond))) –(do … (setf temp (append temp ’((= x 1) 1)) However, we need to be able to insert the variable name provided by the caller as well as the unique list of values/return values –So a macro it is! In the macro though, we have to differentiate whether we are appending to a local variable, or manipulating a parameter –The former is done outside of the ` scope, the latter is done within the ` scope (defmacro switch (var &body lis) (let ((temp '(cond))) (do ((i 0 (+ i 2))) ((>= i (length lis))) (setf temp (append temp `(((=,var,(nth i lis)),(nth (+ i 1) lis)))))) temp)) Do lis elements in pairs, (nth i lis) being the value, used in (= var val) and the next element being the return value Notice this is the only part of the macro that has ` scope

17. An Alternative Approach (defmacro switch (var &body b) (let ((temp '(cond)) (count 0) temp2 temp3 item) (do ( ) ((null b)) (setf item (car b)) (setf b (cdr b)) (if (evenp count) (setf temp2 `(=,var,item)) (progn (setf temp3 (append (list temp2) `(,item))) (setf temp (append temp (list temp3))))) (setf count (+ 1 count))) temp)) The previous approach was concise but perhaps not as easy to understand as it could be –Here we more clearly build each switch argument as two separate lists, one storing (= var item) and the other being an executable statement –We then tack each together into a list and then append it to the growing cond statement When called with (switch x 0 (print ’a) 1 (print ’b) 2 (print ’c)) unfolds into (cond ((= x 0) (print ’a)) ((= x 1) (print ’b)) ((= x 2) (print ’c)))

18. Improving Our Macro What if we want to permit a default? –Since the default case should be the last in our list, we will exit our loop once we find the default –Notice the use of `t – this permits a default case to be selected when default is found in the list we could alternatively use `default if we want the programmer to use the word default when calling the macro –We could invoke this macro with (switch2 x 1 10 2 20 t 30) if x were 2, we would get 20, if x were 5, we would get 30 (defmacro switch2 (var &body lis) (let ((temp '(cond))) (do ((i 0 (+ i 2))) ((>= i (length lis))) (if (equal (nth i lis) `t) (setf temp (append temp `((t,(nth (+ i 1) lis))))) (setf temp (append temp `(((=,var,(nth i lis)),(nth (+ i 1) lis))))))) temp))

19. Macro Expanding When you define a macro, you are returning a new form which can be called directly –From the REPL interface –From another function But the macro is actually performing substitution by taking what you defined and substituting parameters –This is done through macro-expansion –What is returned is then executed You can view the intermediate form without execution by using macroexpand-1 –(macroexpand-1 ’(macroname params)) For example, I want to see the code generated by doprimes passing it the parameters (n 12) and (print n): –(macroexpand-1 '(doprimes (n 12) (print n))) –(DO ((N 2 (+ N 1))) ((= N 12)) (IF (ISPRIME N) (PRINT N))) –T Notice that macroexpand-1 prints out the macro as expanded by the parameters, and then returns T (or nil if there is a failure)

20. Using Macros to Define Functions To this point, our macros have expanded into function calls and the return of the function call is what the macro call returns –consider switch which expands to a cond statement that is then executed We can also use macros to generate defun statements –this lets us generate functions to be called later Consider a database using a list of structures –we could define search functions for all of our slots, but this could be time consuming –or we could define a macro that generates various search functions for us How? –we need to have access to all of the structures’ slots and their types, for each slot, we generate one or more defun statements from our macro

21. Starting The Macro Consider that we have a student structure which stores the slots: name, major, hours, gpa of types string, string, integer, number –We want to define search functions similar to the following: –However, we don’t want to have to write them all So we want a macro which will generate defun statements This macro will need to know the names of the slots –in this way, we can access the structure’s slot as we did above with (student-hours a) And the type of each slot –so that we know which comparison functions to generate (an integer will probably require, and =, but a string might include others such as contains, does-not-contain, (defun search-hours> (db target) (dolist (a db) (if (> (student-hours db) target) (print a))))

22. Partial Macro Solution (defmacro generate-a-search (struct-name slot-name slot-type) (let ((function-name (intern (concatenate 'string (symbol-name struct-name) "-" (symbol-name slot-name) ">"))) (compare-type (if (or (equal slot-type 'number) (equal slot-type 'integer)) #'> #'String>)) (accessor-name (intern (concatenate 'string (symbol-name struct-name) "-" (symbol-name slot-name))))) `(defun,function-name (db target) (dolist (a db) (if (apply,compare-type (,accessor-name a) target) (print a)))))) Now, we call this for each slot-name/slot-type in the structure Example: (generate-a-search STUDENT MAJOR STRING) We have to expand on this if we want to go beyond simple >, <, = searches

23. Continued Let’s go beyond the search producing macro –We would like to pass a structural definition to a macro, it can produce for us an entire database The struct definition Functions for search A function to print the structure in a formatted way A function to test two structures for equality (equal if they have the same slot values) A function to input a new structure –Etc –The structural definition should include the slot names, but here we might also want to include each slot’s type we can decide to what search functions to produce (e.g., a search based on whether a string contains a substring rather than simply string, string=) we can use the proper form of equality (=, equal, char=, string=, etc) we can generate a proper format statement for an output function –On the next slide, we have a macro that produces the defstruct and an equal’s function Notice the weird use of `,variable –This became necessary in order to pass parameters properly

24. Example Continued (defmacro generate-struct (name slots types) (let (slot type temp1 temp2 fname (s1 (gensym)) (s2 (gensym)) andlist eq-func) (setf fname (intern (concatenate 'string (symbol-name `,name “-equals”)))) (setf andlist '(and)) (dotimes (i (length slots)) (setf slot (nth i slots)) (setf type (nth i types)) (if (or (equal type 'integer) (equal type 'number)) (setf eq-func '=) (if (equal type 'character) (setf eq-func 'char=) (setf eq-func ' string=))) (setf temp1 (list (intern (concatenate 'string (symbol-name `,name) "-" (symbol-name `,slot))) `,s1)) (setf temp2 (list (intern (concatenate 'string (symbol-name `,name) "-" (symbol-name `,slot))) `,s2)) (setf andlist (append andlist (list `(,eq-func,temp1,temp2))))) ;; andlist becomes (and (equal (slot-name1 s1) (slot-name1 s2)) …) for each slot `(progn (defstruct,name,@slots);; produce the defstruct (defun,fname (,s1,s2),andlist))));; produce the equal function

25. A Problem Consider the following macro definition: –(defmacro foo (a b) `(let ((sum 0)) (do ((,a 1 (+ 1,a))) ((=,a,b)) (setf sum (+ sum,a))) sum)) This macro merely constructs a do loop that iterates from 1 to b, summing up the values and returning the sum – obviously we don’t need a macro for this, but it illustrates an important point No big deal right? –If we were to do (foo a 10) then it sums up 1-9 and returns 45 as we would expect –What do you suppose will happen if we do (foo sum 10)? Let’s macro expand this (foo sum 10) and find out: –(LET ((SUM 0)) (DO ((SUM 1 (+ 1 SUM))) ((= SUM 10)) (SETF SUM (+ SUM SUM))) SUM) –Notice our terminating condition is not what we had expected – instead of stopping once the loop variable reaches 10 because our loop variable’s name is replaced with sum –And the problem here is that (= sum 10) is never true (sum takes on the values 1, 3, 7, 15, 31, …

26. The Problem: Same Named Items The problem with our last example was that a parameter that we passed to the macro to substitute for,a happened to be the same name as a local variable –What are the odds of that happening? –Probably not very high, but if it does happen, it produces behavior that we don’t want The solution? –Renaming the parameter isn’t an adequate solution because we can’t control what the other programmers might do and since we can’t predict what they will do, we need to play safe –Renaming the local variable would work if we could come up with a name that we are guaranteed that they will never ever use how about Slartibartfast??? Ok, someone out there might be compelled to name a variable this!

27. The Solution: Gensym Gensym is a CL function that generates a unique symbol, one that is not used elsewhere in the system –These symbols will differ from user defined symbols because they have #:G generated followed by a unique number as in #:G708 –Using gensym several times in a row will generate several symbols with consecutive integer values (so that the next would be #:G709) This does not seem very usable – what good is it to generate a seemingly random symbol? –Its use is this: we will name any local variable using gensym so that the name of the local variable is unique (let ((sum (gensym))) … now for the remainder of the macro, refer to,sum instead of sum, sum is the parameter passed to the macro,,sum is a symbol like #:G708 – there is no way that a programmer can pass #:G708 it is disallowed, therefore we are guaranteed a unique name

28. Another Example In this example, there doesn’t seem to be a problem since we are not declaring local variables –(defmacro cube (n) `(*,n,n,n)) –This works fine when called with (cube 2) or (cube x) where x was set to some value previously or even (cube (- x y)) where x and y were both previously set But what happens if we call cube as (cube (incf x))? –Let’s macroexpand this: (macroexpand-1 ’(cube (incf x)))  (cube (incf x) (incf x) (incf x)) –Recall that incf is destructive the value of x changes every time it is called so if we originally set x to 2, then we get (cube 3 4 5)  60! To get around this, lets again use gensym –(defmacro cube (x) (let ((name (gensym))) `(setf,name x) (*,name,name,name)))

29. Revised Macro Here we have pulled the let statement out to appear prior to the ` –This allows us to generate a symbol for sum, and then reference that generated symbol so that we don’t have to take the chance that the programmer called foo with sum If we were to macro-expand (foo sum 10) now we get: –(PROGN (SETF #:G829 0) (DO ((SUM 1 (+ SUM 1))) ((= SUM 10)) (SETF #:G829 (+ #:G829 SUM))) #:G829) –Notice how #:G829 has been generated to replace our local variable so that our terminating condition no longer is affected by that variable (defmacro foo (a b) (let ((sum (gensym))) `(progn (setf,sum 0) (do ((,a 1 (+,a 1))) ((=,a,b)) (setf,sum (+,sum,a))),sum)))

30. Do We Need A Macro? Consider this macro: –Do we really need to make it into a macro? Could we not just use defun? –Yes, just use defun – so under what circumstance(s) do we want to use defmacro? Both defun and defmacro generate new pieces of code but defmacro gives us the ability to control when things are evaluated so that we can make use of macro substitution –If at the time I am writing the code, I do not know how the code will be used by another programmer, I should use defmacro –Alternatively, if I have a routine that depends on another function (like dosequence or doprimes), I could write a function which is passed the function to apply using funcall (defmacro is-greater (x y) `(>,x,y))