1. Cse321, Programming Languages and Compilers 1 6/12/2015 Lecture #9, Feb. 12, 2007 A data structure for grammars Computing Nullable and First in SML

2. Cse321, Programming Languages and Compilers 2 6/12/2015 Assignments Project 1 –Recall Project #1, the scanner is due Wednesday Feb. 14 th Midterm Exam. Recall the midterm is scheduled for 1 week from Today.

3. Cse321, Programming Languages and Compilers 3 6/12/2015 fun inputc fname = (fn n => TextIO.inputN(TextIO.openIn fname,n):string); val lex = Mlex.makeLexer (inputc "test.english"); fun inputc h = (fn n => TextIO.inputN(h,n):string); val lex = let val h = TextIO.openIn “test.english” in Mlex.makeLexer (inputc h) end;

4. Cse321, Programming Languages and Compilers 4 6/12/2015 Datatype for grammars val empty = "empty"; fun emptyP x = x=empty; datatype Grammar = Gram of (string list) * (* Non Terminals *) (string list) * (* Terminals *) (string * (string list)) list * (* Productions *) string; (* Start symbol *)

5. Cse321, Programming Languages and Compilers 5 6/12/2015 Example Grammar val g1 = Gram(["Expr","Term","Factor","Expr'","Term'"],["ident","+","-","*","div","(",")","num"],[("Expr",["Term","Expr'"]),("Expr'",["+","Term","Expr'"]),("Expr'",["-","Term","Expr'"]),("Expr'",[]),("Term",["Factor","Term'"]),("Term'",["*","Factor","Term'"]),("Term'",["div","Factor","Term'"]),("Term'",[]),("Factor",["(","Expr",")"]),("Factor",["num"]),("Factor",["ident"]) ],"Expr");

6. Cse321, Programming Languages and Compilers 6 6/12/2015 Operations on Grammars fun termP (Gram(nts,ts,ps,s)) x = List.exists (fn y => y=x) ts; fun nontermP (Gram(nts,ts,ps,s)) x = List.exists (fn y => y=x) nts; fun rhssFor (Gram(nts,ts,ps,start)) symbol = let fun test (lhs,rhs) = lhs=symbol fun rhs (l,r) = r in List.map rhs (List.filter test ps) end;

7. Cse321, Programming Languages and Compilers 7 6/12/2015 Tables Nullable, First, and Follow compute a table by incrementally adding stuff, until no new stuff can be added. A table is simply a list of (string * a ref) pairs (* Create a table with all slots initialized by the function “initf” *) fun init symbols initf = let fun f x = (x,ref (initf x)) in List.map f symbols end;

8. Cse321, Programming Languages and Compilers 8 6/12/2015 Updating When we update a table we need to know if the update makes a change. Since we continue until no new changes are possible. (* Update the slot at "s" with the function "updatef" *) (* returns true if it makes a change, false otherwise *) fun update s updatef pairs = case List.find (fn (y,r) => s=y) pairs of SOME(_,r) => let val old = !r val new = updatef old in if new=old then false else (r := new; true) end | NONE => false

9. Cse321, Programming Languages and Compilers 9 6/12/2015 Basic Strategy let val table = init … symbols val changed = ref true in while (!changed) do (changed := false ; changed := onePass symbols); table end Initialize table Record changes Apply rules until no changes are made in a single pass.

10. Cse321, Programming Languages and Compilers 10 6/12/2015 Code fun nullTable (gram as(Gram(nts,ts,ps,start))) = let val allsymbols = nts @ ts val table = init allsymbols (fn _ => false) val changed = ref true fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun nullify b = List.exists (nullableRhs table) rhss val b1 = update x nullify table val b2 = onePass xs in b1 orelse b2 end in while (!changed) do (changed := false; changed := onePass allsymbols); table end

11. Cse321, Programming Languages and Compilers 11 6/12/2015 and nullable table s = case List.find (fn (y,r) => y=s) table of NONE => false | SOME(_,r) => !r and nullableRhs table [] = true | nullableRhs table (x::xs) = (nullable table x) andalso (nullableRhs table xs);

12. Cse321, Programming Languages and Compilers 12 6/12/2015 val NullT = nullTable g1; - NullT; val it = [("Expr",ref false), ("Term",ref false), ("Factor",ref false), ("Expr'",ref true), ("Term'",ref true), ("ident",ref false), ("+",ref false), ("-",ref false), ("*",ref false), ("div",ref false), ("(",ref false), (")",ref false), ("num",ref false)] : (string * bool ref) list

13. Cse321, Programming Languages and Compilers 13 6/12/2015 Onepass for “first” (* returns true if it makes a change *) fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun first old = let val listOflists = map (firstRhs gram table) rhss val new = List.concat listOflists in norm(old @ new) end val b1 = update x first table val b2 = onePass xs in b1 orelse b2 end

14. Cse321, Programming Languages and Compilers 14 6/12/2015 First for a RHS and first table s = case List.find (fn (y,r) => y=s) table of NONE => [] | SOME(_,r) => !r and firstRhs gram table [] = [empty] | firstRhs gram table [x] = first table x | firstRhs gram table (x::xs) = let val temp = first table x in case List.find emptyP temp of NONE => temp | SOME _ => temp @ firstRhs gram table xs end;

15. Cse321, Programming Languages and Compilers 15 6/12/2015 The main loop fun firstTable (gram as(Gram(nts,ts,ps,start))) = let val termTable = init ts (fn x => [x]) val nontermTable = init nts (fn x => []) val table = termTable @ nontermTable val changed = ref true fun onePass [] = false | onePass (x::xs) = let val rhss = rhssFor gram x fun first old = let val listOflists = map (firstRhs gram table) rhss val new = List.concat listOflists in norm(old @ new) end val b1 = update x first table val b2 = onePass xs in b1 orelse b2 end in while (!changed) do (changed := false; changed := onePass nts); table end

16. Cse321, Programming Languages and Compilers 16 6/12/2015 val FirstT = firstTable g1; val it = [("ident",ref ["ident"]),("+",ref ["+"]),("-",ref ["-"]),("*",ref ["*"]),("div",ref ["div"]),("(",ref ["("]),(")",ref [")"]),("num",ref ["num"]),("Expr",ref ["(","ident","num"]),("Term",ref ["(","ident","num"]),("Factor",ref ["(","ident","num"]),("Expr'",ref ["+","-","empty"]),("Term'",ref ["*","div","empty"]) ] : (string * string list ref) list

17. Cse321, Programming Languages and Compilers 17 6/12/2015 Possible exam questions Define a grammar. Prove that a particular grammar is ambiguous. –i.e. find a single string accepted by the grammar with more than 1 parse. Give a regular expression that describes comments (or some other given lexeme). Translate a regular expression into a finite state automata – where the FSA has transitions – where the FSA has no transitions Be able to construct an abstract (inductive) data definition for binary trees (or some other data type)

18. Cse321, Programming Languages and Compilers 18 6/12/2015 Possible exam questions (cont. 2) Write a function over an abstract (inductive) data type by using patterns. Produce the First, and Last sets from a regular expression. Simulate a top-down parse of a string by hand for a particular grammar. Remove left recursion from a grammar. Layer an ambiguous grammar to produce non- ambigous grammar.

19. Cse321, Programming Languages and Compilers 19 6/12/2015 Possible exam questions (cont. 3) Write a recursive descent parser for a simple grammar. Construct the First and Follow sets from a context free grammar Given the first and follow sets, construct a predictive parsing table for a given grammar. Tell if such a grammar is ambiguous. Simulate a bottom up (shift-reduce) parse by hand using a stack and an input stream, for a given string and a given grammar. Disambiguate a shift reduce parser by using precedence and associativity information.

20. Cse321, Programming Languages and Compilers 20 6/12/2015 Possible exam questions (cont. 4) Write simple anonymous function in ML Use the List library functions List.map, List.filter, List.exists, Lists.all, List.find to write simple functions.