May 21, 2002 The metafront Tool AoPL, S'02 Language Transformation: The metafront Tool Claus Brabrand Michael I. Schwartzbach BRICS, University of Aarhus, Denmark

May 21, 2002 The metafront Tool AoPL, S'02 Outline Introduction Specificity parsing Language transformation (by example) Current metafront limitations Exercises

May 21, 2002 The metafront Tool AoPL, S'02 Last Week Extend language with macros: New productions New nonterminals (via inductive morphings) Safety: Guaranteed termination Only legal syntax after expansion Errors caught at macro definition-time

May 21, 2002 The metafront Tool AoPL, S'02 metafront Macros are just a special case usage: A is an extension of B: m: L+ => L Make sure only need to write delta:  = L+ \ L metafront x: A => B AB program.a program.b transformation input language input program(s)output program(s) output language

May 21, 2002 The metafront Tool AoPL, S'02 Lex/Yacc? Extensibility: New productions and nonterminals? New terminals (lexical structure)? Errors: S/R, R/R: non-local error propagation Scanner: Scanner/Parser state-correspondence

May 21, 2002 The metafront Tool AoPL, S'02 Lex/Yacc? Extensibility: New productions and nonterminals? New terminals (lexical structure)? Errors: S/R, R/R: non-local error propagation Scanner: Scanner/Parser state-correspondence Instead…

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Grammar Definition:  base alphabet T  Reg(  ) set of regular languages over  N set of nonterminals s  N start nonterminal  : N  P(E*) \ Ø, where E = (T U N) G = ( ,T,N,s,  )

May 21, 2002 The metafront Tool AoPL, S'02 Example: While.l language While { terminal Num = { [0-9]+ } terminal Id = { [a-zA-Z0-9]+ } stm[ass]  := ; [if1]  if ( ) [if2]  if ( ) else [while]  while ( )  … // block and sequence exp[add]  + // and -,*,/ [base]  base[num]  [id]  } language While { terminal Num = { [0-9]+ } terminal Id = { [a-zA-Z0-9]+ } stm[ass]  := ; [if1]  if ( ) [if2]  if ( ) else [while]  while ( )  … // block and sequence exp[add]  + // and -,*,/ [base]  base[num]  [id]  } if(n)x:=0;elsex:=x+1;

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing parse(A: P(E*),  :  *) = 1.Calculate first-set (based on production set) 2.Determine applicability (first-set vs. input string) 3.Terminal Selection (among applicable terminals) 4.Production Selection (based on winner terminal) 5.Parse entity (based on winner production) case t: eat terminal t    ’ case n: parse n (recursively): parse(  (n),  )    ’ case  : stop! 6.Parse tail: parse(advance(A),  ’) parse(A: P(E*),  :  *) = 1.Calculate first-set (based on production set) 2.Determine applicability (first-set vs. input string) 3.Terminal Selection (among applicable terminals) 4.Production Selection (based on winner terminal) 5.Parse entity (based on winner production) case t: eat terminal t    ’ case n: parse n (recursively): parse(  (n),  )    ’ case  : stop! 6.Parse tail: parse(advance(A),  ’)

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing Challenge rounds: 1.Calculate first-set: { if, while, } 2.Determine applicability: { if : 2, while : -1, : 2 } stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  #1 #2 #3 #4 #5 #6 #7 #8 input string: if(n)x:=0;elsex:=x+1;

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing 3.Terminal selection: { if : 2, while : -1, : 2 } –Selection priority: lexicographically –length: prefer longest match; –terminal specificity:prefer most specific match –In example: { if : 2, while : -1, : 2 }  winner: if -terminal if < id because { if }  { [a-zA-Z0-9]+ } t < t’ := regexp(t)  regexp(t’)

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing 4.Production(s) selection: given if - terminal –Selection priority: terminal < nonterminal <  prefer most specific –Also, n < n’ := first(n)  first(n’) –In Example: stm[ass,while] die!; winner: if -entity stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  #1 #2 #3 #4 #5 #6 #7 #8

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing 5.Parse entity: if 6.Parse tail: advance if(n)x:=0;elsex:=x+1; stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  stm[ass]  := ;  [if1]  if ( )  [if2]  if ( ) else  [while]  while ( )  #1 #2 #3 #4 #5 #6 #7 #8 stm[if1]  ( )  [if2]  ( ) else  stm[if1]  ( )  [if2]  ( ) else 

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing parse(A: P(E*),  :  *) = 1.Calculate first-set (based on production set) 2.Determine applicability (first-set vs. input string) 3.Terminal Selection (among applicable terminals) 4.Production Selection (based on winner terminal) 5.Parse entity (based on winner production) case t: eat terminal t    ’ case n: parse n (recursively): parse(  (n),  )    ’ case  : stop! 6.Parse tail: parse(advance(A),  ’) parse(A: P(E*),  :  *) = 1.Calculate first-set (based on production set) 2.Determine applicability (first-set vs. input string) 3.Terminal Selection (among applicable terminals) 4.Production Selection (based on winner terminal) 5.Parse entity (based on winner production) case t: eat terminal t    ’ case n: parse n (recursively): parse(  (n),  )    ’ case  : stop! 6.Parse tail: parse(advance(A),  ’)

May 21, 2002 The metafront Tool AoPL, S'02 Comments and Whitespace Semantics (“ omit o; ”) : language While { terminal { Whitespace = { [ \t\n\r]+ } EndOfLineComment = { "//".. \n } MultiLineComment = { "/*".. "*/" } Omit = { ( | | )* } } omit Omit; // omit defaults to { [ \t\n\r]+ } … } language While { terminal { Whitespace = { [ \t\n\r]+ } EndOfLineComment = { "//".. \n } MultiLineComment = { "/*".. "*/" } Omit = { ( | | )* } } omit Omit; // omit defaults to { [ \t\n\r]+ } … } n  e 1 e 2 … e n corresponds to n  o e 1 o e 2 o … e n o

May 21, 2002 The metafront Tool AoPL, S'02 Well-formedness No left-recursion As for metamorphisms last week… Derivability As for metamorphisms last week… Unique final winner: Production specificity Terminal specificity

May 21, 2002 The metafront Tool AoPL, S'02 Lookahead? Cannot discern stm[decl] from stm[exp] by looking at *** specificity clash: stm[decl vs. exp] round #1 on language JavaSubset { stm[decl]  [exp]  ; decl[var]  ; exp[id]   … } language JavaSubset { stm[decl]  [exp]  ; decl[var]  ; exp[id]   … }

May 21, 2002 The metafront Tool AoPL, S'02 Attractors “Try-to-parse” semantics: false  kill true  select –First([  ]  ) := First(  )  ignored! language JavaSubset { stm[decl]  [ ] [exp]  ; decl[var]  ; exp[id]   … } language JavaSubset { stm[decl]  [ ] [exp]  ; decl[var]  ; exp[id]   … }

May 21, 2002 The metafront Tool AoPL, S'02 The Case Trap Consider: stms will never stop on input “ case… ” because  First( stms ) and case  First( stms) Add “case trap”!: Recall: stm[switch]  switch ( ) { } swb[one]  [more]  case[case]  case : stm[switch]  switch ( ) { } swb[one]  [more]  case[case]  case : First([  ]  ) := First(  )  ignored! stm[trap]  [case] 

May 21, 2002 The metafront Tool AoPL, S'02 The Conjunction Trap Consider: Cannot parse: exp[and] will always attempt to steal an ampersand Add “conjunction trap”!: exp1[exp2]  [andand]  && exp2[exp3]  [and]  & exp3[id]  exp1[exp2]  [andand]  && exp2[exp3]  [and]  & exp3[id]  exp2[trap]  [&&]  x && y & higher precedence than &&

May 21, 2002 The metafront Tool AoPL, S'02 Specificity Parsing Advantages Scanner implicitly derived from grammar Deterministic disambiguation Local conflict resolution Avoids keywordification Selection independent of definition-order Convenient overloading Commit: no explosion; no backtracking

May 21, 2002 The metafront Tool AoPL, S'02 Language Transformation - by example -

May 21, 2002 The metafront Tool AoPL, S'02 Uminus Extend While language’s syntax: Specify transformation: Shorthand: language MyWhile extends While { exp[uminus]  - } language MyWhile extends While { exp[uminus]  - } morph MyWhile2While: MyWhile ==> While { metamorph stms,stm,exp,base; exp[uminus](B) base(B)=>xB ==> >> } morph MyWhile2While: MyWhile ==> While { metamorph stms,stm,exp,base; exp[uminus](B) base(B)=>xB ==> >> } bindersinductive transformations > B=>xB short for base(B)=>xB

May 21, 2002 The metafront Tool AoPL, S'02 More Examples

May 21, 2002 The metafront Tool AoPL, S'02 Current metafront Limitations metafront (170K Java, 6300 lines): –Current limitations: No terminal transformations (micros) All metamorphs must be explicitly declared Attractors must be nonterminals (no in-lining) No metamorph arguments –Current inefficiencies: No terminal selection caching Attractor implementation

May 21, 2002 The metafront Tool AoPL, S'02 Future Language/transformation calculus: Aidx(A): A => A x: A => B,source(x): A x: A => B,target(x): B x: A => B, y: B => Cx;y: A => C x: A => B, y: A => Cx*y: A => B*C …

May 21, 2002 The metafront Tool AoPL, S'02 Exercises

May 21, 2002 The metafront Tool AoPL, S'02 Exercises (any 3 out of 4) 1.Extend Java with a maybe construction: 1.Write Maybe.l, extending Java.l 2.Write Maybe2Java.x 2.Extend Java with a foreach construction: 1.Write Foreach.l, extending Java.l 2.Write Foreach2Java.x 3.Extend While with a switch construction: 1.Write Switch.l, extending While.l 2.Write Switch2While.x 4.Extend Java with any construction of your own choice: 1.Write.l, extending Java.l 2.Write 2Java.x

May 21, 2002 The metafront Tool AoPL, S'02 FIN