Copyright 2005 Eric Niebler xpressive: Library Design on the Edge or, How I Learned to Stop Worrying and Love Template Meta- Programming

Copyright 2005 Eric Niebler “Why program by hand in five days what you can spend five years of your life automating?” -- Terrence Parr, author ANTLR/PCCTS

Copyright 2005 Eric Niebler Overview Domain-Specific (Embedded) Languages xpressive and Dual-Mode DSEL Design Programming at the Compile-time / Runtime boundary

Copyright 2005 Eric Niebler Domain-Specific Languages Mini-languages, everywhere! – GNU Make – Backus Naur Form – Regular expressions Syntax, constructs and abstractions for working efficiently in some narrow domain Not general purpose!

Copyright 2005 Eric Niebler Why DSLs? General purpose languages are low-level, procedural DSLs are high-level and declarative Solution space shares concepts with problem space DSL code is easier to write, read, reason about and maintain.

Copyright 2005 Eric Niebler Embedded DSLs A DSL in a library All the wholesome goodness of a DSL in the host language of your choice! C++ is a good host language – operator overloading – low abstraction penalty

Copyright 2005 Eric Niebler Expression Templates (Ab)uses C++ operator overloading to approximate the syntax of the embedded language Must map embedded language syntax and constructs into C++ Used by: – Blitz++ : high-performance scientific computing – Spirit : parser generator (EBNF)

Copyright 2005 Eric Niebler DSEL example: EBNF Calculator group ::= '(' expr ')' fact ::= integer | group term ::= fact (('*' fact) | ('/' fact))* expr ::= term (('+' term) | ('-' term))* rule<> group, fact, term, expr; group = '(' >> expr >> ')'; fact = integer | group; term = fact >> *(('*' >> fact) | ('/' >> fact)); expr = term >> *(('+' >> term) | ('-' >> term));

Copyright 2005 Eric Niebler Types of DSELs: Dynamic Example: SQLCommand c = "SELECT * from Employees"; Advantages: – Unconstrained syntax – Statements can be specified at runtime Disadvantages: – Syntax errors discovered at runtime – Performance costs of interpretation

Copyright 2005 Eric Niebler Types of DSELs: Static Example: double d = (matrix * vector)(3, 4); Advantages: – Syntax errors checked at compile-time – Aggressive inlining, domain-specific codegen Disadvantages: – Constrained by rules for legal C++ expressions – Cannot accept new statements at runtime

Copyright 2005 Eric Niebler Static DSELs... Dynamic DSELs... Can’t I have it both ways? Hrrmmm...

Copyright 2005 Eric Niebler Dual-Mode DSEL Interface Provide both a static and dynamic interface! Sounds good but... – Can we really get the advantages of both? – Can we share the implementation to avoid code duplication?

Copyright 2005 Eric Niebler "\\w" "\\w+" "a\\w" "a|b" "(\\w)\\1" "[^a-z]" "(?=foo)" _w +_w 'a' >> _w as_xpr('a') | 'b' (s1= _w) >> s1 ~range('a', 'z') before("foo") Expression Template Regular Expression

Copyright 2005 Eric Niebler Get a Date // Match a date of the form "\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?"; regex date = _d >> !_d >> '-' // match month >> _d >> !_d >> '-' // match day >> _d >> _d >> !(_d >> _d); // match year # Match a date of the form /\d\d?-\d\d?-\d\d(?:\d\d)?/ // Match a date of the form regex date = regex::compile( "\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?");

Copyright 2005 Eric Niebler Regex aliases, anyone? regex date = /*... */; // A line in a log file is a date followed by a // space, and everything up to the newline. regex log = date >> ' ' >> +~set['\n']; regex date = /*... */; // A line in a log file is a date followed by a // space, and everything up to the newline. regex log = date >> ' ' >> +~set['\n'];

Copyright 2005 Eric Niebler Semantic Constraints // Only match valid dates regex date = (_d >> !_d)[if_is_month()] >> '-' >> (_d >> !_d)[if_is_day()] >> '-' >> (_d >> _d >> !(_d >> _d))[if_is_year()]; // Only match valid dates regex date = (_d >> !_d)[if_is_month()] >> '-' >> (_d >> !_d)[if_is_day()] >> '-' >> (_d >> _d >> !(_d >> _d))[if_is_year()];

Copyright 2005 Eric Niebler Two great tastes that taste great together // A line in a log file is a date followed by a // space, and everything up to the newline. regex date = regex::compile(get_date_pattern()); regex log = date >> ' ' >> +~set['\n'];

Copyright 2005 Eric Niebler “Some people, when confronted with a problem, think, ‘I know, I’ll use regular expressions.’ Now they have two problems.” --Jamie Zawinski, in comp.lang.emacs

Copyright 2005 Eric Niebler Recursive regexen! regex parens; parens // A balanced set of parens... = '(' // is an opening paren... >> // followed by... *( // zero or more... keep( +~(set='(',')') ) // of a bunch of things that are // not parens... | // or... by_ref(parens) // a balanced set of parens ) // (ooh, recursion!)... >> // followed by... ')' // a closing paren ;

Copyright 2005 Eric Niebler A Regex Calculator?! regex group, fact, term, expr; group = '(' >> by_ref(expr) >> ')'; fact = +_d | group; term = fact >> *(('*' >> fact) | ('/' >> fact)); expr = term >> *(('+' >> term) | ('-' >> term));

Copyright 2005 Eric Niebler Wife: It's a floor wax! Husband: No, it's a dessert topping! Announcer: Stop! You're both right. It's a floor wax and a dessert topping! -- Saturday Night Live C++ library design at the edge

Copyright 2005 Eric Niebler STL, MPL and Fusion, Oh My! // Just the data, ma’am std::list integers; // Just the types, ma’am typedef mpl::list types; // Types and data, please! fusion::tuple data = fusion::make_tuple(1, 3.14, "hello");

Copyright 2005 Eric Niebler The Fusion Library Heterogeneous data structures STL-influenced – containers, iterators, algorithms MPL-compatible by Joel de Guzman, part of Boost.Spirit –

Copyright 2005 Eric Niebler A Simple Fusion-esque List struct nil = {}; template struct cons { Car car; Cdr cdr; cons(Car const & a, Cdr const & d = Cdr()) : car(a), cdr(d) {} }; inline cons make_cons(Car const & a, Cdr const & d) { return cons(a, d); }

Copyright 2005 Eric Niebler Simple Fusion-esque List, cont. cons > > data = make_cons(1, make_cons(3.14, make_cons(std::string("hello"))));

Copyright 2005 Eric Niebler Fusion-esque algorithms template void for_each(nil, F) {} template void for_each(cons const & l, F f) { f(l.car); for_each(l.cdr, f); }

Copyright 2005 Eric Niebler Programming challenge!!! Write the type of a std::pair where Second is a pointer to the whole std::pair... std::pair std::pair *>std::pair *> *> std::pair<int, std::pair<int, std::pair<int, std::pair<int,

Copyright 2005 Eric Niebler Dual-Mode DSEL Design Strategy recursive algorithms one modular core two binding policies: static and dynamic acyclic data structures

Copyright 2005 Eric Niebler xpressive Matchers // Match any single character struct any_matcher { template bool match(Iter i1, Iter i2, Next const &n) const { if ( i1 == i2 ) { return false; } return n.match( ++i1, i2 ); } };

Copyright 2005 Eric Niebler xpressive static Scaffold template struct static_xpression { Matcher matcher; Next next; template bool match( Iter i1, Iter i2 ) const { return matcher.match( i1, i2, next ); } };

Copyright 2005 Eric Niebler xpressive dynamic Scaffold template struct matchable { virtual bool match( Iter, Iter ) const = 0; };

Copyright 2005 Eric Niebler xpr: dynamic Scaffold, cont. template struct dynamic_xpression : matchable { Matcher matcher; matchable * pnext; bool match( Iter i1, Iter i2 ) const { return matcher.match( i1, i2, *pnext ); } };

Copyright 2005 Eric Niebler Separation of Concerns Matchers implement core functionality Scaffolds implement binding policy (static or dynamic) Matchers are binding-neutral, reusable Best of both worlds – perf of static binding – flexibility of dynamic dispatch

Copyright 2005 Eric Niebler References xpressive: – Spirit and Fusion – by Joel de Guzman –

Copyright 2005 Eric Niebler Questions?