Metaprogramming from University to Industry Zoltán Porkoláb Dept. of Programming Languages and Compilers, Faculty of Informatics Eötvös Loránd University, Budapest

2OTS 2007 Maribor Agenda I always knew C++ templates were the work of the Devil, and now I'm sure... - Cliff Click cited by Todd Veldhuisen Parameterized types C++ Template Metaprograms Power of generative metaprograms Sample usages Open questions

3OTS 2007 Maribor Parameterized types (Generics) Widely used in modern programming languages: ADA generics Eiffel generics C++ templates Java generics: Pizza, GJ, Java 1.5 C# generics Clean, Generic Haskell, other functional langs.

4OTS 2007 Maribor Why generics? int max( int a, int b) { if ( a > b ) return a; else return b; } double max( double a, double b) { if ( a > b ) return a; else return b; if ( a > b ) return a; else return b;}//… class date { /* … */ }; date d = max ( d1, d2); Conventional techniques are working only for complete types:

5OTS 2007 Maribor Preprocessor Macro #define MAX(a,b) a > b ? a : b Works, because a macro is typeless. Processed not by the compiler, therefore there are a number of "secondary effects": MAX( x, y)*2 -> x > y ? x : y*2 MAX( ++x, y) -> ++x > y ? ++x : y

6OTS 2007 Maribor Macro – the limits void swap( int& x, int& y) void swap( int& x, int& y) { int temp = x; x = y; y = temp; int temp = x; x = y; y = temp; } Does not work with macro: a macro is typeless. We need a facility to use type parameters.

7OTS 2007 MariborTemplates template template void swap( T& x, T& y) void swap( T& x, T& y){ T temp = x; x = y; y = temp; } Ada generics: "a form of context-sensitive macro" C++ templates: "a clever kind of macro that obeys the scope, naming, and type rules of C++" Does not require different types used as arguments to be explicitly related. In particular, the argument types used as a template need not be from a single inheritance hierarchy.

8OTS 2007 Maribor C++ Function Templates Template is not a single function A schema to instantiate functions on request Parameter deduction Parameter deduction Template instantiation Template instantiation Compilaton time template T max( T a, T b) { if ( a > b ) return a; if ( a > b ) return a; else return b; else return b;} int i = 3, j = 6, k; double x = 3.14, y = 4.15, z; k = max(i,j); z = max(x,y);

9OTS 2007 Maribor C++ Function Templates 2. Strong type system rules are applied int i = 3; double y = 3.14, z; z = max(i,y); // error template template T max( T a, S b) { if ( a > b ) return a; if ( a > b ) return a; else return b; else return b;} z == 3.0; No deduction on return type No runtime information could be used

10OTS 2007 Maribor Explicit specialization int i = 3; double y = 3.14, z; template template R max( T a, S b) { if ( a > b ) return a; if ( a > b ) return a; else return b; else return b;} z = max (i,y);

11OTS 2007 Maribor User specialization const char *s1 = “Hello”, *s2 = “world”; const char *s = max(s1,s2); template <> const char *max(const char *a, const char *b) { if ( strcmp(a,b) < 0 ) return a; if ( strcmp(a,b) < 0 ) return a; else return b; else return b;}

12OTS 2007 Maribor Template overloading You can provide overloadd template definitions The compiler selects the most specific template cout << max (4,5); cout (3.14,’6’); cout << max (“this”, “greater”); R max(S,T) char *max(char *,char *) T max(T, T)

13OTS 2007 Maribor C++ Class Templates 1. Similar way template classes could be defined template class matrix {public: matrix( int i, int j ); matrix( int i, int j ); matrix& operator=( const matrix &other); matrix& operator=( const matrix &other); T at(int i, int j); T at(int i, int j); //… //…private: vector v; vector v;}; Created always with explicit specialisation matrix m(4,5);

14OTS 2007 Maribor C++ Class Templates 2. User specialization template <> class matrix template <> class matrix {public: matrix( int i, int j ); matrix( int i, int j ); matrix( const matrix &other) matrix( const matrix &other) bool at(int i, int j); bool at(int i, int j);private: aBetterRepresentation v; aBetterRepresentation v;}; Used the same way matrix m(4,5);

15OTS 2007 Maribor C++ Templates The C++ templates were first implemented in the early ’90s Accepted as part of the ANSI/ISO in 1994 Erwin Unruh: 1994 unruh.cpp 30: conversion from enum to D requested

16OTS 2007 Maribor Power of C++ templates The Compiler “executes” template metaprograms The result is a non-templated program executed in “run-time” executed in “run-time” In 1966 Böhm and Jacopini proved: Turing machine implementation conditional and looping constructions Turing machine implementation conditional and looping constructions The C++ templates are Turing-complete in compilation time in compilation time

17OTS 2007 Maribor The Factorial example The run-time solution int factorial( int n) { if ( n == 1 ) return 1; if ( n == 1 ) return 1; else return n*factorial(n-1); else return n*factorial(n-1);} int main() { cout << factorial(15) << endl; cout << factorial(15) << endl; return 0; return 0;}

18OTS 2007 Maribor The Factorial example The metaprogram solution template struct Factorial { enum { value = N * Factorial ::value }; enum { value = N * Factorial ::value };}; template <> struct Factorial template <> struct Factorial { enum { value = 1 }; enum { value = 1 };}; int main() { const int fact15 = Factorial ::value; const int fact15 = Factorial ::value; std::cout << fact15 << endl; std::cout << fact15 << endl; return 0; return 0;}

19OTS 2007 Maribor Conditional statement template template struct IF { typedef Then RET; typedef Then RET;}; template template struct IF struct IF { typedef Else RET; typedef Else RET;}; template template IF ::RET max(T t, S s) { if (t > s) return t; if (t > s) return t; else return s; else return s;}

20OTS 2007 Maribor (Run-time) Programs vs. Metaprograms Function (runtime) Data VariableConditionLoopAssignmentClass Type and constant Symbolic names Type selection Recursion No assignment

21OTS 2007 Maribor Data in Template Metaprograms Referential transparency: No assignment Still possible to store, modify, and retrieve data Typelist struct NullType {}; typedef Typelist< char, Typelist<signed char, Typelist > > Charlist; Typelist > > Charlist; char signed char NullType unsigned char

22OTS 2007 Maribor Data handling template struct Length; template <> struct Length struct Length { enum { value = 0 }; enum { value = 0 };}; template template struct Length > { enum { value = 1 + Length ::value }; enum { value = 1 + Length ::value };};

23OTS 2007 MariborMotivation int main() { const unsigned int di = 12; const unsigned int di = 12; const unsigned int oi = 014; const unsigned int oi = 014; const unsigned int hi = 0xc; const unsigned int hi = 0xc; const unsigned int bi0 = binary_value("1101"); const unsigned int bi0 = binary_value("1101"); const unsigned int bi1 = binary ::value; const unsigned int bi1 = binary ::value;} template struct binary { // prepend higher bits to lowest bit // prepend higher bits to lowest bit static const int value=binary ::value*2+N%10; static const int value=binary ::value*2+N%10;}; template<> struct binary // specialization { static unsigned const value = 0; static unsigned const value = 0;};

24OTS 2007 MariborMotivation Constant expression – array size, case label, etc… Better code compiled Faster in run-time Syntactically checked – the language semantic is extended t Design timeCompilation timeRun-time No change in type system Change in run-time values Template metaprograms apply, automatic config of the program Decisions on Stategies, policies

25OTS 2007 Maribor Generative Metaprograms metaprogramming: writing programs that represent and manipulate other programs or themselves (ie:reflection). Metaprograms are programs about programs. introspection: the ability of a program to observe its own state the ability of a program to observe its own stateintercession: the ability to modify its own state the ability to modify its own state Open compilers: transformations on AST Hygenic macros (Scheme) Two level languages: AspectJ, Template Haskell

26OTS 2007 Maribor Areas of Template Metaprogramming Expression templates Blitz+, PETE Blitz+, PETE Static interface checking Early catch of syntactical/semantical errors Early catch of syntactical/semantical errors Extending the C++ type system Introspection Code adaption/optimalization Language embedding

27OTS 2007 Maribor Expression templates Array a, b, c, d, e; // Object-oriented way of a = b + c + d + e double* _t1 = new double[N]; for ( int i=0; i<N; ++i) _t1[i] = b[i] + c[i]; double* _t2 = new double[N]; for ( int i=0; i<N; ++i) _t2[i] = _t1[i] + d[i]; double* _t3 = new double[N*M]; for ( int i=0; i<N; ++i) _t3[i] = _t2[i] + e[i]; for ( int i=0; i<N; ++i) a[i] = _t3[i]; delete [] _t3; delete [] _t2; delete [] _t1; // Fortran like solution: for ( int i=0; i<N; ++i) a[i] = b[i] + c[i] + d[i] + e[i];

28OTS 2007 Maribor Language embedding SQL Gil, et.al.: AraRat Gil, et.al.: AraRat XML parsing Jarvi, et.al.: type-safe XML library Jarvi, et.al.: type-safe XML library Regular expressions Boost::Xpressive Boost::Xpressive Compiler embedding boost::spirit boost::spirit

29OTS 2007 Maribor Language embedding SQL example string s = ”select form ”+tName + ” where 1=1” ; if ( cond1 ) s += ” and fName1 = ” + field1; if ( cond2 ) s += ” and fName2 < ” + field2; if ( cond3 ) s += ” and fName3 = ” + field3; Run-time errors Injection attacks Conversion problems

30OTS 2007 Maribor Language embedding SQL example void f() { const string s = ( (tName / (fName1 == field1 && fName2 < field2) ) [ fName1, fName3 ] ).asSQL(); }

31OTS 2007 Maribor Language embedding Boost::Xpressive example std::string hello( "hello world!" ); sregex rex1 = sregex::compile( "(\\w+)(\\w+)!" ); sregex rex2 = (s1= +_w) >> ' ' >> (s2= +_w) >> '!'; smatch what; if( regex_match( hello, what, rex1 ) ) { std::cout << what[0] << '\n'; // whole match std::cout << what[0] << '\n'; // whole match std::cout << what[1] << '\n'; // first capture std::cout << what[1] << '\n'; // first capture std::cout << what[2] << '\n'; // second capture std::cout << what[2] << '\n'; // second capture}

32OTS 2007 Maribor Embedding alternatives If you are (accidently) not a C++ programmer.NET platform, VB, C#: LINQ project LINQ project Java platform: Eric Van Wijk et.al.: Attribute Grammar-based Language Extensions for Java Eric Van Wijk et.al.: Attribute Grammar-based Language Extensions for JavaStratego TU Delft TU Delft Charles Simonyi: Intentional programming

33OTS 2007 Maribor Intentional programming Charles Simonyi XEROX Palo Alto Bravo XEROX Palo Alto Bravo MicrosoftWord, Excel MicrosoftWord, Excel Intentional softwarehttp:// Intentional softwarehttp://

34OTS 2007 Maribor Open questions The real expressive power Standard tools but: Loki from Andrei Alexandrescu but: Loki from Andrei Alexandrescu boost::mpl boost::mpl Garantees (number of instantiations, etc…) How to design? How to debug?

Metaprogramming from University to Industry Zoltán Porkoláb Dept. of Programming Languages and Compilers, Faculty of Informatics Eötvös Loránd University, Budapest Thank you! Questions?