Cling – The New C++ Interpreter for ROOT 6 V. Vassilev, P. Canal, A. Naumann, P. Russo CERN, PH-SFT & FermiLab

Role of C++ Interpreter in ROOT ... IO PyROOT TPluginManager ROOT TClass CINT Load/Store C++ objects Runtime Dynamism TFile::Open(“http://...”) gDirectory->Get(“hist”) python runReco.py Fast Prototyping Reflection Type Info vvassilev / CHEP 2012.05.21

Role of C++ Interpreter in ROOT ROOT uses the interpreter much more than one would expect: vvassilev / CHEP 2012.05.21

Cling Is Better Than CINT Full C++ support STL + templates Path to C++11 Correctness Better type information and representations Always compile in memory Much less code to maintain vvassilev / CHEP 2012.05.21

Cling's Dual Personality An interpreter – looks like an interpreter and behaves like an interpreter Cling follows the read-evaluate-print-loop (repl) concept. More than interpreter – built on top of compiler libraries (Clang and LLVM) Contains interpreter parts and compiler parts. More of an interactive compiler or an interactive compiler interface for clang. No need to compile Cling/ROOT with Clang/LLVM vvassilev / CHEP 2012.05.21

Cling Uses Clang & LLVM LLVM and Clang “The LLVM Project is a collection of modular and reusable compiler and toolchain technologies...” vvassilev / CHEP 2012.05.21

Cling's Codebase Other ROOT – 1400K SLOC* CINT+Reflex – 230K SLOC* Cling – 7K SLOC* Make cling visible in the pie chart * No testsuites included. Credits: generated using David A. Wheeler's 'SLOCCount' vvassilev / CHEP 2012.05.21

Cling's Codebase Other ROOT – 1400K SLOC* CINT+Reflex – 230K SLOC* Cling – 7K SLOC* externals: LLVM + Clang – 800K SLOC* Make cling visible in the pie chart * No testsuites included. Credits: generated using David A. Wheeler's 'SLOCCount' vvassilev / CHEP 2012.05.21

Challenges Incompatible concepts like compilation and interpretation Many tasks that are trivial for an interpreter become a nightmare for a compiler. Make C++ usable at the prompt Incorporate the experience we have with CINT. First step: adopt the successful usability extensions from CINT. vvassilev / CHEP 2012.05.21

Cling in a Nutshell vvassilev / CHEP 2012.05.21

Extending C++ Language [cling]$ sin(12); void wrapper() { sin(12); } We want to be able to run statements void wrapper1() { int i = 12; } ? [cling]$ int i = 12; sin(i); void wrapper2() { sin(i); } vvassilev / CHEP 2012.05.21

Extending C++ Language Wrap the input Look for declarations Extract the declarations one level up, as global declarations int i = 12; void wrapper1() { int i = 12; printf("%d\n",i); } [cling]$ int i = 12; printf("%d\n",i); printf("%f\n",sin(i)); void wrapper2() { printf("%f\n", sin(i)); } vvassilev / CHEP 2012.05.21

Streaming Execution Results No semicolon (;) No semicolon (;) [cling]$ int i = 12 (int) 12 sin(i) (double const) -5.365729e-01 vvassilev / CHEP 2012.05.21

Error Recovery Filled input-by-input Incorrect inputs must be discarded as a whole vvassilev / CHEP 2012.05.21

Work in progress: Code Unloading // Calculator.h class Calculator { int Add(int a, int b) { return a - b; } ... }; [cling]$ .L Calculator.h Calculator calc; calc.Add(3, 1) (int) 2 //WTF!?* .U Calculator.h (int) 4 // // Calculator.h class Calculator { int Add(int a, int b) { return a + b; } ... }; * What's That Function vvassilev / CHEP 2012.05.21

Late Binding { TFile* F = 0; if (is_day_of_month_even())  Defined in the root file { TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); hist->Draw(); }  Opens a dynamic scope. It tells the compiler that cling will take over the resolution of possible unknown identifiers  The root file is gone. Issue an error. vvassilev / CHEP 2012.05.21

Automatically transformed into valid C++ code Late Binding { TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); gCling->EvaluateT<void>("hist->Draw()", ...); } hist->Draw(); Automatically transformed into valid C++ code Tell the compiler the identifier will be resolved at runtime Wrap it into valid C++ code Partially recompile at runtime vvassilev / CHEP 2012.05.21

Challenges Error recovery Even though the user has typed wrong input at the prompt cling must survive, i.e issue an error and continue to work. Initialization of global variables Cling depends on global variables, which need to be initialized. However, the global variables continue to be added (potentially) with every input line. Late binding Cling needs to provide a way for symbols unavailable at compile-time a second chance to be provided at runtime. Value printer The interactive mode obeys the repl concept and there should be way of easy print value and type of expression in a user-extensible way. Running statements CINT-specific C++ extension improving the user interaction with the interpreter from the terminal. vvassilev / CHEP 2012.05.21

Cling In The World Announced in July 2011 as working C++ interpreter Cling and OpenGL http://www.youtube.com/watch?v=eoIuqLNvzFs Cling and QT http://www.youtube.com/watch?v=BrjV1ZgYbbA MATLAB to C++ translator Regular bug reports from outside HEP vvassilev / CHEP 2012.05.21

Cling In ROOT vvassilev / CHEP 2012.05.21

Dictionaries Describe compiled code Incarnation of the type information and reflection in ROOT. The only way of crossing the border between interpretation and compilation. CINT and Reflex dictionaries: Double the size of the libraries Multiple copies of the dictionary data in the memory Incompatible reflection formats Do not cover 100% of C++ CINT Reflection TPluginManager IO ... PyROOT Type Info TClass vvassilev / CHEP 2012.05.21

CINT Dictionary Use Source of ROOT's class description Dictionary.(h|cxx) TClass Member functions injected by the ClassDef macro ClassDef Call Stubs Reflection Info root_with_cint$ find . -name "G__*.h" -o -name "G__*.cxx" |wc -l 194 (Dictionary files) root_with_cint$ find . -name "G__*.h" -o -name "G__*.cxx" > /tmp/slocfiles sloccount /tmp/slocfiles 942307 (Lines of real c++ code) Explain what call stubs are used for and reflection info. Helps crossing over in the compiled world. The stubs are called in order CINT to be able to call functions without knowing about the target architecture Target-independent, normalized signatures used to call compiled functions. Teaches CINT what a class contains vvassilev / CHEP 2012.05.21

LLVM Just-In-Time Compiler Compiles lazily just before the function is called Has full knowledge about the target architecture Much faster than interpreting the LLVM Bit Code Cling Clang LLVM JIT Compiled libs(.so) LLVM Bit Code Machine Code (x86, Alpha, ...) vvassilev / CHEP 2012.05.21

Cling + ROOT = ROOT 6 ROOT 6 in November See plenary by Fons on Wednesday Windows support to arrive after 6.00 Work in progress in clang “genreflex”-compatible dictionary generator after 6.00 vvassilev / CHEP 2012.05.21

ROOT 6.x Reduce size of dictionaries Compiled TFormula Mid term: Call Stubs & Reflection Info goes away! Long term: No dictionaries at all Compiled TFormula Dictionary.(h|cxx) TClass ClassDef Call Stubs Reflection Info vvassilev / CHEP 2012.05.21

Additional Features World class performance and optimizations OpenCL C Objective C[++] ... vvassilev / CHEP 2012.05.21

In Action Come and see at our booth! C N G In Action Come and see at our booth! vvassilev / CHEP 2012.05.21

Thank you! Fons's plenary on Wednesday Google Tech Talk LLVM Euro Dev Meeting 2011 http://www.llvm.org/devmtg/2011-09-16/EuroLLVM2011- ImplementingDynamicScopesInCling.pdf LLVM Dev Meeting 2011 http://www.llvm.org/devmtg/2010-11/Naumann-Cling.pdf http://www.llvm.org/devmtg/2010-11/videos/Naumann_Cling- desktop.mp4 Thank you! vvassilev / CHEP 2012.05.21

Backup slides vvassilev / CHEP 2012.05.21

Pre-Compiled Headers/Modules Carefully crafted data structures designed to improve translator's performance: Reduce lexical, syntax and semantic analysis Loaded “lazily” on demand http://clang.llvm.org/docs/PCHInternals.html vvassilev / CHEP 2012.05.21

Pre-Compiled Headers/Modules Design advantages: Loading is significantly faster than re-parsing Minimize the cost of reading Read times don't depend on size Cost of generating isn't large Lego Blocks and Матрьошка example vvassilev / CHEP 2012.05.21

PCH vs PCM PCH PCM And thus PCM much more flexible Lego Blocks and Матрьошка example PCH PCM And thus PCM much more flexible vvassilev / CHEP 2012.05.21

Expressive Diagnostics Column numbers and caret diagnostics CaretDiagnostics.C:4:13: warning: '.*' specified field precision is missing a matching 'int' argument printf("%.*d"); ~~^~ Range highlighting RangeHighlight.C:14:39: error: invalid operands to binary expression ('int' and 'A') return y + func(y ? ((SomeA.X + 40) + SomeA) / 42 + SomeA.X : SomeA.X); ~~~~~~~~~~~~ ^ ~~~~~~~ Fix-it hints FixItHints.C:7:27: warning: use of GNU old-style field designator extension struct point origin = { x: 0.0, y: 0.0 }; ^~ .x = vvassilev / CHEP 2012.05.21

LLVM Compiler Toolchain LLVM Core “The LLVM Project is a collection of modular and reusable compiler and toolchain technologies...” More than 120 active contributors Apple, ARM, Google, Qualcomm, QuIC, NVidia, AMD and more ~250 commits/week Clang “The goal of the Clang project is to create a new C, C++, Objective C and Objective C++ front-end for the LLVM compiler.“ More than 100 active contributors Apple, ARM, AMD and more ~150 commits/week * Stats from last year until 14.10.2011 vvassilev / CHEP 2012.05.21

Correctness http://root.cern.ch/drupal/content/requested-cling- features vvassilev / CHEP 2012.05.21