Efficient Language-Level Virtualization GraalVM Efficient Language-Level Virtualization This is a Title Slide with Picture slide ideal for including a picture with a brief title, subtitle and presenter information. To customize this slide with your own picture: Right-click the slide area and choose Format Background from the pop-up menu. From the Fill menu, click Picture and texture fill. Under Insert from: click File. Locate your new picture and click Insert. To copy the Customized Background from Another Presentation on PC Click New Slide from the Home tab's Slides group and select Reuse Slides. Click Browse in the Reuse Slides panel and select Browse Files. Double-click the PowerPoint presentation that contains the background you wish to copy. Check Keep Source Formatting and click the slide that contains the background you want. Click the left-hand slide preview to which you wish to apply the new master layout. Apply New Layout (Important): Right-click any selected slide, point to Layout, and click the slide containing the desired layout from the layout gallery. Delete any unwanted slides or duplicates. To copy the Customized Background from Another Presentation on Mac Click New Slide from the Home tab's Slides group and select Insert Slides from Other Presentation… Navigate to the PowerPoint presentation file that contains the background you wish to copy. Double-click or press Insert. This prompts the Slide Finder dialogue box. Make sure Keep design of original slides is unchecked and click the slide(s) that contains the background you want. Hold Shift key to select multiple slides. Apply New Layout (Important): Click Layout from the Home tab's Slides group, and click the slide containing the desired layout from the layout gallery. Eric Sedlar Vice-President & Technical Director, Oracle Labs

We live in a multilingual computing world Long tail of programming languages If you run more than a few apps you probably run more than a few languages We use different languages for different jobs: R or Python for data science Ruby or JavaScript for front ends Java or C/C++ for server back ends Unfortunately, most runtimes can’t run most languages very well JVM doesn’t run JS, Python or native code well CLR doesn’t run languages not from MSFT very well

GraalVM is a meta-runtime for language virtualization Doesn’t just run apps—designed to run things that run apps (i.e. interpreters) GraalVM languages are built on the “Truffle” API (in Java) for constructing interpreters Interpreters define the semantics of the language, and are also used to profile program behavior to guide compilation of frequently-used code (e.g. loops) GraalVM includes multiple language interpreters, and learns the type system and language from what they do in order to generate an efficient compiler based on that interpreter Since the interpreters are all implemented in Java, merging them together into a single engine looks the same to GraalVM like a single (very complicated) language (with zero language crossing costs) Most language runtimes are programs designed to run a particular programming language. Graal is designed to run programs that run languages. It’s one level “meta” versus a traditional language runtime. Compiled code snippet JavaScript Interpreter Ruby Interpreter R Interpreter Python Interpreter Compiled code snippet Graal compiler Compiled code snippet Compiled code snippet Compiled code snippet Compiled code snippet Truffle API Java Runtime Graal watches Truffle API activity to learn the language

Let’s fill in the complete GraalVM architecture (I) Step one: run each language’s application, libraries & frameworks on the interpreters Looks easy, right? … JavaScript application JavaScript libraries Node.js Ruby application Ruby gems & libraries Rails Compiled code snippet JavaScript Interpreter Ruby Interpreter R Interpreter Python Interpreter Compiled code snippet Graal compiler Compiled code snippet Compiled code snippet Compiled code snippet Compiled code snippet Truffle API Java Runtime

But many language runtimes have a lot of native libraries They use native code for better performance in core areas or for systems functionality If you run native code in GraalVM, doesn’t that break all of the abstraction you want from a virtual machine? Is that a secure way to run applications? Native code isn’t in a safe sandbox … JavaScript application JavaScript libraries Node.js Ruby application Ruby gems & libraries Rails JavaScript Interpreter Ruby Interpreter R Interpreter Python Interpreter Graal compiler Compiled code snippet Compiled code snippet Compiled code snippet Truffle API Compiled code snippet Compiled code snippet Java Runtime Compiled code snippet Native Libraries & OS ???

We have a solution for that—another Truffle interpreter LLVM (“low level virtual machine”) bitcode interpreter LLVM supports dozens of (static) languages (C, C++, FORTRAN, COBOL, Rust, Julia, Swift, Objective-C, …) The LLVM compiler generates “bitcode” (i.e. a .bc file) for all of the languages as an intermediate step We can write a Truffle interpreter for LLVM bitcode and run in the (bounds-checked) VM sandbox! … JavaScript application JavaScript libraries Node.js Ruby application Ruby gems & libraries Rails R native libraries Node.js native libs Ruby/Rails native libs Compiled code snippet JavaScript Interpreter Ruby Interpreter R Interpreter Python Interpreter LLVM .bc Interpreter Compiled code snippet Graal compiler Compiled code snippet Compiled code snippet Compiled code snippet Compiled code snippet Truffle API Java Runtime

This picture of the GraalVM architecture is simplified The parts look a bit different depending on where GraalVM is embedded Notice we aren’t showing how to run Java apps yet? … JavaScript application JavaScript libraries Node.js Ruby application Ruby gems & libraries Rails R native libraries Node.js native libs Ruby/Rails native libs Compiled code snippet JavaScript Interpreter Ruby Interpreter R Interpreter Python Interpreter LLVM .bc Interpreter Compiled code snippet GraalVM compiler Compiled code snippet Compiled code snippet Compiled code snippet Compiled code snippet Truffle API Java Runtime Threads / Scheduler Filesystem Network GC / heap Code cache

GraalVM embedded in the Java VM The parts look a bit different depending on where GraalVM is embedded When GraalVM is embedded in the HotSpot JVM, you can run Java apps directly on the JVM without Truffle In this case, JVM uses Graal as a Java bytecode compiler via the JVM Compiler Interface (JVMCI, or JEP 243) Note: Truffle gives an “Abstract Syntax Tree” (language-agnostic representation of work to do) to Graal, not Java bytecodes JavaScript application Ruby application … R native libraries Node.js native libs Ruby/Rails native libs JavaScript libraries Ruby gems & libraries eval("application/x-ruby", "def add(a, b) a + b; end;"); Node.js Rails Polyglot API JavaScript Interpreter Ruby Interpreter R Interpreter LLVM .bc Interpreter Compiled code snippet Compiled code snippet Truffle API Java application Compiled code snippet Graal compiler Truffle CI Compiled code snippet Compiled code snippet JavaVM Runtime Compiled code snippet JVMCI Threads / Scheduler Filesystem Network GC / heap Code cache

GraalVM embedded in the Oracle Database Embedding GraalVM allows substitution of the underlying engine’s service layer Oracle database takes on many functions of an OS: scheduling, resource management, I/O GraalVM allows mixing & matching system components from the underlying runtime JavaScript application Ruby application … R native libraries Node.js native libs Ruby/Rails native libs JavaScript libraries Ruby gems & libraries eval("application/x-ruby", "def add(a, b) a + b; end;"); Node.js Rails Truffle Polyglot API JavaScript Interpreter Ruby Interpreter R Interpreter LLVM .bc Interpreter Compiled code snippet Compiled code snippet Truffle API Compiled code snippet Graal compiler Truffle CI Compiled code snippet Compiled code snippet GraalVM Runtime GC / heap Code cache Compiled code snippet Oracle RDBMS Runtime Net Filesystem Threads / Scheduler

Why is GraalVM Different? Multilingual Embeddable Secure Efficient Multilingual Same debugger/profiler/instrumentation/sandboxing for all languages Build a library and spend zero time on providing multiple language bindings Reuse code from ANY language’s libraries that you want wherever you want—improving productivity An environment isn’t truly multilingual if there is a performance cost to cross the language boundary Embeddable Designed to integrate seamlessly with other engines (RDBMS, MySQL, REDIS, …) Manageable extensibility for SaaS Secure Safe & sandboxed execution of native code

GraalVM Language Status Groovy FORTRAN Kotlin Clojure Scala F# Available 2017 Available 2018 Proposed / Prototyping The rest of the languages are not really applicable on a server, or are legacy (except Go)

Most language-level runtimes are bad at multilingualism What are the issues? Language-level runtimes generally designed with a one or two languages in mind The semantics of the original language pollute the VM, so other languages don’t fit well Foreign languages running on a language-level VM generally have a hard time achieving high degrees (99%+) levels of compatibility Performance of multilingual runtimes is often poor outside of the base language JDK Nashorn is 3x – 5x slower than Google V8, Apple Safari Generally a performance penalty for crossing language barriers (e.g. JNI) Running multiple separate VMs means two separate heaps to size, to separate engines to tune & manage, and 10x-1000x higher overhead to cross language boundaries Examples: JVM (Java), Microsoft CLR (C# + Visual Basic), V8 (JavaScript) Compatibility JRuby runs only 95% of standard MRI Ruby Nashorn JS implementation can’t run node.js (most common server JS framework) Crossing costs: * 10x for a function call, 50x for access to foreign object / data structure, 1000x if you are using REST rather than a call

Combining languages in one runtime Multilingual Demo Combining languages in one runtime

Embeddability GraalVM allows any portions of the runtime (memory management, thread scheduling, code cache, etc.) to be supplied by the embedding engine We’ll demonstrate this in tomorrow’s session, “Accelerating Your Database Applications Simply with Graal” (using the Oracle RDBMS & MySQL)

GraalVM is a hybrid of static & dynamic runtimes SubstrateVM provides standalone & embeddable infrastructure to run GraalVM Application code can be precompiled or dynamically loaded via SVM Confidential Oracle Internal/Rstricted/Highly Restricted

Flexibility and security with GraalVM two-layer design Host Language (Java + native) Guest Language (Ruby, JavaScript, Python, R, Java, native) Ahead-of-time compiled (closed world) 100x faster startup 2x lower footprint * Still managed (garbage collected, bounds- checked, & secured against stack overflows) Trusted code should run here Only whitelisted code is guest accessible Reflection is limited to white lists as well Dynamically compiled (open world) Runtime-profiled against the shape of actual data to maximize peak performance Add in new code to running system Flexible, productive (dynamic) Memory-boxed & time-boxed so that totally untrusted code can run here Even native extensions to guest language libraries can be run safely No language-crossing overhead for extensions No code cache, no metadata, no compiler Competition is C code embedding V8 C code is unmanaged and unsecure Interop between C code & JavaScript is unwieldy from performance cost Only JavaScript, and no native extensions allowed in JavaScript

High-Performance for Each Language

Application Level Performance What about real applications, not just benchmarks According to Twitter, Graal runs Twitter 25% faster than Java8 WebLogic Server has 10% better throughput running basic web pages

Multiplicative Value-Add of Graal Ecosystem Your Code Languages * Graal * Embeddings Performance improvements on these benchmarks, on which FastR is already good, were not the focus for this year. But despite all the improvements in completeness, we’ve been able to keep our significant lead in these types of benchmarks. Java JavaScript Ruby R Python C/C++. FORTRAN, … HotSpot JVM MySQL Oracle RDBMS Spark nginx … Optimizations Tooling Interoperability

GraalVM Community Currently we use graal-dev@openjdk.java.net for all external discussions We want to move most traffic off the OpenJDK site other than information related to integration of the Graal compiler with OpenJDK New mailing list: graalvm-users@oss.oracle.com Web: otn.oracle.com/oracle-labs/program-languages/overview/index.html These demos were done with graalvm-0.28 Graal projects on github: github.com/graalvm

Possible University Collaborations with GraalVM One thing academia does better than industry is evaluate developer usability GraalVM’s PolyGlot API allows foreign language objects to be used in other languages in a natural way (e.g. field accesses, etc.) to the extent possible Are there confusing semantics for multilingual interop? Where? Let’s build some applications taking advantages of mixing/matching libraries from multiple languages Truffle makes writing a new language easy & efficient It also solves the biggest problem with domain-specific languages, which is what happens when you reach the edge of your domain Interesting DSL explorations for Truffle & GraalVM?