Runtime Feedback in a Meta-Tracing JIT for Efficient Dynamic Languages Writer: Carl Friedrich Bolz Introduced by Ryotaro IKEDA at 2011/09/06

Overview This paper describes about… How to make it more efficient to apply JIT compiler with PyPy PyPy : Well-known as fast Python implementation. However, in actual, it is one of framework to implement interpreter with JIT and GC! ( Python implementation is just a demo! ) What is PyPy? Framework which enables to write interpreter implementation with Restricted Python The project mainly intends to give environments to implement dynamic interpreter much efficient

PyPys JIT Automatic Implementation Architecture PyPys RPython interpreter Any interpreter that is written in RPython Target code that is written in any language Give some hints to enable to run JIT compiler efficiently Run It is implemented by PyPy user! The most bottom one performs JIT compilation and optimization to the middle one In result, JIT compiler that is suitable for any language is automatically implemented

How to treat non-language-specific JIT compilation Typical JIT Compiler Uses language-specific feature because each JIT compiler is dedicated to compile only one language PyPys JIT Compiler Though it is for RPython, PyPy cant use any language-specific feature which PyPy user want to implement. It is what we called Meta-Tracing How can we make it much faster with applying efficient method for Meta-Tracing?? = Objective

What Merit Using PyPy Rather than JIT of other implementation Widen compilation / optimization area Typical JIT Implementation It is too challenging for JIT compiler to target data structure operation PyPys JIT Implementation PyPys JIT Implementation It traces,and only looks to whole RPython code, so it can target data structure operation which written in RPython by developer.

Hinting Mechanism PyPy RPython Code Hint Main concept Giving hints to enable JIT compiler to compile efficiently is the most important A hint to turn arbitrary variables into constants in the trace by feeding back runtime information into compilation A way to annotate operations which the constant folding opti- mization then recognizes and exploits. General techniques for refactoring code to expose constant folding opportunities of likely runtime constants. MAIN HINTS

PyPys Meta-Tracing JIT Compilers Tracing To check and determine which control path to compile Cond Op x = 100 y = 200 x = x + y Trace (cycles, to be compiled): Cond -> x = x >Cond …. Also constant-folded Cycle: Trace Optimizations are also performed during this trace form

for x in sequence : t = x + …. … PyPys Tracer Trace Area PyPy (can / by default) traces only hot paths. -> Trace will be invoked frequently executed path 1000 CounterIt indicates how many times the loop is executed When it crosses threshold, it is regarded as hot As mentioned before, PyPys tracer doesnt trace user program directly, but interpreter implementation written in RPython instead.

Optimization Passes Remove/simplify operations in the trace – Constant folding – Common subexpression elimination – Allocation removal – Store/load propagation – Loop invariant code motion These can be applied because traces are absolutely linear form Operate during RPython form

Running Example Arrangements of shown examples Simple and bare-bones object model. Just supports classes and instances No inheritance Class contains method and variable Instance have a class, if no requested method / variable found in the instance, it searches among the class.

Example Implementation Use dictionary to manage class method Use dictionary to manage instance attributes(variables/methods) To search requested method To register given method Dictionarys get method costs too much. To solve the problem, it is required to make it target to JIT compilation ( The way to do this is described later discussion )

Hints for Controlling Optimization Two hints that enables to increase the optimization opportunities for constant folding Applied only to interpreter written in RPython, not user program. Promotion Trace-Elidable Enable propagation to find Constant- foldable variables via trace guard Annotation to notify which variables are assumed as constant variable Though each of them never break codes behavior, Using them incorrectly will definitely deteriorate its speed.

What Guard is Dynamic Language test = x + y; That both x and y are number, or string is OK Static Language test = x + y; That both x and y are either number or string, types cannot be canged It is necessary to assure each variables type are same to compile Dynamic Language to Static Language Native code is one of static language, its needed Guard

How Guard works Guard assures that interpreter is running compiled trace in same condition as when it is compiled at first time. y = 10 z = 100 for x in sequence: x = y + z y += 1 …. = func(x) 100 Now it Becomes hot! Source code guard(x == int() ) guard(y = int()) guard(z = 100) x = y + z y += 1 …. = func(x) Trace result Assure conditions to compile them to machine code During execution of compiled machine code… If conditions described in guard is true, it continues to run. If conditions described in guard is false, it stops to run and switch to interpreter exec.

Traced root (will be / already compiled) Normal execution root (interpreter) Promotion Technique to operate constant-fold using guard x = somefunc() y = func(x) Source Code x = somefunc() guard( x == 200) y = func(x) Trace Result PROMOTEPROMOTE x = somefunc() guard( x == 200) y = func(200) Result after Promotion x = somefunc() Trace tree guard(x == 200) y = func(200) TRUE y = func(x) FALSE

Promote how to Later discussion! (Soon!) Use promote() embedded method which is given by PyPy RPython interpreter to give it a hint that indicates promote can be applied during this scope. Assume the trace here usually be with a condition that self and val are expected to not so frequently varied. Guard-fall is expected not so occurs frequently It may not consume overhead so much and can be expected that constant-folding will bring great improvement.

Trace-Elidable helps to apply Promote To tell the truth, promote cannot be invoked annotation in the example. Trace-Elidable: Assure specific method never change any variables. Though tracer want to promote method f, tracer doesnt know whether self.c() returns always same value or not… Tracer considers not to use value-specific guard but type-specific guard… never annotations shows that given method is immutable This hint enables tracer to promote f()!

Result trace after these 2 hints applied Before After COMMON This trace is created without any hints given. Constant-folding is applied and promote.

Technique to increate trace-elidable Putting It All Together Increasing the amount of Trace-elidable method increases chance to apply constant-folding and to help Promote. Prepare original Map class to manage Instances attributes instead of using dictionary To annotations! for index map (described in next slide)

Index map Efficient / Suitable data structure for PyPy Map: To manage data location (index) v1 : 0 string : 1 x : 3 List: Stores actual data 1234Hello,world! …. Prepare getindex with this impelemntation, though it is immutable, trace-elidable can be used!

How does Instance use the Map? This class which is used for manage instances no longer uses dictionary! Whole methods belong to map are trace-elidable. So the promote will work correctly! No longer use dictionary

Versioning of Classes Using only trace-elidable dont satisfy requirements In Python, annotation is given, the method may yield not same value because any attributes can be changed. class A: def __init__(self): x = def X(self): return x inst = A() How do you feel if inst.x = -1 is executed? It is necessary to handle this possible changes They propose Versioning

Use Guard Feature to Versioning Dummy class to use guard feature When some of methods is changed, Yield new VersionTag and save it to self.version This promote helps to create value-specified guard with current version. So, it is still trace-elidable but can handle methods changing.

Evaluations Environment: Intel Core2 Duo P8400 processor with 2.26 GHz and 3072 KB of cache on a machine with 3GB RAM running Linux No hints given Algorithm for board game BZ2 decoder OS Kernel Simulation Decimal floating Point calculations It uses many OOPs features

Conclusions Two hints that can be used in the source code of an interpreter written with PyPy. They give control over runtime feedback and optimization to the language implementor. They are expressive enough for building well- known virtual machine optimization techniques, such as maps and inlining.

Effects to my Study Use PyPy as infrastructure – It can emit C source code from RPython implementation Applying P.T seems easy – Parallelized Template for Rpython This paper performs optimizations in RPython form. How do you think that I consider to implement template code in RPython?