The Simplest Heuristics May Be The Best in Java JIT Compilers ACM SIGPLAN 2003 Presented by Mithuna Soundararaj 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivation Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Introduction Simplest strategy in Java JIT compiler is to compile each java method the first time it is called. Better performance if selectively compiled Selectively compiled based on some heuristics like Frequency of method calls Size of the method More complicated heuristics do not provide any additional benefits. 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivation Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Motivation To see how addition of selective compilation heuristics can improve the performance of a particular existing Virtual machine and JIT compiler. To speed up, the slow start up time of many graphical user interfaces,which may lead to users changing their behavior regarding the application. 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivation Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

Architecture of JVM and JIT Java Class Loaded Initialization for JIT usage Invoker Field->JIT compiler When method compiled, JIT translates method’s byte code to native code When method Re Invoked Invoker Field->Generated m/c code Executes machine code or native code 11/8/2018 University of Nebraska, Lincoln

Compiling Methods: Original VS Modified JVM Originally in sun versions of sunwjit Methods are compiled unconditionally the first time they are called Methods interpreted when JIT suppressed by command line JIT internal error while compiling Method belongs to primordial class In the modified version of sunwjit by caldera, Selective compilation is introduced. 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivations Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

Selective compilation Selectively compiling methods based on heuristics like Simple heuristics Based on frequency of method execution Based on size of methods Complicated heuristics “Jit when jitted” heuristic “Square decision” heuristic “Backward branches” heuristic “core classes known to compile” heuristic 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivations Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

Frequency of Method Call To compile methods that are likely to be executed most number of time. Methods compiled many times are likely to be executed many times in the future as well Environment variable set to some positive integer N Method will be compiled after it has executed N times as set in the environment variable 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Method Size Heuristic depends on bytes of byte code for the method Larger method take longer to execute and compile? Compile a small method that’s going to execute only once? Compile large methods that execute many times? 11/8/2018 University of Nebraska, Lincoln

Boundaries of the decision Point where a method size is big enough that you want to compile it Even if it executes once( large methods) Point where method compiled, if it executes a certain number of times. Even if it is very small 11/8/2018 University of Nebraska, Lincoln

Selective Compilation Based on Method size and times executed 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivations Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

Results-Simple heuristics 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Start up time results Startup time is important for psychological reasons(an area ripe for optimization) 77% of execution time java compilation over head, occurs in the initial 10% of program execution 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Profiling results 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Profiling results 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Profiling results 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Outline Introduction Motivation Background Selective Compilation Simple heuristics Results Complicated heuristics Conclusion 11/8/2018 University of Nebraska, Lincoln

Complicated heuristics “Jit when called by jitted “ heuristic Compile method X =>compile every method X calls To take advantage of locality of reference Method X gets used a lot => methods Y, Z etc.that X calls will get used a lot too. Compile Y,Z right away Tarantella object manager is 20% , SwingSet2 is 21% slower and javac is 3% slower to startup and run respectively No test results run faster with it => no locality of reference exists. 11/8/2018 University of Nebraska, Lincoln

Complicated Heuristics “Square decision” heuristic Decision could be either-or: if a method exceeds the minimum size or minimum number of times to be executed, it is compiled Shape of the graph would be a right angle between JIT_MIN_TIMES and JIT_MIN_SIZE SPEC JVM98 is 1% slower and Tarantella Object manager startup is 3% worse than the default scheme This scheme was found to be less flexible than the default scheme, hence not used 11/8/2018 University of Nebraska, Lincoln

Complicated Heuristics Method size in bytes “Square decision” heuristic 11/8/2018 University of Nebraska, Lincoln

Complicated heuristics “backward branches” heuristic Identifies backward branches in methods Assume the branches are loops and hot spots Such methods would be compiled immediately (as in IBM JIT compiler) Implementing this heuristic in SCO/Caldera JIT difficult Similar results to default scheme 11/8/2018 University of Nebraska, Lincoln

Complicated heuristics “core classes known to compile” heuristic Uses data captured by off line profiling Decisions on whether to compile a method immediately- made at the beginning of the application execution Based on whether it is “known that the method will reach its “crossover point” where compilation is sure to be beneficial. Optimizations no more transparent to the user Lot of off-line processing, user visible feedback loop Used efficiently in java core libraries that are used heavily in all applications Javac was 9% faster and SPEC JVM98 was 2% better. 11/8/2018 University of Nebraska, Lincoln

Summary of previous work IBM JIT compiler no of times method executes; modified by detection of loops HP JIT(for embedded ) Minimum execution of 2 times before compilation Krintz and calder used metrics found by experiment 25% of most frequently executed methods must be compiled for optimum performance Use profiling data to identify those 25% methods offline 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Conclusion Adding simple selective compilation heuristics, to an existing Java JIT compiler significantly improves its performance for short and medium-lived applications and for start-up time in longer-lived applications Results have shown that the combined decision strategy is superior to either one or to unconditionally compiling and not using JIT at all 11/8/2018 University of Nebraska, Lincoln

University of Nebraska, Lincoln Thank You! Questions?? 11/8/2018 University of Nebraska, Lincoln