1. The Simplest Heuristics May Be The Best in Java JIT Compilers
ACM SIGPLAN 2003
Presented by
Mithuna Soundararaj
11/8/2018
University of Nebraska, Lincoln

2. University of Nebraska, Lincoln
Outline
Introduction
Motivation
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

3. University of Nebraska, Lincoln
Introduction
Simplest strategy in Java JIT compiler is to
compile each java method the first time it is called.
Better performance if selectively compiled
Selectively compiled based on some heuristics like
Frequency of method calls
Size of the method
More complicated heuristics do not provide any additional benefits.
11/8/2018
University of Nebraska, Lincoln

4. University of Nebraska, Lincoln
Outline
Introduction
Motivation
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

5. University of Nebraska, Lincoln
Motivation
To see how addition of selective compilation heuristics can improve the performance of a particular existing Virtual machine and JIT compiler.
To speed up, the slow start up time of many graphical user interfaces,which may lead to users changing their behavior regarding the application.
11/8/2018
University of Nebraska, Lincoln

6. University of Nebraska, Lincoln
Outline
Introduction
Motivation
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

7. Architecture of JVM and JIT
Java Class Loaded
Initialization for JIT usage
Invoker Field->JIT compiler
When method compiled,
JIT translates method’s
byte code to native code
When method Re Invoked
Invoker Field->Generated m/c code
Executes machine code or native code
11/8/2018
University of Nebraska, Lincoln

8. Compiling Methods: Original VS Modified JVM
Originally in sun versions of sunwjit
Methods are compiled unconditionally the first time they are called
Methods interpreted when
JIT suppressed by command line
JIT internal error while compiling
Method belongs to primordial class
In the modified version of sunwjit by caldera, Selective compilation is introduced.
11/8/2018
University of Nebraska, Lincoln

9. University of Nebraska, Lincoln
Outline
Introduction
Motivations
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

10. Selective compilation
Selectively compiling methods based on heuristics like
Simple heuristics
Based on frequency of method execution
Based on size of methods
Complicated heuristics
“Jit when jitted” heuristic
“Square decision” heuristic
“Backward branches” heuristic
“core classes known to compile” heuristic
11/8/2018
University of Nebraska, Lincoln

11. University of Nebraska, Lincoln
Outline
Introduction
Motivations
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

12. Frequency of Method Call
To compile methods that are likely to be executed most number of time.
Methods compiled many times are likely to be executed many times in the future as well
Environment variable set to some positive
integer N
Method will be compiled after it has executed N times as set in the environment variable
11/8/2018
University of Nebraska, Lincoln

13. University of Nebraska, Lincoln
Method Size
Heuristic depends on bytes of byte code for the method
Larger method take longer to execute and compile?
Compile a small method that’s going to execute only once?
Compile large methods that execute many times?
11/8/2018
University of Nebraska, Lincoln

14. Boundaries of the decision
Point where a method size is big enough that you want to compile it
Even if it executes once( large methods)
Point where method compiled, if it executes a certain number of times.
Even if it is very small
11/8/2018
University of Nebraska, Lincoln

15. Selective Compilation Based on Method size and times executed
11/8/2018
University of Nebraska, Lincoln

16. University of Nebraska, Lincoln
Outline
Introduction
Motivations
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

17. Results-Simple heuristics
11/8/2018
University of Nebraska, Lincoln

18. University of Nebraska, Lincoln
Start up time results
Startup time is important for psychological
reasons(an area ripe for optimization)
77% of execution time java compilation over
head, occurs in the initial 10% of program
execution
11/8/2018
University of Nebraska, Lincoln

19. University of Nebraska, Lincoln
Profiling results
11/8/2018
University of Nebraska, Lincoln

20. University of Nebraska, Lincoln
Profiling results
11/8/2018
University of Nebraska, Lincoln

21. University of Nebraska, Lincoln
Profiling results
11/8/2018
University of Nebraska, Lincoln

22. University of Nebraska, Lincoln
Outline
Introduction
Motivation
Background
Selective Compilation
Simple heuristics
Results
Complicated heuristics
Conclusion
11/8/2018
University of Nebraska, Lincoln

23. Complicated heuristics
“Jit when called by jitted “ heuristic
Compile method X =>compile every method X calls
To take advantage of locality of reference
Method X gets used a lot => methods Y, Z etc.that X calls will get used a lot too.
Compile Y,Z right away
Tarantella object manager is 20% , SwingSet2 is 21% slower and javac is 3% slower to startup and run respectively
No test results run faster with it => no locality of reference exists.
11/8/2018
University of Nebraska, Lincoln

24. Complicated Heuristics
“Square decision” heuristic
Decision could be either-or: if a method exceeds the minimum size or minimum number of times to be executed, it is compiled
Shape of the graph would be a right angle between JIT_MIN_TIMES and JIT_MIN_SIZE
SPEC JVM98 is 1% slower and Tarantella Object manager startup is 3% worse than the default scheme
This scheme was found to be less flexible than the default scheme, hence not used
11/8/2018
University of Nebraska, Lincoln

25. Complicated Heuristics
Method size in bytes
“Square decision” heuristic
11/8/2018
University of Nebraska, Lincoln

26. Complicated heuristics
“backward branches” heuristic
Identifies backward branches in methods
Assume the branches are loops and hot spots
Such methods would be compiled immediately
(as in IBM JIT compiler)
Implementing this heuristic in SCO/Caldera JIT difficult
Similar results to default scheme
11/8/2018
University of Nebraska, Lincoln

27. Complicated heuristics
“core classes known to compile” heuristic
Uses data captured by off line profiling
Decisions on whether to compile a method immediately- made at the beginning of the application execution
Based on whether it is “known that the method will reach its “crossover point” where compilation is sure to be beneficial.
Optimizations no more transparent to the user
Lot of off-line processing, user visible feedback loop
Used efficiently in java core libraries that are used heavily in all applications
Javac was 9% faster and SPEC JVM98 was 2% better.
11/8/2018
University of Nebraska, Lincoln

28. Summary of previous work
IBM JIT compiler
no of times method executes; modified by detection of loops
HP JIT(for embedded )
Minimum execution of 2 times before compilation
Krintz and calder
used metrics found by experiment
25% of most frequently executed methods must be compiled for optimum performance
Use profiling data to identify those 25% methods offline
11/8/2018
University of Nebraska, Lincoln

29. University of Nebraska, Lincoln
Conclusion
Adding simple selective compilation heuristics, to an existing Java JIT compiler significantly improves its performance for short and medium-lived applications and for start-up time in longer-lived applications
Results have shown that the combined decision strategy is superior to either one or to unconditionally compiling and not using JIT at all
11/8/2018
University of Nebraska, Lincoln

30. University of Nebraska, Lincoln
Thank You!
Questions??
11/8/2018
University of Nebraska, Lincoln