1. Dynamic Compilation and Modification CS 671 April 15, 2008

2. CS 671 – Spring 2008 1 So Far… Static Compilation Compiler High-Level Programming Languages Machine Code Error Messages High-Level Programming Languages Machine Code Front End Back End Error Messages Compiler Digging Deeper…

3. CS 671 – Spring 2008 2 Alternatives to the Traditional Model Static Compilation All work is done “ahead-of-time” Just-in-Time Compilation Postpone some compilation tasks Multiversioning and Dynamic Feedback Include multiple options in binary Dynamic Binary Optimization Traditional compilation model Executables can adapt

4. CS 671 – Spring 2008 3 Move More of Compilation to Run Time Execution environment may be quite different from the assumptions made at compile time Dynamically loaded libraries User inputs Hardware configurations Dependence on software vendors Apps on tap Incorporate profiling

5. CS 671 – Spring 2008 4 Just-in-Time Compilation High-Level Programming Languages Machine Code Front End Back End Error Messages Ship bytecodes (think IR) rather than binaries Binaries execute on machines Bytecodes execute on virtual machines Compiler

6. CS 671 – Spring 2008 5 Just-in-Time Compilation javac the Java bytecode compiler java the Java virtual machine Bytecode: machine independent, portable Step One: “Compile” Circle.java % javac Circle.java -> Circle.class Step Two: “Execute” % java Circle.class javac sourcebytecode java bytecodeexecute

7. CS 671 – Spring 2008 6 Each frame contains local variables and an operand stack Instruction set Load/store between locals and operand stack Arithmetic on operand stack Object creation and method invocation Array/field accesses Control transfers and exceptions The type of the operand stack at each program point is known at compile time Bytecodes

8. CS 671 – Spring 2008 7 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) Bytecodes (cont.)

9. CS 671 – Spring 2008 8 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 Bytecodes (cont.)

10. CS 671 – Spring 2008 9 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 2 Bytecodes (cont.)

11. CS 671 – Spring 2008 10 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 2 Bytecodes (cont.)

12. CS 671 – Spring 2008 11 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 7 2 Bytecodes (cont.)

13. CS 671 – Spring 2008 12 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 49 2 Bytecodes (cont.)

14. CS 671 – Spring 2008 13 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 0 98 Bytecodes (cont.)

15. CS 671 – Spring 2008 14 Example: iconst 2 iload a iload b iadd imul istore c Computes: c := 2 * (a + b) a b c 42 7 98 Bytecodes (cont.)

16. CS 671 – Spring 2008 15 Executing Bytecode java Circle.class - What happens? Interpreting map each bytecode to a machine code sequence, for each bytecode, execute the sequence Translation to machine code map all the bytecodes to machine code (or a higher level intermediate representation) massage them (e.g., remove redundancies) execute the machine code

17. CS 671 – Spring 2008 16 Hotspot Compilation A hybrid approach Initially interpret Find the “hot” (frequently executed) methods Translate only hot methods to machine code

18. CS 671 – Spring 2008 17 The Virtual Machine An extreme version of an old idea Previously: Now: MyApp x86 P IIIP IV MyApp alpha 2116421264 MyApp pa-risc PA-8000PA-7000 MyApp JVM P IIIP IV2116421264PA-8000PA-7000 VM

19. CS 671 – Spring 2008 18 Compile-Time Multiversioning Multiple versions of code sections are generated at compile-time Most appropriate variant is selected at runtime based upon characteristics of the input data and/or machine environment Multiple variants can cause code explosion –Thus typically only a few versions are created

20. CS 671 – Spring 2008 19 Another Alternative Modify a traditional application as it executes Why? Don’t have source code! ???? binary modified binary

21. CS 671 – Spring 2008 20 A Dynamic Optimization System? Transforms* an application at run time * {translate, optimize, extend} Application Transform Code Cache Execute Profile

22. CS 671 – Spring 2008 21 Classification Dynamic binary optimizers (x86  x86opt) Complement the static compiler –User inputs, phases, DLLs, hardware features –Examples: DynamoRIO, Mojo, Strata Dynamic translators (x86  PPC) Convert applications to run on a new architecture –Examples: Rosetta, Transmeta CMS, DAISY Binary instrumentation (x86  x86instr) Inspect and/or add features to existing applications –Examples: Pin, Valgrind JITs + adaptive systems (Java bytecode  x86)

23. CS 671 – Spring 2008 22 Dynamic Instrumentation Demo Pin Four architectures – IA32, EM64T, IPF, XScale Four OSes – Linux, FreeBSD, MacOS, Windows

24. CS 671 – Spring 2008 23 What is Instrumentation? A technique that inserts extra code into a program to collect runtime information Instrumentation approaches: Source instrumentation: –Instrument source programs Binary instrumentation: –Instrument executables directly

25. CS 671 – Spring 2008 24 No need to recompile or relink Discover code at runtime Handle dynamically-generated code Attach to running processes Why use Dynamic Instrumentation?

26. CS 671 – Spring 2008 25 How is Instrumentation used in PL/Compiler Research? Program analysis –Code coverage –Call-graph generation –Memory-leak detection –Instruction profiling Thread analysis –Thread profiling –Race detection

27. CS 671 – Spring 2008 26 Trace Generation Branch Predictor and Cache Modeling Fault Tolerance Studies Emulating Speculation Emulating New Instructions How is Instrumentation used in Computer Architecture Research?

28. CS 671 – Spring 2008 27 Pin Features Dynamic Instrumentation: Do not need source code, recompilation, post-linking Programmable Instrumentation: Provides rich APIs to write in C/C++ your own instrumentation tools (called Pintools) Multiplatform: Supports x86, x86-64, Itanium, Xscale Supports Linux, Windows, MacOS Robust: Instruments real-life applications: Database, web browsers, … Instruments multithreaded applications Supports signals Efficient: Applies compiler optimizations on instrumentation code

29. CS 671 – Spring 2008 28 Using Pin Launch and instrument an application $ pin –t pintool –- application Instrumentation engine (provided in the kit) Instrumentation tool (write your own, or use one provided in the kit) Attach to and instrument an application $ pin –t pintool –pid 1234

30. CS 671 – Spring 2008 29 Pin Instrumentation APIs Basic APIs are architecture independent: Provide common functionalities like determining: –Control-flow changes –Memory accesses Architecture-specific APIs e.g., Info about segmentation registers on IA32 Call-based APIs: Instrumentation routines Analysis routines

31. CS 671 – Spring 2008 30 Instrumentation vs. Analysis Instrumentation routines define where instrumentation is inserted e.g., before instruction  Occurs first time an instruction is executed Analysis routines define what to do when instrumentation is activated e.g., increment counter  Occurs every time an instruction is executed

32. CS 671 – Spring 2008 31 Pintool 1: Instruction Count sub$0xff, %edx cmp%esi, %edx jle mov$0x1, %edi add$0x10, %eax counter++;

33. CS 671 – Spring 2008 32 Pintool 1: Instruction Count Output $ /bin/ls Makefile imageload.out itrace proccount imageload inscount0 atrace itrace.out $ pin -t inscount0.so -- /bin/ls Makefile imageload.out itrace proccount imageload inscount0 atrace itrace.out Count 422838

34. CS 671 – Spring 2008 33 ManualExamples/inscount0.cpp instrumentation routine analysis routine #include #include "pin.h" UINT64 icount = 0; void docount() { icount++; } void Instruction(INS ins, void *v) { INS_InsertCall(ins, IPOINT_BEFORE, (AFUNPTR)docount, IARG_END); } void Fini(INT32 code, void *v) { std::cerr << "Count " << icount << endl; } int main(int argc, char * argv[]) { PIN_Init(argc, argv); INS_AddInstrumentFunction(Instruction, 0); PIN_AddFiniFunction(Fini, 0); PIN_StartProgram(); return 0; }

35. CS 671 – Spring 2008 34 Pintool 2: Instruction Trace sub$0xff, %edx cmp%esi, %edx jle mov$0x1, %edi add$0x10, %eax Print(ip); Need to pass ip argument to the analysis routine (printip())

36. CS 671 – Spring 2008 35 Pintool 2: Instruction Trace Output $ pin -t itrace.so -- /bin/ls Makefile imageload.out itrace proccount imageload inscount0 atrace itrace.out $ head -4 itrace.out 0x40001e90 0x40001e91 0x40001ee4 0x40001ee5

37. CS 671 – Spring 2008 36 ManualExamples/itrace.cpp argument to analysis routine analysis routine instrumentation routine #include #include "pin.H" FILE * trace; void printip(void *ip) { fprintf(trace, "%p\n", ip); } void Instruction(INS ins, void *v) { INS_InsertCall(ins, IPOINT_BEFORE, (AFUNPTR)printip, IARG_INST_PTR, IARG_END); } void Fini(INT32 code, void *v) { fclose(trace); } int main(int argc, char * argv[]) { trace = fopen("itrace.out", "w"); PIN_Init(argc, argv); INS_AddInstrumentFunction(Instruction, 0); PIN_AddFiniFunction(Fini, 0); PIN_StartProgram(); return 0; }

38. CS 671 – Spring 2008 37 Examples of Arguments to Analysis Routine IARG_INST_PTR Instruction pointer (program counter) value IARG_UINT32 An integer value IARG_REG_VALUE Value of the register specified IARG_BRANCH_TARGET_ADDR Target address of the branch instrumented IARG_MEMORY_READ_EA Effective address of a memory read And many more … (refer to the Pin manual for details)

39. CS 671 – Spring 2008 38 Instrumentation Points Instrument points relative to an instruction: Before (IPOINT_BEFORE) After: –Fall-through edge (IPOINT_AFTER) –Taken edge (IPOINT_TAKEN_BRANCH) cmp%esi, %edx jle mov$0x1, %edi : mov $0x8,%edi count()

40. CS 671 – Spring 2008 39 Instruction Basic block –A sequence of instructions terminated at a control-flow changing instruction –Single entry, single exit Trace –A sequence of basic blocks terminated at an unconditional control-flow changing instruction –Single entry, multiple exits Instrumentation Granularity sub$0xff, %edx cmp%esi, %edx jle mov$0x1, %edi add$0x10, %eax jmp 1 Trace, 2 BBs, 6 insts Instrumentation can be done at three different granularities:

41. CS 671 – Spring 2008 40 Pintool 3: Faster Instruction Count sub$0xff, %edx cmp%esi, %edx jle mov$0x1, %edi add$0x10, %eax counter += 3 counter += 2 basic blocks (bbl)

42. CS 671 – Spring 2008 41 Modifying Program Behavior Pin allows you not only to observe but also change program behavior Ways to change program behavior: Add/delete instructions Change register values Change memory values Change control flow

43. Pin Internals

44. CS 671 – Spring 2008 43 Pin’s Software Architecture JIT Compiler Emulation Unit Virtual Machine (VM) Code Cache Instrumentation APIs Application Operating System Hardware Pin Pintool Address space

45. CS 671 – Spring 2008 44 Dynamic Instrumentation Original code Code cache Pin fetches trace starting block 1 and start instrumentation 7’ 2’ 1’ Pin 2 3 1 7 4 5 6 Exits point back to Pin

46. CS 671 – Spring 2008 45 Dynamic Instrumentation Original code Code cache Pin transfers control into code cache (block 1) 2 3 1 7 4 5 6 7’ 2’ 1’ Pin

47. CS 671 – Spring 2008 46 Dynamic Instrumentation Original code Code cache 7’ 2’ 1’ Pin Pin fetches and instrument a new trace 6’ 5’ 3’ trace linking 2 3 1 7 4 5 6

48. CS 671 – Spring 2008 47 Implementation Challenges Linking –Straightforward for direct branches –Tricky for indirects, invalidations Re-allocating registers Maintaining transparency Self-modifying code Supporting MT applications…

49. CS 671 – Spring 2008 48 Thread-safe accesses Pin, Pintool, and App –Pin: One thread in the VM at a time –Pintool: Locks, ThreadID, event notification –App: Thread-local spill area Providing pthreads functions to instrumentation tools Pin’s Multithreading Support Pintool Application System’s libpthread signalhandler signalhandler set up signal handlers Pin’s mini-libpthread Redirect all other pthreads function calls to application’s libpthread

50. CS 671 – Spring 2008 49 Pin Overhead SPEC Integer 2006

51. CS 671 – Spring 2008 50 Adding User Instrumentation

52. CS 671 – Spring 2008 51 Dynamic Optimization Summary Complement the static compiler Shouldn’t compete with static compilers Observe execution pattern Optimize frequently executed code –Optimization overhead could degrade performance Exploits opportunities Arise only at runtime –DLLs –Runtime constants –Hardware features, user patterns, etc. Too expensive to fully exploit statically –Path-sensitive optimizations