1. Alternative Dispatch Techniques for the Tcl VM Benjamin Vitale Mathew Zaleski

2. 2 Tcl VM Dispatch Outline How the VM Interprets Bytecode Dispatch speed on pipelined CPUs The Context Problem Context Threading Results

3. 3 Tcl VM Dispatch Running a Tcl Program Tcl Source BytecodeInterpreter Bytecode Compiler

4. 4 Tcl VM Dispatch Compiling to Bytecode # find first power of 2 # greater than 100 proc find_pow {} { set x 1 while {$x < 100} { incr x $x } return $x } 0push10 # x = 1 2storeScalar10 4pop 5jump1+7 7loadScalar10# x += x 9incrScalar10 11pop 12loadScalar10# if x < 100 14push11#goto 7 16lt 17jumpTrue1-10 19loadScalar10 # return x 21done

5. 5 Tcl VM Dispatch Interpreter push1 0 storeScalar1 0 pop jump1 7 loadScalar1 0 incrScalar1 0 … vpc for (;;) { opcode = * vpc; switch (opcode) { case PUSH1: // real work… vpc += 2; break; case POP: … Bytecode Representation

6. 6 Tcl VM Dispatch Performance Problem Interpreting bytecode is faster than interpreting source But still slow One problem for some VMs is high dispatch overhead How does switch() dispatch work?

7. 7 Tcl VM Dispatch How C compiles switch() push_work: addr6, 4, r6 ldub[r4+1], o0 ld[fp+72], o2 … bra.switch_end pop_work: ld[r2], g1 addr2, -4, r2 movg1, l0 … bra.switch_end &&push_work &&pop_work &&add_work &&sub_work … Code Addresses

8. 8 Tcl VM Dispatch Executing switch() ldubopc = [vpc] // Opcode load (unaligned) cmpopc, max_opc // Bounds check (useless) bgswitch_default setr5 = switch_table // Table lookup (avoidable) mulr1 = r4 * 4 ld[r5 + r1], r1 jmpr1 + r5 // Indirectly jump to work 17 cycles

9. 9 Tcl VM Dispatch Direct Threading ldaddress = [vpc] // Opcode load (aligned) jmpaddress // Indirect jump 12 Cycles portably expressed in Gnu C –we should consider this for Tcl 2 insns in 12 cycles. What is CPU doing?

10. 10 Tcl VM Dispatch CPU Pipeline FDLE… Instruction Cache L2 Cache addr6 += 4 ldr1 = [r4] ldr2 = [fp+8] ldaddr = [vpc] jmpaddr ??? Keeping pipeline full requires pre-fetching. But which instructions?

11. 11 Tcl VM Dispatch Branch Target Predictor 0addr6 += 4 4ldaddr = [r1] 8cmpr6, 12 12bg6 16jmpaddr 20ldr2 = [r3] 24sllr2 = r2, 2 28jmpr2 pc jmp pc target 1642 281000 Branch Target Address Cache Predict branch target from past behavior

12. 12 Tcl VM Dispatch ?  X pc jmp target … Context Problem Example BTAC for switch addprint…push pc jmp target switchpush … pc jmp target switchadd … pc jmp target switchprint … for switch addprint…push for switch addprint… push for switch addprint…push for switch addprint… push for switch addprint…push for switch add print…push for switch addprint…push for swit ch add print …push push2 push3 add print Interpreter Bytecode Program

13. 13 Tcl VM Dispatch Context Problem Hardware is using PC for prediction –Only one branch means one BTAC entry VM is using vpc –branch depends on vpc, has many targets –[Ertl03] 85% mispredicts, costs 10+ cycles How can we avoid misprediction?

14. 14 Tcl VM Dispatch Subroutine Threading Old idea. Great for modern CPUs Correlates native pc with virtual pc 6 cycle dispatch 0push10 2storeScalar10 4pop 5loadScalar10 7incrScalar10 9pop callpush1 callstoreScalar1 callpop callloadScalar1 callincrScalar1 callpop Native Code (“CTT”)Bytecode

15. 15 Tcl VM Dispatch Context Threading Our implementation of subroutine threading CGO’05 Keep bytecode around for operands, etc. Optimizations exploit CTT’s flexibility

16. 16 Tcl VM Dispatch Inlining Small Opcodes callpush push callstoreScalar callpop callincrScalar callpop pop incrScalar storeScalar push

17. 17 Tcl VM Dispatch Virtual Branches become Native jump becomes two native instructions jumpTrue uses native branch, but also calls 0jump1+7 2loadScalar10 4incrScalar10 6pop 7loadScalar10 9push11 11lt 12jumpTrue1-10 14loadScalar10 0b16 4setvpc = 7 8callloadScalar 12callincrScalar 16callpop 20callloadScalar1 24callpush1 28calllt 32calljumpTrue1 36beq8 40callloadScalar1

18. 18 Tcl VM Dispatch Conditional Branch Peephole Opt We can eliminate the call to jumpTrue –Profile: what precedes cond. branches? gt, lt, tryConvertNumeric, foreachStep4 –move the branch code into CTT and gt Tcl 8.5 has a similar optimization, but bigger payoff for native Loops go faster

19. 19 Tcl VM Dispatch Cond. Branch Peephole Opt Demo gt: c = do_compare… o = new_bool (c) push (o) vpc++ return jump_true: 91 insns o = pop () coerce_bool (o) if (o.bool) vpc = target v else vpc = fall_thru v asm (“cmp o.bool, 0”) return gt _ jump: c = do_compare… vpc++ return gt _ jump: c = do_compare… asm (“cmp o.bool, 0”) vpc++ return callgt calljumpTrue beqtarget n callgt_jump beqtarget n callgt _ jump setvpc = target v beqtarget n setvpc = fall_thru v callgt calljumpTrue beqtarget n

20. 20 Tcl VM Dispatch Catenation IVME ’04 Inline everything –Specialize operands –Eliminate vpc Complicated 0 cycle dispatch pop storeScalar push add

21. 21 Tcl VM Dispatch Results Tclbench –microbenchmarks, only 12 with more than 100,000 dispatches –de-facto standard –focus on 60 with > 10,000 dispatches UltraSPARC III Use switch interpreter as baseline

22. 22 Tcl VM Dispatch Performance Summary Dispatch type Geo. mean speedup Number of benchmarks improved Direct Threading 4.3%88% Catenation4.073 Context Threading 5.488 CT + peephole 12.097

23. Tclbench Speedup versus Switch 23

24. Performance Details

25. 25 Tcl VM Dispatch Tcl Opcodes are Big Java25% Ocaml37% Tcl5% Context Threading Speedup

26. 26 Tcl VM Dispatch Conclusions and Future Work Context Threading is simple & effective –fast dispatch (not Tcl’s problem) –facilitates optimization –inline more opcodes, port to x86, PowerPC 12% speedup trivial: Tcl 10x slower than C –micro opcodes and a real JIT

27. 27 Tcl VM Dispatch Low Dispatch Overhead Tcl Opcodes DISPATCH Ocaml Opcodes

28. 28 Tcl VM Dispatch Branch prediction on Sparc Ultra 1 had “NFA” in I-cache UltraSPARC III –What kind of branch target predictor? –“prepare-to-branch” instruction? –Consider two virtual programs, on the next slide:

29. 29 Tcl VM Dispatch Jekyll and Hyde Programs start:Vop1 Vop2 Vop1 Vop3 Vop1 Vop4 Vop1 Vop5 Vop1 gotostart Unpredictable start:Vop1 Vop1 gotostart Predictable

30. 30 Tcl VM Dispatch Mispred vs. predict Dispatch TypeUltraSPARC IIIPentium 4 switch17.319.2 indirect mispred14.218.6 indirect predict14.211.8 direct mispred11.218.7 direct predict11.211.3 subroutine6.38.4

31. 31 Tcl VM Dispatch CT, Tcl, Sparc Branch Delay Slot Big Tcl bodies nearly all contain calls –Calls clobber link register ( o7 ) –We save link register in a reserved reg callbigop movo7, save_ret … bigop: callruntime … movsave_ret, o7 retl incvpc, 4

32. 32 Tcl VM Dispatch Compilation Time We include compile time in every iteration Tclbench amortizes Dispatch Type Native Compile time relative to ByteCode Direct Threading 6% Catenation44 Context Threading 35 CT + peephole38 Varies significantly across benchmarks