Early Experiences with Developing Sorav Bansal IIT Delhi An Optimizing Virtualization Layer

Virtualization Software VMware Workstation/ESX Server Citrix XenServer Microsoft Hyper-V Virtual Iron Parallels Desktop …

What they do Full-system Emulation Consolidation Sandboxing Live Migration Hardware Fault Tolerance and much more…

How they do, what they do Binary Translation – VMware (1998) Hardware-Assisted Virtualization – VMware, Hyper-V, XenServer, Virtual Iron, … Para-virtualization – XenServer

What they don’t do Optimize code Security Bug-tolerance

What are we doing A virtualization layer for x86 from grounds-up – Runs unmodified OS – Can dynamically optimize code (binary translation) – Can specify security policies enforceable at instruction-level granularity – Can record and replay an execution – Can install on an existing OS – Transparent to user – Simple

Traditional Picture OS Hardware Application 1 Application 2

Virtualized Picture OS Application 1 Application 2 Optimizing VMM

Translation Blocks Divide code into “translation blocks” – A translation block ends if Reach a control-flow instruction Or, MAX_INSNS instructions have been translated

A Simple Scheme Original code fragment Binary Translator x: Translated code fragment tx:

Use a Cache Original code fragment Binary Translator x: Translated code fragment tx: Translation Cache Lookup using xsave found not-found

Direct Jump Chaining a bc d TaTa TbTb TcTc TdTd lookup(b ) lookup(c) lookup(d)

Indirect Jumps a b f call ret TaTa TfTf TbTb lookup(retaddr ) push b jmp T f pop retaddr tmp  JTABLE[retaddr & MASK] if (tmp.src == retaddr) goto tmp.dst

Lower is Better

printf Overheads logarithmic scale

Effect of Maximum Size of Translation Block Max Size of Translation Block

Effect of Translation Cache Size Number of 4k pages in Translation Cache clock random

Optimizations Peephole Optimizations Trace Optimizations Cross-layer optimizations

An Example ld M, r1 ld M, r0 mov r0, r1

Interrupts ld M, r1 ld M, r0 mov r0, r1 Delay Interrupt delivery till end of current translation

Precise Exceptions retld (sp),t0 add $4, sp … jmp t0 Page fault sub $4, sp restore t0 rollback code page fault handler

A Simple Scheme to Prevent Stack-Overflows call ret … push ra, shadow … ra  pop ra1  pop shadow if (ra != ra1) error …

Record-Replay Record – Direct I/O ( in instructions) – Interrupts – Memory-mapped I/O Can use this to tolerate certain classes of bugs

Slowdowns with Record/Replay ProgramSlowdown bubsort216x emptyloop507x euclid320x fibo_iter282x fibo_rec309x hanoi1236x hanoi2182x hanoi3233x printf7x

Conclusions The virtualization layer is a good place to do many interesting things Can we make the virtual machine appear __________________ than the real machine? faster more secure more reliable