1. Georgia Tech November 2006 J. E. Smith
Virtual Machines: Supporting Changing Technology and New Applications
Georgia Tech
November 2006
J. E. Smith

2. Introduction Why virtual machines?
They allow transcending of standardized interfaces
(which sometimes are an obstacle to innovation)
They enable innovation in flexible, adaptive software & hardware, security, network computing (and others)
They involve computer architecture in a pure sense
Virtualization technologies will be a key part of most future computer systems
VMs (c) 2006, J. E. Smith

3. Outline Virtualization Virtual Machine Architecture
Virtual Machine Implementation
Computer Architecture Applications
Co-Designed VMs
Private Virtual Machines
VMs (c) 2006, J. E. Smith

4. Abstraction Computer systems are built on levels of abstraction
file
abstraction
Higher level of abstraction hide details at lower levels
Example: files are an abstraction of a disk
VMs (c) 2006, J. E. Smith

5. Virtualization Similar to abstraction Construct Virtual Disks
Except
Same level of detail
Construct Virtual Disks
As files on a larger disk
Map state
Map operations
VMs: do the same thing with the whole “machine”
Key concepts: Map state; Map Operations
file
virtualization
VMs (c) 2006, J. E. Smith

6. The Family of Virtual Machines
Including things not called “virtual machines”
IA-32 EL
HP Dynamo
Transmeta Crusoe
There are lots of “virtual machines”
IBM VM/370
Java
VMware products
“The subjects of virtual machines and emulators have been treated as entirely separate. … they have much in common. Not only do the usual implementations have many shared characteristics, but this commonality extends to the theoretical concepts on which they are based”
-- Efrem G. Wallach, 1973
VMs (c) 2006, J. E. Smith

7. “Machines” Different perspectives on what the Machine is: OS developer
Compiler developer
Application programmer
Instruction Set Architecture
ISA
Major division between hardware and software
Application
Programs
Libraries
Operating System
Execution Hardware
Application Binary Interface
ABI
User ISA + OS calls
Application Program Interface
API
User ISA + library calls
Memory
Translation
System Interconnect
(bus)
I/O devices
Main
and
Memory
Networking
VMs (c) 2006, J. E. Smith

8. System Virtual Machines
ISA level
Provide a system environment
VMM manages guest OS + apps
Persistent
Examples: IBM VM/360, VMware, Transmeta Crusoe
guest
guest
guest
guest
guest
guest
process
process
process
process
process
process
Guest OS
Guest OS2
VMM
VMM
HOST PLATFORM
virtual
network communication
VMs (c) 2006, J. E. Smith

9. Process Virtual Machines
ABI level
Runtime manages guest process
Guest processes may intermingle with host processes
Not persistent
Guest and host OSes are often the same
Dynamic optimizers are a special case
Examples: IA-32 EL, FX!32, Dynamo
host
guest
process
process
runtime
guest
guest
process
host
process
process
runtime
runtime
create
HOST OS
file sharing
Disk
network communication
VMs (c) 2006, J. E. Smith

10. High Level Language Virtual Machines
Constructed at API level
User higher level virtual ISA
OS abstracted as standard libraries
A form of process VM
HLL Program
Intermediate Code
Memory Image
Object Code (
ISA )
Compiler front-end
Compiler back-end
Loader
Portable Code
Virtual ISA
Host Instructions
Virt. Mem. Image
Compiler
VM loader
VM Interpreter/Translator
Traditional
HLL VM
VMs (c) 2006, J. E. Smith

11. Virtual Machine Architectures
Process VMs
System VMs
different
different
same ISA
same ISA
ISA
ISA
Multi
Dynamic
Classic
Whole
programmed
Translators
OS VMs
System VMs
Systems
Dynamic
Hosted
Co-Designed
HLL VMs
Binary
Optimizers
VMs
VMs
VMs (c) 2006, J. E. Smith

12. VM Technology – State Mapping
VM SW re-maps virtual state to real state
Recall virtual disk
Registers to registers
Registers to memory
Memory to memory
Memory to disk
Host
Host Registers
Register
Space
Guest Registers
Runtime
Data
Runtime
Code
Host ABI
Guest Data
Address
Space
Guest Code
VMs (c) 2006, J. E. Smith

13. VM Technology – Operation Mapping
VM SW re-maps operations on state
Instruction-level state changes
Emulation
Protected state changes
OS operations
Done under VMM control
Key concepts: Emulation and Control
VMs (c) 2006, J. E. Smith

14. VM Technology – Emulation
Interpretation
Software loop decodes and dispatches each instruction
interpreter
source code
routines
"data"
accesses
dispatch
loop
VMs (c) 2006, J. E. Smith

15. VM Technology – Emulation
Binary translation and code caching
Translate blocks of instructions at a time
Hold translated blocks in code cache
This was a key enabler for VMware success
binary translated
target code
source code
binary
translator
VMs (c) 2006, J. E. Smith

16. VM Technology – Emulation
Staged Emulation
Emulation techniques invoked in staged manner
Based on performance tradeoffs
Binary Memory
Image
Profile Data
Interpreter
Emulation
manager
Code Cache
Translator/
Optimizer
VMs (c) 2006, J. E. Smith

17. Code Caches Contain Used in many VMs Basic blocks
Superblocks (one entrance, multiple exits)
Optimized Superblocks
Used in many VMs
Dynamic binary translators: Intel IA-32 EL, Compaq FX!32
Dynamic binary optimizers: Dynamo family
Co-designed virtual machines: Transmeta, IBM DAISY
High performance Java virtual machines
System VMs with “inefficiently virtualizable” ISAs
“Sandboxing” secure VMs (x86 DynamoRIO)
VMs (c) 2006, J. E. Smith

18. Code Caching with Chaining
Chaining of blocks in code cache minimizes VM overhead
Code Cache
Superblock
Dispatch table lookup code
Superblock
Superblock
Superblock
VMs (c) 2006, J. E. Smith

19. VM Technology – Control
Interpretation
Fine grain control
Every dynamic instruction “inspected” before execution
Binary translation and code caching
Coarser grain control
Every static instruction inspected before execution
Jumps to VM SW can be inserted anywhere
Protection levels
Very coarse grain control
Every resource-related instruction trapped by protection system
Otherwise, use interpretation/translation techniques
Used in system VMs
VMs (c) 2006, J. E. Smith

20. Resource Control in System VMs
Application
Traps and interrupts (& sys calls)
Transfer to VMM
VMM determines appropriate Guest OS
VMM transfers to Guest OS
Guest OS “return” to user app.
VMM bounces return back to Guest app.
Resource sensitive instructions
Trap to VMM
VMM checks correctness
VMM reads/modifies guest resource
Returns to Guest
system call/trap
Guest OS
privileged operation
next instruction
virtual vector location:
system return
Figs7/tramplne
VMM
check privileges
perform operation
return
vector location:
VMs (c) 2006, J. E. Smith

21. VMs and Computer Architecture
Use virtualization to give computer architects a layer of software
Beneath all conventional software
Maintains vision of hardware as seen by conventional software
Performance optimizations via Co-designed VMs
VM SW can alter/enhance architecture via emulation
Resource management – Private Virtual Machines
VM SW can manage microarchitecture resources
VMs (c) 2006, J. E. Smith

22. Co-Designed Virtual Machines
Separate the hardware/software interface from the ISA level of abstraction
Restore the ISA to its “natural” place
 as an Implementation ISA that reflects actual hardware
Support existing ISAs
 as a Virtual ISA
Let processor designers use both
hardware and software
A form of system VM
Hardware OS
libs.
User Applications
ISA OS
libs.
User Applications
V-ISA
I-ISA
Hardware
Software
VMs (c) 2006, J. E. Smith

23. VM Technology -- Concealed Memory
VM software resides in memory concealed from all conventional software
This software is available to hardware designer
Code
Cache
ICache
concealed memory
Hierarchy
VM Code
VM Data
Processor
Core
Source ISA Code
DCache
conventional
Hierarchy
memory
Source ISA Data
VMs (c) 2006, J. E. Smith

24. Co-Designed VMs Of interest to both architects and micro-architects
Offers opportunities for performance, power saving, fault tolerance and other implementation-dependent features
Allows transcending conventional ISAs
Don’t confuse them with VLIW!
Early examples: IBM Daisy and Transmeta Crusoe
“pioneers are the ones with arrows in their backs”
VMs (c) 2006, J. E. Smith

25. Another Way of Doing Things
conventional
Func.
Unit
Translation
Func.
Cache
Processor
Main Memory
Unit
Unit
Hierarchy
Pipeline
(form uops)
. ..
Func.
Unit
Main Memory
dynamic translation
Software
Translator
Func.
Unit
. ..
Cache
Processor
Code Cache
Translation
Hierarchy
Pipeline
Unit
(form uops)
Func.
Unit
VMs (c) 2006, J. E. Smith

26. Fused Microarchitecture
Fuse dependent pairs of micro-ops to macro-ops
Current Intel approach
Use co-designed SW to achieve wider-scale fusing
Process & execute fused macro-ops as single Instructions throughout the entire pipeline
Allows pipelined wake-up/select issue logic
Overview of macro-op execution
VMs (c) 2006, J. E. Smith

27. Fusible Instruction Set
RISC-ops with unique features:
Fuse bit per instruction fuses two dependent instructions
Dense instruction encoding, 16/32-bit ISA design
Special Features to Support the x86 ISA
Condition codes
Addressing modes
Aware of long immediate & displacement values
Core 32 -
bit instruction formats F 10
b opcode 21
bit Immediate /
Displacement F 10
b opcode 16
bit immediate /
Displacement 5
b Rds F 10
b opcode 11
b Immediate /
Disp 5
b Rsrc 5
b Rds F 16
bit opcode 5
b Rsrc 5
b Rsrc 5
b Rds
Add - on 16 -
bit instruction formats for code density F
Processor Architecture 5
b op 10
b Immd /
Disp F 5
b op 5
b Rsrc 5
b Rds F 5
b op 5
b Rsrc 5
b Rds
Fusible ISA Instruction Formats
VMs (c) 2006, J. E. Smith

28. Percentage of Dynamic Instructions
Fusing Profile
About 50% of operations are fused
Only 5-10% of non-fused are single-cycle ALU ops 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
164.gzip
175.vpr
176.gcc
181.mcf
186.crafty
197.parser
252.eon
253.perlbmk
254.gap
255.vortex
256.bzip2
300.twolf
Average
Percentage of Dynamic Instructions
ALU
FP or NOPs BR ST LD
Fused
VMs (c) 2006, J. E. Smith

29. Nomarlized IPC speedup (%)
Performance
Base + Code Cache
+ fusing
+ shorter pipe
+ 3-1 ALU 70 60 50 40
Nomarlized IPC speedup (%) 30 20 10
-10
164.gzip
175.vpr
176.gcc
181.mcf
186.crafty
252.eon
254.gap
197.parser
255.vortex
256.bzip2
300.twolf
253.perlbmk
Harmonic
VMs (c) 2006, J. E. Smith

30. Virtual Private Machines
Multi-core systems will have many hardware-level shared resources
Multi-threaded processors
Multi-level shared caches
Shared memory ports
Spares for fault tolerance
And a number of important implementation dependences
Non-uniform memory delays
Power optimization features
Fault tolerance features
VMs (c) 2006, J. E. Smith

31. Virtual Private Machines
Co-design a MicroVisor to provide software with Virtual Private Machines
Insulates conventional software from complicated implementation-dependent features
Provides performance virtualization
Unlike classic VMs
Quality of Service (QoS)
Performance Isolation
System
System VM VM
Conventional OS
Functional
Hypervisor (VMM)
Virtualization
VPM
VPM
Performance
MicroVisor
Virtualization
Multi-Core Hardware
VMs (c) 2006, J. E. Smith

32. MicroVisor Virtualizes performance, not functionality
Co-Designed software to support hardware resource management
Concealed/isolated from all conventional software
Much larger than microcode, does not consume processor chip real estate
Uses conventional instructions (extended) so there is more likelihood of some cross-system portability
Data
μV Code
ICache
Hierarchy
DCache
Processor
Core
Code
μV Data
concealed memory
conventional
memory
VMs (c) 2006, J. E. Smith

33. “Real-izing” Processors/Memory
Separate Real Processors from Physical Processors
OS assigns processes to Real Processors
MicroVisor maps real processors to physical processors
MicroVisor also maps real memory to physical memory
Extend to cache memories
Processes
Real
Processors
Physical
OS Maps
MicroVisor Maps
VMs (c) 2006, J. E. Smith

34. Virtual Private Machines
MicroVisor maps high level requirements to hardware configuration
Requires mechanisms to provide microarchitecture level QoS
Main Memory
Main Memory
Memory Controller
Mem
Mem
Mem
Mem
Controller
Controller
Controller
Controller
L2 Cache
L2 Cache
L2 Cache
L2 Cache
L2 Cache
Interconnection Net
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
Proc. 0
Proc. 1
Proc. 0
Proc. 0
Proc. 0
Proc. 0
Threads
Threads
Thread 0
Thread 0
Thread 0
Thread 0
0 & 1
2 & 3
VPM 0
VPM 1
VPM 2
VPM 3
VMs (c) 2006, J. E. Smith

35. Applications Performance optimization Power management Fault tolerance
Deal with NUMA
Provide QoS and performance isolation in multi-threaded systems
Power management
Adjust resources to match power constraints
Requires inferring demand for resources
in contrast to conventional OS
Fault tolerance
Detected fault triggers MicroVisor
Diagnose, reconfigure, re-map memory/processors
VMs (c) 2006, J. E. Smith

36. Summary Many types of VMs An important system component
But common virtualization technologies
An important system component
Should be studied/taught as a discipline on its own
Alongside OS, Application SW, HW
Many avenues for computer architecture research
Co-designed VMs
Virtual Private Machines
Adaptive microarchitecture
Fault-tolerant implementations
Primitives for supporting efficient VMs
VMs (c) 2006, J. E. Smith