1. Container-based Operating System Virtualization: A scalable, High-performance Alternative to Hypervisors
Stephen Soltesz, Herbert Potzl, Marc E. Fiuczynski, Andy Bavier, Larry Peterson
Stephen Soltesz, Herbert Pötzl, Marc E. Fiuczynski, Andy Bavier, and Larry Peterson Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 (EuroSys '07). ACM, New York, NY, USA, DOI= /

2. Introduction Isolation vs. sharing Workload difference
Operating systems have weak isolation (processes) but significant sharing (pipes, global file-system)
Hypervisors attempt full isolation (VM) but have no sharing (except with virtual networking)
Workload difference
Hypervisors favor isolation over sharing
Significant overhead if each the kernel and systems used between VMs are the same.
Container virtualization approach
Resource containers
Security containers

3. Motivations Virtualization usage cases Specific cases
Development and testing - testing a new kernel without the need to reboot
Server/Hardware consolidation - run multiple database systems on single server
Cloud hosting – host virtual private servers or shared servers
Specific cases
Grid computing
Many users and configurations
Raw performance
Different distributions, but same kernel
Hosting
Similar server software between VMs
Reduce cost per VM

4. Container Virtualization
Trade-off between isolation and efficiency?
Full isolation as a combination of fault isolation, resource isolation and security isolation
Fault Isolation - Limit buggy/malicious VM from affecting state or operation of other guests
Resource Isolation – Account/enforce resource consumption
Physical or logical resource (CPU shares, process limit, PID)
Guarantee vs. best-effort
Security Isolation – Limit access to logical and possibly physical resources
Configuration independence – Global names per VM
Safety – Shared data/code cannot be modified by other VMs

5. Comparison Feature Hypervisor Container Multiple Kernels* ✓ ✗
Root Access
Checkpoint/Resume
Live Migration
Live Updates
* Container systems cannot run multiple kernels since they rely on the kernel as part of the virtualization/isolation mechanism. They also cannot run different operating systems such as Windows
Situations not requiring high-efficiency but not full isolation are better suited for the Container approach.

6. Comparison

7. Hypervisor Architecture

8. Container Architecture
Virtual Platform
Guest VM1 /proc
/home
/usr
/dev
Apache
Guest VMn
/proc
Postgres
Shared OS / Kernel
Host VM
Admin/Services
Hardware

9. Details Security Isolation Resource Isolation (CPU)
Uses OS internals (PIDs, IPC, UIDs, etc.)
Contexts – separation of name spaces
Global OS objects are in separate namespaces
Filters – Access control to objects
Resource Isolation (CPU)
Token bucket filter on top of the standard CPU scheduler
Upper bound for CPU use. Also Work-conserving and/or fair-scheduling
Token rate depends on reservation or share

10. Details Resource Isolation (I/O) Limits
Hierarchical Token Bucket (reserve rate and share)
Packets are tagged with Context and filtered
Also has upper bound, fair-scheduling, and work conservation
Limits
Resident set size, anonymous memory, number of pinned pages
Daemon to reclaim memory

11. Security Isolation Process filtering (Vserver)
Hides processes outside of a specific scope
Mapping from real init to fake init with PID = 1
Difficult to migrate between hosts, but easier to migrate between VMs on the same host
Process Contexts (OpenVZ, LXC)
Separate UID/PID tables for each namespace
Easier to migrate between hosts.
Networking
Shared network subsystem
VMs can only bind to specific addresses
Added filters to the stack to correctly route packets

12. Other Security Chroot barrier Bound on Linux capabilities
Blocks attempts to escape from the chroot confinement
Bound on Linux capabilities
Stops guests VM from gaining unwanted capabilities
Unified file system with copy-on-write
Hard-linked, shared files
Copied when a process attempts to modify shared file

13. Efficiency 2.6.16.33 Kernel with normalized configuration
Some additions required for Vserver (new scheduler)
Vserver rc1
Xen 3.0.4

14. TCP Bandwidth/Utilization

15. Disk Performance

16. Other Performance
CPU performance relatively similar between Xen, Linux, and Vserver
Scaled performance (multiple instances of PostgreSQL)
CPU Scheduling

17. Thoughts Limited comparison and testing
Xen performance due to para-virtualized devices
What about Vmware?
OpenVZ
Limited to specific kernel. Can’t run other systems
Other security concerns
GRSecurity
Good approach for low-power devices or no virtualize able hardware.
Can run on anything that runs Linux.
Not mutually exclusive
Other security benefits