1. Presenter: Yifan Zhang
Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems
Mike Chen Tal Garfinkel E. Christopher Lewis Pratap Subrahmanyam Carl A. Waldspurger VMware, Inc.
Dan Boneh Jeffrey Dwoskin Dan R.K. Ports Stanford Princeton MIT
ASPLOS 2008
Presenter: Yifan Zhang
The slides are adapted from the conference slides provided by the authors.

2. Motivation Applications handle sensitive data
Financial, medical, insurance, military …
Security provided by commodity OS is inadequate.
Large and complex TCB, broad attack surfaces
OS kernel, drivers, daemons, services …
Hard to configure, manage, maintain
Privilege escalation  game over
Problem: how to run applications securely (i.e., protect data of applications) on untrusted OS?
Copyright © 2008 VMware, Inc. All rights reserved.

3. Limitations of Existing Solutions
Rewrite OS / Applications
Split into low- and high-assurance portions e.g. microkernels, Microsoft Palladium/NGSCB
Expensive, high barriers to adoption
Multiple Virtual Machines
Trusted/untrusted or specialized VMs (e.g. Proxos, Terra)
Cumbersome, still vulnerable to OS compromise
Hardware Approaches
Special-purpose secure co-processors (e.g. IBM 4758)
XOM processor architectures
Require substantial modifications to hardware/OS/apps
Copyright © 2008 VMware, Inc. All rights reserved.

4. Goals 1. Protect Application Data 2. Remove OS from TCB
Privacy: only the owner of the data can access them
Integrity: identify if application data has been tampered with between accesses of the owner
2. Remove OS from TCB
Even if attacker compromises guest OS
Methods:
Data privacy and shrink TCB: technique called “cloaking” – present an application with a cleartext view of its pages, and OS with an encrypted view.
Data integrity: attach hash values of each cloaked resource (e.g., a file or a memory region) as meta-data
Copyright © 2008 VMware, Inc. All rights reserved.

5. Goals (continued) 3. Backwards Compatibility Unmodified commodity OS
Unmodified application binary
Method:
Introduce a shim into each cloaked application’s address space.
Shim
is a OS-specific user-level program;
is the key component in identifying context;
mediates all communications between applications/OS.
Copyright © 2008 VMware, Inc. All rights reserved.

6. Background: Virtual Memory
Traditional OS Approach
Level of Indirection
Virtual  Physical
OS page table maps VPN (virtual page number) to PPN (physical page number)
Cached by hardware TLB
VPN
PPN
hardware TLB OS
page
table
Copyright © 2008 VMware, Inc. All rights reserved.

7. Background: Classical Memory Virtualization
Traditional VMM Approach
Extra Level of Indirection
Virtual  Physical Guest OS page table maps GVPN (virtual page number) to GPPN (physical page number)
Physical  Machine VMM maps GPPN to MPN
Shadow Page Table
Composite of two mappings
Directly maps GVPN to MPN
Cached by hardware TLB
GVPN
GPPN
MPN
hardware TLB
shadow
page table
guest
VMM
Copyright © 2008 VMware, Inc. All rights reserved.

8. Background: Multi-Shadowing
New Way to Leverage VMM
Multiple Views of Memory
GPPN maps to multiple MPNs
Using multiple shadow page tables
View depends on “context” accessing page
General Mechanism
Orthogonal to protection domains defined by OS and processor (e.g., protection rings, page tables)
Enables new protection schemes
GVPN
GPPN
shadow context 2
MPN1
MPN2
shadow context 1
Copyright © 2008 VMware, Inc. All rights reserved.

9. Memory Cloaking
Cloaking is a combination of multi-shadowing and encryption/decryption.
In Overshadow’s multi-shadowing,
Each GPPN is mapped to a single MPN
The content in a machine page is dynamically encrypted/decrypted depending on the context when the page is accessed.
Works well since few pages are accessed by the application and the kernel at the same time.
Copyright © 2008 VMware, Inc. All rights reserved.

10. Memory Cloaking
Cloaking is a combination of multi-shadowing and cryptography.
With Overshadow’s cryptography scheme,
If a plaintext page is accessed from outside of the page’s shadow context (i.e., accessed by non-owners), the page is encrypted.
If a encrypted page is accessed from the page’s shadow context (i.e., accessed by the owner), the page is decrypted.
The hash of ciphertext is generated after encyption. The hash value is verified before decryption  to enforce integrity
Copyright © 2008 VMware, Inc. All rights reserved.

11. Cloaking Example GVPN-A GPPN-B … … Guest OS GVPN-A MPN-C VMM … … … …
OS page table
Guest OS
GVPN-A
MPN-C
VMM
application
shadow
system
shadow … … … …
Hardware
physical
memory
MPN-C
plaintext
At any time point, a GPP is mapped to only one of the two shadow page tables.
The GPPN-B is mapped in the application’s shadow, the corresponding MP is in plaintext.
If it is the application referencing GVPN-A, everything goes as normal.
Copyright © 2008 VMware, Inc. All rights reserved.

12. Cloaking Example GVPN-A GPPN-B … … Guest OS GVPN-A MPN-C GVPN-A MPN-C
OS page table
Guest OS
GVPN-A
MPN-C
GVPN-A
MPN-C
VMM
application
shadow
system
shadow … … … …
Hardware
physical
memory
MPN-C
ciphertext
plaintext
Kernel tries to access the GPPN-B  no such mapping in system shadow  fault to VMM
Unmap the entry from application’s shadow
Ecrypt/hash the content
VMM fault handling
Remap the GPPN-B to the system shadow.
Copyright © 2008 VMware, Inc. All rights reserved.

13. Cloaking Example GVPN-A GPPN-B … … Guest OS GVPN-A MPN-C GVPN-A MPN-C
OS page table
Guest OS
GVPN-A
MPN-C
GVPN-A
MPN-C
VMM
application
shadow
system
shadow … … … …
Hardware
physical
memory
MPN-C
ciphertext
plaintext
App tries to access the GPPN-B  no such mapping in app shadow  fault to VMM
Unmap the entry from system’s shadow
Verify hash value decrypt the content
VMM fault handling
Remap the GPPN-B to the application shadow.
Copyright © 2008 VMware, Inc. All rights reserved.

14. Overshadow Architecture
VMM Protects App Memory
New virtualization barrier
App trusts VMM, but not OS
Cloaking: Two Views of Memory
App sees normal view
OS sees encrypted view
Shim: App/OS Interactions
Interposes on system calls, interrupts, faults, signals
Transparent to application
Other Apps
Shim
Cloaked App
Guest OS Kernel
Virtual Machine
VMM
Hardware
Copyright © 2008 VMware, Inc. All rights reserved.

15. Cloaking Application Resources
Basic Strategy
Protect existing memory-mapped objects e.g. stack, heap, mapped files, shared mmaps
Make everything else look like one e.g. emulate file read/write using mmap
OS Still Manages Application Resources
Including demand-paged application memory
Moves cloaked data without seeing plaintext contents
Encryption/decryption typically infrequent
Copyright © 2008 VMware, Inc. All rights reserved.

16. Shim: Supporting Unmodified Applications
All the challenges below need to be addressed without modifying either the application or the OS
Context identification
Securely identify the guest context accessing a cloaked resource
To use the correct shadow page table
Secure control transfer
The control transfers between OS and app must be adapted to cloaked execution.
Sources of control transfers: interrupts, faults, signals, and system calls
Adapting system calls
Some system calls need to access applications’ address space (e.g., path names, struct sockaddrs prepared by the calling party)
Copyright © 2008 VMware, Inc. All rights reserved.

17. Shim: Supporting Unmodified Applications
Solution: Shim
OS-specific user-level program managing the transitions between the cloaked application and the OS.
Linked into application address space when the application is loaded.
Mostly cloaked, plus uncloaked trampolines and buffers
Cloaked shim: multi-shadowed like the rest of the application; responsible for task where trust is required (e.g., providing well-defined control transfer entry/exit points, moving data between cloaked/uncloaked memory regions)
Uncloaked shim: contains buffer for kernel and app to exchange uncloaked data; contains simple trampoline code to facilitate transitions from the kernel to the cloaked code).
Communicates with VMM via hypercall interface (which is a secure communication mechanism between the guest and the VMM).  allow complex ops (e.g., system call proxying) to placed in user-mode shim code.
Copyright © 2008 VMware, Inc. All rights reserved.

18. Shim: Supporting Unmodified Applications
Solution: Shim
Each application thread has its own shim instance.
A separate page for each shim instance, call cloaked shim context (CTC) page
Used by VMM to save sensitive data for control transfers.
Register contents
Shim function entry points
ID of the shadow context that owns the shim
Copyright © 2008 VMware, Inc. All rights reserved.

19. Shim: Handling Faults and Interrupts
1. App is executing, a fault or interrupt occurs
2. Fault traps into VMM
Saves registers contents to CTC in the cloaked shim
Zeros out all the app’s general purpose registers
Sets up trampoline to shim (where the kernel will return to after servicing the fault)
Transfers control to kernel
3. Kernel executes
Handles fault as usual
Returns to shim via trampoline
Copyright © 2008 VMware, Inc. All rights reserved.

20. Shim: Handling Faults and Interrupts
4. Shim hypercalls into VMM
Self-identifying
Resume cloaked execution
5. VMM returns to the cloaked app
Restores registers
Transfers control to the application
Copyright © 2008 VMware, Inc. All rights reserved.

21. Shim: Handling System Calls
Extra Transitions
Superset of fault handling
Handlers in cloaked shim interpose on system calls
System Call Adaptation
Arguments may be pointers to cloaked memory
Marshal and unmarshal via buffer in uncloaked shim
unmarshal
marshal
Copyright © 2008 VMware, Inc. All rights reserved.

22. Secure Context Identification
Application Contexts
Must identify precisely to switch shadow page tables
Shim-Based Approach
VMM maintains a separate shadow context for each application address space (indentified by a unique ASID).
Shim hypercalls to VMM to initialize its CTC at startup
VMM writes the ASID and a random value to its CTC
VMM returns the ASID to the caller (i.e., the shim)
The random value is protected in cloaked memory
Shim hypercalls from uncloaked to cloaked execution
Self-identifying: the uncloaked shim passes the random value and the address of its CTC to the VMM
VMM verify if the CTC contains the expected random value
Copyright © 2008 VMware, Inc. All rights reserved.

23. Adapting System Calls Pass-through
A majority can be pass through to the OS without special handling
Marshaling
Many other system calls have non-scalar arguments that require OS to read or modify data in the cloaked app’s address space.
Example: path name, struct sockaddrs as system call arguments
The arguments are marshaled into a buffer in the uncloaked shim
Redirect the register so that the new buffer is used for the sys call
The results are copy backed to the cloaked app after the sys call
More complex examples
Emulation, thread creation, signal handling
File I/O
Extend Overshadow’s cryptography protection to file on disk.
Copyright © 2008 VMware, Inc. All rights reserved.

24. Protection Metadata Protected Resource Per-Page Metadata
Ordered set of pages
Portions mapped into application address space
May be persistent or transient
Per-Page Metadata
Required to enforce privacy, integrity, ordering, freshness
IV – randomly-generated initialization vector
H – secure integrity hash
Managed by VMM
Tracks what’s “supposed to be” in each memory page
Shim helps VMM map GVPN  (IV, H)
Copyright © 2008 VMware, Inc. All rights reserved.

25. Implementation Overshadow System Runs Real Applications
Based on 32-bit x86 VMware VMM
Shim for Linux 2.6.x guest OS
Full cloaking of application code, data, files
Lines of code: to VMM, ~ in shim
Not heavily optimized
Runs Real Applications
Apache web server, PostgreSQL database
Emacs, bash, perl, gcc, …
Copyright © 2008 VMware, Inc. All rights reserved.

26. Microbenchmark Performance
System Calls
Simple PASSTHRU
MARSHALL args
Processes
FORKW – fork/wait process creation, COW overheads
File-Backed mmaps
MMAPW – write word per page, flush to disk
MMAPR – read words back from buffer cache
% Uncloaked Performance
Copyright © 2008 VMware, Inc. All rights reserved.

27. Benchmark Performance
Web
Apache web server caching disabled
Remote load generator ab benchmark tool
Database
PostgresSQL server DBT2 benchmark
Compute
SPECint CPU2006
gcc – worst individual SPEC benchmark
% Uncloaked Performance
Copyright © 2008 VMware, Inc. All rights reserved.

28. Thank you! Questions? The End
Copyright © 2008 VMware, Inc. All rights reserved.

29. Q: Can OS Modify or Inject Application Code?
Answer: No.
Application code resides in cloaked memory; it’s encrypted and integrity-protected.
Any modifications will be detected by integrity checks; modified page contents won’t match hash in MDC.
Copyright © 2008 VMware, Inc. All rights reserved.

30. Q: Can OS Modify Application Instruction Pointer?
Answer: No.
Application registers, including the instruction pointer (IP), are saved in the cloaked thread context (CTC) after all faults/interrupts/syscalls, and restored when cloaked execution resumes.
The CTC resides in cloaked memory; it’s encrypted and integrity-protected, so the OS can’t read or modify it.
Copyright © 2008 VMware, Inc. All rights reserved.

31. Q: Can OS Pretend to Be Application and Issue “Resume Cloaked Exec” Hypercall?
Answer: Yes, but it can’t execute malicious code.
When an application returns from a context switch or other interrupt, the uncloaked shim makes a hypercall asking the VMM to resume cloaked execution.
The OS could pretend to be the application, and make this same hypercall, but integrity checking will cause it to fail unless the CTC is mapped in the proper location.
Even if the OS succeeds, the VMM will enter cloaked execution with the proper saved registers, including the IP. All application pages must be unaltered or integrity checks will fail.
Thus, the OS can only cause cloaked execution to be resumed at the proper point in the proper application code, so it still can’t execute malicious code.
Copyright © 2008 VMware, Inc. All rights reserved.

32. Q: Can OS Tamper with Loader?
Answer: No.
Before entering cloaked execution, the VMM can verify that the shim was loaded properly by comparing hashes of the appropriate memory pages with their expected values.
If this integrity check fails, it will prevent the application from accessing any cloaked resources (files or memory), except in encrypted form.
So while the OS could execute an arbitrary program instead, it would be unable to access any protected data.
Copyright © 2008 VMware, Inc. All rights reserved.

33. Cloaked File I/O Interpose on I/O System Calls Cloaked Files
Read, write, lseek, fstat, etc.
Uncloaked files use simple marshalling
Cloaked Files
Emulate read and write using mmap
Copy data to/from memory-mapped buffers
Decrypted automatically when read by app; Encrypted automatically when flushed to disk by kernel
Shim caches mapped file regions (1MB chunks)
Prepend file header containing size, offset, etc.
Copyright © 2008 VMware, Inc. All rights reserved.

34. Managing Protection Metadata
Copyright © 2008 VMware, Inc. All rights reserved.

35. Protection Metadata: Overview
Per-Page Metadata
Required to enforce privacy, integrity, ordering, freshness
IV – randomly-generated initialization vector
H – secure integrity hash
Tracked by VMM
Metadata for pages mapped into application address space
Intuitively, what’s “supposed” to be in each memory page
(ASID, GVPN)  (IV, H)
Copyright © 2008 VMware, Inc. All rights reserved.

36. Protection Metadata: Details
Protected Resource
Need indirection to support sharing and persistence
(RID, RPN) – unique resource identifer, page offset
Ordered set of (IV, H) pairs in VMM “metadata cache”
Protected Address Space
Shim tracks mappings (start, end)  (RID, RPN)
VMM caches in “metadata lookaside buffer”
VMM upcalls into shim on MLB miss
Metadata Lookup
(ASID, VPN)  (RID, RPN)  (IV, H)
Persistent metadata stored securely in guest filesystem
Copyright © 2008 VMware, Inc. All rights reserved.