1. Light-weight Contexts: An OS Abstraction for Safety and Performance
Landon Cox
March 1, 2017

2. What is a process? Informal Formal A program in execution
Running code + things it can read/write
Process ≠ program
Formal
≥ 1 threads in their own address space
(soon threads will share an address space)

3. Parts of a process Thread Address space
Sequence of executing instructions
Active: does things
Address space
Data the process uses as it runs
Passive: acted upon by threads

4. Play analogy Process is like a play performance
Program is like the play’s script
What are the threads?
Threads
What is the address space?
Address space

5. Dynamic address translation
User process
Translator
(MMU)
Physical
memory
Virtual
address
Physical
address
Base and bounds
Segmentation
Paging

6. What changes on a context switch?
Two-level tree 1 …
1023 ?
Level 1
NULL
NULL
Level 2
0: PhysPage, Res, Prot
0: PhysPage, Res, Prot
1: PhysPage, Res, Prot
1: PhysPage, Res, Prot … … ?
1023: PhysPage, Res, Prot
1023: PhysPage, Res, Prot ?
What changes on a context switch?

7. All thread share the same page table
Two-level tree 1 …
1023 ?
Level 1
NULL
NULL
Level 2
0: PhysPage, Res, Prot
0: PhysPage, Res, Prot
1: PhysPage, Res, Prot
1: PhysPage, Res, Prot … … ?
1023: PhysPage, Res, Prot
1023: PhysPage, Res, Prot ?
All thread share the same page table

8. Parts of a process (revised)
Thread
Sequence of executing instructions
Active: does things
Address space
Data the process uses as it runs
Passive: acted upon by threads
File descriptors
Communication channels (e.g., files, sockets, pipes)
Passive: read-from/written-to by threads

9. Parts of a process (revised)
Thread 1
Thread 2
Page table
File descriptors
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
Each thread has its own stack, but everything else is share: same pages, file descriptors

10. Parts of a process (revised)
Threads are not well isolated from each other.
Process
Thread 1
Thread 2
Page table
File descriptors
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
Threads can run in parallel on different cores.

11. Parts of a process (revised)
Thread 1
Thread 2
Page table
File descriptors
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
If one thread fails, entire process fails.

12. Parts of a process (revised)
Thread 2
Page table
File descriptors
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
Compromising a thread, compromises all data.

13. Parts of a process (revised)
Thread 2
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
Compromising a thread, compromises all data.

14. Parts of a process (revised)
Thread 2
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
See: heartbleed, cloudbleed, etc.

15. Heartbleed

16. Heartbleed bug Nasty bug in OpenSSL implementation
Made public by Google in April, 2014
“A missing bounds check in the handling of the TLS heartbeat extension.”
What is a heartbeat message?
A small message to check if machine is alive (like in Raft)
In SSL, keeps peers from needlessly renegotiating keys

17. Heartbleed bug Sequence number + random num struct {
HeartbeatMessageType type;
uint16 payload_length;
opaque payload[HeartbeatMessage.payload_length];
opaque padding[padding_length];
} HeartbeatMessage; …
buffer = OPENSSL_malloc( payload + padding);
Created for response
Payload length (up to 0xFFFF)
Padding length

18. Heartbleed bug

19. Parts of a process (revised)
Thread 1
Thread 2
Page table
File descriptors
Stack 1
Stack 2
Readable pages
Writable pages
TCP
socket
Open file
How do we normally isolate tasks?

20. Parts of a process (revised)
Thread
Page table
File descriptors …
Stack
Readable pages
Writable pages
TCP
socket
Open file …
Separate processes!

21. Example: Chrome browser

22. Separate processes What has to happen to switch processes?
Trap to the kernel (either by syscall or interrupt)
Kernel chooses next process to run
Point PTBR to page table of next process to run
Flush the TLB
Set mode to user
Jump to next process
Lightweight Contexts (lwCs)
Want isolation provided by processes
But with faster, easier switching than processes

23. Processes vs lwCs Processes Fork Exec lwCs Create Switch
Create identical copy of vm
Use CoW for isolation
New process w/ new PID
New process now schedulable
Exec
Launch new program
Overwrite existing vm
lwCs
Create
Create identical copy of vm
New lwC shares PID w/ parent
No execution until switch
Simple way to snapshot state
Switch
Similar to thread yield
Switch back resumes at call
Restrict, overlay, syscall
Define sharing between parent, child
Trap syscalls from child in parent
Key differences:
Processes are scheduled by kernel
lwCs are enabled by user code
Threads execute w/i an lwC

24. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Process starts in main

25. Server initializes its state
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Server initializes its state

26. Server creates a snapshot
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Server creates a snapshot

27. new stores copy (CoW) of current lwC
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
new stores copy (CoW) of current lwC

28. caller is -1 for now (until new is switched to)
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
caller is -1 for now (until new is switched to)

29. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
return new as snapshot

30. Keep ref to snap so that we can go back.
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Keep ref to snap so that we can go back.

31. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Process request.

32. Now want to use snap to return to initial state.
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Now want to use snap to return to initial state.

33. Where will switching to snap will return thread to?
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Where will switching to snap will return thread to?

34. Switching to snap brings us back to create.
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Switching to snap brings us back to create.

35. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
What is the current lwC?

36. What are the values of caller and new?
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
What are the values of caller and new?

37. state from processed request
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Close caller to delete
state from processed request

38. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
What is the current lwC?

39. Why recursively call snapshot?
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Why recursively call snapshot?

40. new stores copy (CoW) of current lwC
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
new stores copy (CoW) of current lwC

41. caller is -1 for now (until new is switched to)
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
caller is -1 for now (until new is switched to)

42. Example 1 Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
return new as snapshot

43. Keep ref to snap so that we can go back …
Example 1
Snapshot and rollback
Want to process requests from same state
Rollback to eliminate residue from other requests
Isolate influence of requests on each other
Keep ref to snap so that we can go back …

44. Example 2 Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Process starts in main

45. Load key into variable privkey
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Load key into variable privkey

46. Create new snapshot lwC that can overlay any part of parent memory
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Create new snapshot lwC that can overlay any part of parent memory

47. caller is -1 (until child is switched to)
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
caller is -1 (until child is switched to)

48. Destroy the key. Which lwC still has a copy?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Destroy the key. Which lwC still has a copy?

49. Restrict current lwC from accessing child memory. Why?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Restrict current lwC from accessing child memory. Why?

50. Loop to process requests.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Loop to process requests.

51. To sign data switch to child lwC, passing in buffer from current lwC.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
To sign data switch to child lwC, passing in buffer from current lwC.

52. Why will child be able to access buffer?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Why will child be able to access buffer?

53. Thread starts running in child lwC.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Thread starts running in child lwC.

54. What is the value of caller?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
What is the value of caller?

55. Example 2 Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Why a stub function?

56. Map content pointed to by arg in caller to arg in current lwC.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Map content pointed to by arg in caller to arg in current lwC.

57. Example 2 Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Why is this allowed?

58. Perform the actual signing using the key and mapped in arg.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Perform the actual signing using the key and mapped in arg.

59. Why are writes to arg in current lwC visible in caller?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Why are writes to arg in current lwC visible in caller?

60. Why unmap arg after signing?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Why unmap arg after signing?

61. Where will thread resume?
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Switch back to caller.
Where will thread resume?

62. Back in parent lwC (i.e., previous caller).
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Back in parent lwC (i.e., previous caller).

63. Parent lwC can see updates to arg performed in child.
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
Parent lwC can see updates to arg performed in child.

64. With data signed, process request, loop back for more …
Example 2
Sensitive data isolation
Only want specific lwC to access crypto key
Other lwCs shouldn’t access key
With data signed, process request, loop back for more …

65. Evaluation Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)

66. What is true for each approach (on FreeBSD)?
Evaluation
Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
What is true for each approach (on FreeBSD)?

67. Note: swapcontext in FreeBSD is a syscall
Evaluation
Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
Note: swapcontext in FreeBSD is a syscall

68. Note: swapcontext in Linux is not asyscall
Evaluation
Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
Note: swapcontext in Linux is not asyscall

69. On Linux this number of 6% of lwC …
Evaluation
Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
On Linux this number of 6% of lwC …

70. Why are processes and k-threads twice as slow as lwCs?
Evaluation
Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
Why are processes and k-threads twice as slow as lwCs?

71. Evaluation Is switching between lwCs fast?
Vs processes (different page tables)
Vs kernel threads (thread queues in kernel)
Vs user threads (thread queues in user space)
lwCs avoid:
* scheduling overhead (go right to next lwC)