1. David Evans http://www.cs.virginia.edu/~evans
Lecture 17:
Defeating Malcode
(Shameless Self-Promotion)
Background
just got here last week
finished degree at MIT week before
Philosophy of advising students
don’t come to grad school to implement someone else’s idea
can get paid more to do that in industry
learn to be a researcher
important part of that is deciding what problems and ideas are worth spending time on
grad students should have their own project
looking for students who can come up with their own ideas for research
will take good students interested in things I’m interested in – systems, programming languages & compilers, security
rest of talk – give you a flavor of the kinds of things I am interested in
meant to give you ideas (hopefully even inspiration!) but not meant to suggest what you should work on
CS551: Security and Privacy
University of Virginia
Computer Science
David Evans

2. Menu: Malcode Defenses
Constrain program behavior
Reference Monitors
In-line Reference Monitors: Naccio
Prevent possibly harmful code from running
Safe Languages
Proof-Carrying Code
INFOSEC Panel Talk
6 April 2019
University of Virginia CS 551

3. University of Virginia CS 551
Program Execution
Speakers
SuperSoaker 2000
Monitor
Program
Disk
Memory
Network
6 April 2019
University of Virginia CS 551

4. University of Virginia CS 551
Program Execution
Speakers
SuperSoaker 2000
Monitor
Reference Monitor
Program
Disk
Memory
Network
6 April 2019
University of Virginia CS 551

5. Ideal Reference Monitor
Sees everything a program is about to do before it does it
Can instantly and completely stop program execution (or prevent action)
Has no other effect on the program or system
Can we build this?
Probably not unless we can build a time machine...
6 April 2019
University of Virginia CS 551

6. Ideal Reference Monitor
Real
Ideal Reference Monitor
most things
Sees everything a program is about to do before it does it
Can instantly and completely stop program execution (or prevent action)
Has no other effect on the program or system
limited
6 April 2019
University of Virginia CS 551

7. University of Virginia CS 551
Operating Systems
Provide reference monitors for most security-critical resources
When a program opens a file in Unix or Windows, the OS checks that the principal running the program can open that file
Doesn’t allow different policies for different programs
No flexibility over what is monitored
OS decides for everyone
Hence, can’t monitor inexpensive operations
6 April 2019
University of Virginia CS 551

8. Reference Monitor as Finite State Automaton [Schneider99]
All other instructions
All other instructions
Aim
Aim 1 2
Fire
Aim
Fire
All other instructions
STOP
Policy Violation
6 April 2019
University of Virginia CS 551

9. SASI (Security Automaton SFI Implementation)
[Erlingsson2000]
Extend SFI to enforce arbitrary safety policies
Compile the policy state machine into the untrusted program
Try to optimize out unnecessary code
6 April 2019
University of Virginia CS 551

10. University of Virginia CS 551
What’s SFI?
Software Fault Isolation [Wahbe, Lucco, PLDI ’1993]
Collusa Software (bought by Microsoft)
Get memory safety by inserting checking instructions around load, store and jump instructions
Do clever things (use delay slots, masking references) to make overhead low (~5%)
6 April 2019
University of Virginia CS 551

11. University of Virginia CS 551
SASI Example
All other instructions
All other instructions
Aim
Aim
x := 3
aim (x);
fire (); 1 2
Fire
Aim
Fire
All other instructions
STOP
Policy Violation
6 April 2019
University of Virginia CS 551

12. University of Virginia CS 551
SASI Example
STATE := 0
if (STATE = 1) then STATE := 0
else if (STATE = 2) then STATE := 0
x := 3
if (STATE = 0) then STATE := 1
else if (STATE = 1) then STATE := 1
else if (STATE = 2) then STATE := 1
aim (x);
if (STATE = 0) then goto VIOLATION
else if (STATE = 1) then STATE := 2
fire ();
...
6 April 2019
University of Virginia CS 551

13. What policies can be enforced?
Assume:
Security Automaton can see entire state of world, everything about instruction about to execute
Security Automaton has unlimited memory, can do unlimited computation
Are there interesting policies that still can’t be enforced?
6 April 2019
University of Virginia CS 551

14. What’s a Security Policy?
What’s a program?
A set of possible executions
What’s an execution?
A sequence of states
What’s a security policy?
A predicate on a set of executions
6 April 2019
University of Virginia CS 551

15. University of Virginia CS 551
More Formally...
 : set of all possible executions (can be infinite)
S: set of executions possible by target program S
P: security policy
set of executions  Boolean
S is safe iff P (S ) is true.
6 April 2019
University of Virginia CS 551

16. Reference Monitors cannot enforce all Security Policies
Some policies depend on:
Knowing about the future
If the program charges the credit card, it must eventually ship the goods
Knowing about all possible executions
Information flow – can’t tell if a program reveals secret information without knowing about other possible executions
Reference Monitors can only know about past of this particular execution
6 April 2019
University of Virginia CS 551

17. University of Virginia CS 551
Safety Policies
Reference monitors can only enforce safety policies
Safety policy is a predicate on a prefix of states (see Schneider98 for more formal definition)
Cannot depend on future: prefix means once it is false, it is always false
Cannot depend on other possible executions
6 April 2019
University of Virginia CS 551

18. University of Virginia CS 551
SASI Enforcement
In theory, SASI can enforce all safety policies.
In practice, it is not possible to observe entire state without modifying it.
In practice, it is impractical to have a FSM between every instruction.
6 April 2019
University of Virginia CS 551

19. University of Virginia CS 551
Naccio
Untrusted
Program
[Evans99]
Most reasonable safety policies don’t depend on every instruction
Most security-critical tasks involve calls to system API
Naccio
Safe
Program
6 April 2019
University of Virginia CS 551

20. University of Virginia CS 551
Problem
Policy Author
understands resources like files, network, etc.
System view:
program manipulates resources by calling system library functions
Problem: need to enforce policy at level of code and system calls.
Describe policy a level author understands, enforce at system level.
[click]
User’s View
System View
Program
tar cf *
WriteFile (fHandle, …)
System Library
Policy
Platform Interface
OS Kernel
Resources
Files
Disk
6 April 2019
University of Virginia CS 551

21. Safety Policy Definition
Resource descriptions: abstract operational descriptions of resources (files, network, threads, display, …)
Platform interface: mapping between system events (e.g., Java API calls, Win32 API calls) and abstract resources
Resource use policy: constraints on manipulating those resources
6 April 2019
University of Virginia CS 551

22. University of Virginia CS 551
Resource Description
global resource RFileSystem
openRead (file: RFile)
Called before file is opened for reading
openWrite (file: RFile)
Called before existing file is opened for writing
write (file: RFile, nbytes: int)
Called before nbytes are written to file
… // other operations for observing properties of files, deleting, etc.
resource RFile
RFile (pathname: String)
Constructs object corresponding to pathname
6 April 2019
University of Virginia CS 551

23. Platform Interface (Win32)
wrapper BOOL
WriteFile (HANDLE file, char *buffer, long nBytes,…) {
// Get corresponding RFile object
RFile rf = lookupRFileByHandle (file);
if (rf != NULL)
RFileSystem.write (rf, nBytes);
%%% // original method call }
6 April 2019
University of Virginia CS 551

24. University of Virginia CS 551
Resource Use Policy
Properties are defined by attaching checking code to resource operations.
openWrite different from openCreate
Next: more complicated LimitBytesWritten property
policy LimitWrite
LimitBytesWritten ( ), NoOverwrite
property LimitBytesWritten (n: int)
requires TrackBytesWritten;
check RFileSystem.write (file: RFile, nbytes: int)
if (bytes_written > n) violation (“Writing more than …”);
stateblock TrackBytesWritten
addfield RFileSystem.bytes_written: int = 0;
precode RFileSystem.write (file: RFile, nbytes: int)
bytes_written += nbytes;
6 April 2019
University of Virginia CS 551

25. Naccio Architecture Current Platforms:
Divide task into two pieces
Left-side: run by policy author
once per-policy, per-platform
Takes safety policy definition,
Produces pdf and p-e library
Right-side: run to prepare program to enforce a policy
satisfies necessary low-level safety properties to prevent a program from circumventing h-l mechanisms.
More about these pieces later
Per policy
Per application
Safety policy definition
Program
Application
transformer
Policy
compiler
Policy description file
Version of program that:
Uses policy-enforcing system library
Satisfies low-level code safety
Policy-enforcing system library
Current Platforms:
JavaVM – program is collection of Java classes
Win32 – program is Win32 executable and DLLs
6 April 2019
University of Virginia CS 551

26. Policy Compiler Policy-enforcing system library Resource use policy
Policy compiler – run by policy author to produce a new policy, or enforce policy on a new platform.
Mostly platform independent – most work is in analyzing policy to determine what resource operations do meaningful checking.
Some platform dependent – generating policy-enforcing system library.
PDF: where to find p-e library
Look at resource impls generation next.
Resource use policy
Platform
independent
analyses
Resource descriptions
Platform interface
Describes Win32 API
Policy description file
Platform dependent
analyses and code generation
System library
System DLLs
(kernel32.dll, user32.dll)
Policy-enforcing system library
Implementations of resource operations
Perform checking described by resource use policy
Wrapper DLLs replace and call system DLLs
Call abstract resource operations as directed by platform interface
6 April 2019
University of Virginia CS 551

27. Application Transformer
Program
Win32 Executable and DLLs
Application Transformer
Policy description file
Version of program that:
Uses policy-enforcing library
• Replace DLL names in import table
• Wrapper for LoadLibrary
Satisfies low-level code safety
6 April 2019
University of Virginia CS 551

28. University of Virginia CS 551
Low-Level Code Safety
Need to make sure programs cannot:
Circumvent checking code
Manipulate a protected resource without using modified library routine
Jump to address after checking is done
Load unprotected version of library
Tamper with checking code or state
Keep executing after a violation is detected
6 April 2019
University of Virginia CS 551

29. University of Virginia CS 551
Low-Level Code Safety
Naccio/JavaVM
Bytecode verifier provides type and memory safety
Wrappers on class loader, reflection to prevent circumvention
Naccio/Win32 [Twyman99]
Scan for kernel traps
SFI to prevent jumps around wrappers (hard to implement)
Read-only pages and wrappers to protect memory (single threaded only)
6 April 2019
University of Virginia CS 551

30. University of Virginia CS 551
Summary
Most interesting policies are safety policies  can be enforced by a reference monitor
Defusing malcode is an active research area
Enforcement is pretty easy
Policy is the hard part (but not many people work on it)
Next time: Proof-Carrying Code, JDK Security
6 April 2019
University of Virginia CS 551

31. Charge: Why are there no new reading assignments?
I read the CD article that it was cruel for professors to expect students to read over “reading holiday” and am mending my evil ways.
The office staff revolted after I asked them to copy both sides of the pages.
I want to give you enough time to read the Green and Libertarian party platforms before Tuesday.
I want to give you enough time to solve the graph isomorphism problems you got trick-or-treating.
So you can have time to work on your projects.
6 April 2019
University of Virginia CS 551