1. Enforcing Non-safety Security Policies with Program Monitors
Jay Ligatti (Princeton University); joint work with:
Lujo Bauer (Carnegie Mellon University),
David Walker (Princeton University)

2. Security Policy Enforcement
News flash: Software sometimes does bad stuff
Bugs
Malicious design
One mitigation is run-time monitoring
Ensure that software adheres to run-time constraints specified by a security policy
Stack inspection, file access control, applet sandboxing, firewalls, resource monitors, …

3. Run-time Program Monitors
Untrusted
Target
Program
Monitor
Executing
System
Open(f,“w”)
is OK
Open(f,“w”)
Open(f,“w”)
Monitors enforce policies by:
Interposing between untrusted code and the system executing the untrusted code
Making sure only safe code is executed

4. Research Agenda
What are the limits of program monitors? Which policies can monitors enforce?
Understanding limits helps us when selecting enforcement mechanisms
Allows us to choose the simplest mechanism that enforces a policy
Prevents us from attempting to enforce a policy with an insufficiently strong mechanism

5. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

6. Systems and Executions
System = a state machine that transitions states by executing actions
We specify a system according to the possibly countably infinite set of actions it can execute
A = { open f, close f } (where f is any file name)
Execution = possibly infinite sequence of actions
open f; close f
open f1; close f1; open f2; close f2; …

7. Execution Notation
On a system with action set A, A* = set of all finite executions Aω = set of all infinite executions A∞ = set of all executions
The empty sequence (execution) is ●
The concatenation of sequences s and u is s;u
Prefix notation: s≤u (or u≥s)
Means: s is a finite prefix of possibly infinite u
Read: “s prefixes u” (or “u extends s”)
We’ll need some notation for building up executions and sets of executions from actions.

8. Policies [Schneider ’00]
Policy = predicate P on sets of executions
Set of executions S Í A∞ satisfies P iff P(S)
Termination policy
P(S) iff every execution in S is finite
Key uniformity policy
P(S) iff the cryptographic keys used in all executions in S form uniform distribution

9. Properties [Schneider ’00]
Policy P is a property iff $ predicate p on executions such that "S Í A∞ , P(S) Û "uÎS : p(u)
Properties cannot specify relationships between executions
Termination is a property
Key uniformity is not a property
Monitors enforce properties, not general policies
A policy is a property iff we can decide whether some set of executions satisfies the policy just by examining every execution in the set in isolation.

10. Safety and Liveness [Lamport ’77; Alpern, Schneider ’85]
Two types of properties have been studied a lot
Safety: “Bad executions cannot be made good”
"sÎA∞ : Øp(s) Þ $s’≤s : "u≥s’ : Øp(u)
Access-control (cannot “undo” illegal accesses)
Liveness: “Finite executions can be made good” "sÎA* : $u≥s : p(u)
Termination and nontermination

11. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

12. Operation of Monitors: Accepting an OK Action
Untrusted
Target
Program
Monitor
Executing
System
Open(f,“w”)
Open(f,“w”)
is OK
Open(f,“w”)
Monitor inputs actions from target and
outputs actions to the executing system Here, input action is safe to execute, so
monitor accepts it (makes it observable)

13. Operation of Monitors: Suppressing an Action
Untrusted
Target
Program
Monitor
Executing
System
Open(f,“w”)
Open(f,“w”)
is not OK
Input action is not safe to execute, so monitor suppresses it and allows target to continue executing

14. Operation of Monitors: Inserting an Action
Untrusted
Target
Program
Monitor
Executing
System
Open(f,“w”)
Open(f,“w”)
is not OK
Close(f,“w”)
Input action is not safe to execute, so monitor inserts another action, then reconsiders the original action

15. Modeling Monitors t : Q x A ® Q x (A U {●})
Model a monitor that can accept, suppress, and insert actions as an edit automaton (Q,q0,t)
Q is finite or countably infinite set of states
q0 is initial state
A complete, deterministic, and TM-decidable function
t : Q x A ® Q x (A U {●})
this transition function does not directly model the acceptance of an action.
accept an action by first inserting it into the output and then suppressing it from the input
current
state
input
action
new
state
action
to insert
suppress
input
action

16. Operational Semantics
A monitor’s transition function defines how it behaves on individual input actions
For the definition of enforcement, we need to consider the way monitors transform entire input executions
Monitors are execution transformers
Untrusted input
Valid output
a1;a2;a2;a3;…
a1;a2;a2;a4;…
Monitor

17. Operational Semantics Judgments
Desired judgment: (q0,s) X ß u
Automaton X starting in state q0 transforms input sequence s into output sequence u
Build up to this judgment
1. Single-step judgment (q,s) X ®u (q’,s’)
2. Multi-step judgment (q,s) X Þu (q’,s’)
3. Transforms judgment (q0,s) X ß u

18. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

19. Effective Enforcement
Effective enforcers adhere to two principles
Soundness:
Monitors’ outputs (i.e., observable executions) must satisfy the desired property
Transparency:
Monitors must not alter the semantics of valid inputs
Conservative definition: on a valid input sequence s, a monitor must output s
Target can expect that if it behaves well, it’s actions will execute properly.

20. Enforcing Properties
Automaton X starting in q0 enforces p on a system with action set A iff "sÎA∞: $uÎA∞: 1. (q0,s) X ß u 2. p(u) [Soundness] 3. p(s) Þ (s=u) [Transparency]

21. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

22. Enforcement Powers Related Work
Previous work has examined the enforcement bounds of monitors [Schneider ’00; Viswanathan ’00; Hamlen, Morrisett, Schneider ’03; Fong ’04]
In the previous models, monitors respond to dangerous actions by halting the target outright
Our framework considers more expressive monitors that can accept, suppress, and insert actions

23. Enforcing Properties with Edit Automata
Powerful enforcement technique
Suppress (feign execution of) potentially bad actions, and later, if the suppressed actions are found to be safe, re-insert them
Limitation: model assumes monitors have same computational power as observing system
In practice, some actions cannot be feigned:
Actions requiring an outside system to execute
Time-dependent actions

24. Renewal Properties
Technique of suppressing dangerous actions allows edit automata to enforce any reasonable infinite renewal property
Renewal: “Infinite executions are good iff they are good infinitely often”
"sÎAω : p(s) Û {u≤s | p(u)} is an infinite set

25. Example Renewal Property
Check subsystem configuration after rebooting:
Whenever reboot is called, it must be followed by a call to checkConf
Is a renewal property:
A valid infinite execution has infinitely many valid prefixes (in which checkConf always follows reboot)
An invalid infinite execution has only finitely many valid prefixes
There must be some invalid prefix where the next action after a reboot is not checkConf — all successive prefixes are invalid

26. Edit Automata Enforcement (Lower Bound)
Theorem: " properties p such that 1. p is a renewal property, 2. p(●), and "sÎA* : p(s) is decidable, $ an edit automaton that enforces p.
Edit automata can enforce any reasonable renewal property

27. Edit Automata Enforcement (Lower Bound)
Proof idea: Technique of suppressing actions until they are known to be safe causes every valid prefix, and only valid prefixes, of the input to be output
Given a renewal property p, construct an edit automaton X that uses this technique
In all cases, X correctly enforces p
If input s has finite length, X outputs longest valid prefix of s
Else if Øp(s) and s is infinite, X outputs the longest valid (finite) prefix of s (since p is a renewal property)
Else if p(s) and s is infinite, X outputs every prefix of s and only prefixes of s

28. Edit Automata Enforcement (Precise Bounds)
Edit automata enforce exactly the set of reasonable renewal properties, plus some extra “almost renewal” properties
Details are in ESORICS ’05 proceedings
“Almost renewal” properties may consider some infinite executions to be valid even though they have only finitely many valid prefixes
To enforce an “almost renewal” property on an execution u, an automaton needs to decide while inputting u that u is the only valid extension of its current input
At that point, the automaton can enter an infinite loop to insert the remaining actions of u

29. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

30. Definition Reminder Safety: “Bad executions cannot be made good”
Liveness: “Finite executions can be made good”
Renewal: “Infinite executions are good iff they are good infinitely often”

31. Example Renewal Property, Revisited
Check subsystem configuration after rebooting:
Whenever reboot is called, it must be followed by a call to checkConf
Is not a safety property: Øp(reboot) but p(reboot;checkConf)
Is not a liveness property:
reboot;a cannot be “made good” when a¹checkConf

32. Safety, Liveness, Renewal
All Properties
1 File access control
2 Trivial
3 Eventually audit
4 Check configuration
after rebooting
5 Termination
6 Termination +
File access control
Renewal
Safety
Liveness 1 2 3 5 4 6

33. Outline Motivation Defining “monitor enforces a policy”
Which policies can monitors enforce?
Defining “monitor enforces a policy”
Define policies
Define monitors
Define enforcement of policies by monitors
Delineating the enforceable policies
Infinite renewal properties
Comparison of renewal, safety, and liveness
Summary

34. Summary
Monitors modeled by edit automata can enforce any reasonable renewal property
Renewal properties include all safety properties, some liveness properties, and some properties that are neither safety nor liveness

35. End
Thanks / Questions