1 Jay Ligatti (Princeton University); joint work with: Lujo Bauer (Carnegie Mellon University), David Walker (Princeton University) Enforcing Non-safety Security Policies with Program Monitors

2 Security Policy Enforcement News flash: Software sometimes does bad stuff –Bugs –Malicious design One mitigation is run-time monitoring –Ensure that software adheres to run-time constraints specified by a security policy –Stack inspection, file access control, applet sandboxing, firewalls, resource monitors, …

3 Run-time Program Monitors Untrusted Target Program Monitor Executing System Open(f,“w”) is OK Open(f,“w”) Monitors enforce policies by: Interposing between untrusted code and the system executing the untrusted code Making sure only safe code is executed

4 Research Agenda What are the limits of program monitors? Which policies can monitors enforce? Understanding limits helps us when selecting enforcement mechanisms –Allows us to choose the simplest mechanism that enforces a policy –Prevents us from attempting to enforce a policy with an insufficiently strong mechanism

5 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

6 Systems and Executions System = a state machine that transitions states by executing actions We specify a system according to the possibly countably infinite set of actions it can execute A = { open f, close f } (where f is any file name) Execution = possibly infinite sequence of actions open f; close f open f1; close f1; open f2; close f2; …

7 Execution Notation On a system with action set A, A* = set of all finite executions A ω = set of all infinite executions A ∞ = set of all executions The empty sequence (execution) is ● The concatenation of sequences s and u is s;u Prefix notation: s≤u (or u≥s) –Means: s is a finite prefix of possibly infinite u –Read: “s prefixes u” (or “u extends s”)

8 Policies [Schneider ’00] Policy = predicate P on sets of executions –Set of executions S   A ∞ satisfies P iff P(S) –Termination policy P(S) iff every execution in S is finite –Key uniformity policy P(S) iff the cryptographic keys used in all executions in S form uniform distribution

9 Properties [Schneider ’00] Policy P is a property iff  predicate p on executions such that  S   A ∞, P(S)  u  S : p(u) Properties cannot specify relationships between executions –Termination is a property –Key uniformity is not a property Monitors enforce properties, not general policies

10 Safety and Liveness [Lamport ’77; Alpern, Schneider ’85] Two types of properties have been studied a lot Safety: “Bad executions cannot be made good”  s  A ∞ :  p(s)   s’≤s :  u≥s’ :  p(u) –Access-control (cannot “undo” illegal accesses) Liveness: “Finite executions can be made good”  s  A* :  u≥s : p(u) –Termination and nontermination

11 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

12 Operation of Monitors: Accepting an OK Action Untrusted Target Program Monitor Executing System Open(f,“w”) is OK Open(f,“w”) Monitor inputs actions from target and outputs actions to the executing system Here, input action is safe to execute, so monitor accepts it (makes it observable)

13 Operation of Monitors: Suppressing an Action Untrusted Target Program Monitor Executing System Open(f,“w”) is not OK Open(f,“w”) Input action is not safe to execute, so monitor suppresses it and allows target to continue executing

14 Operation of Monitors: Inserting an Action Untrusted Target Program Monitor Executing System Open(f,“w”) is not OK Open(f,“w”)Close(f,“w”) Input action is not safe to execute, so monitor inserts another action, then reconsiders the original action

15 Modeling Monitors Model a monitor that can accept, suppress, and insert actions as an edit automaton (Q,q 0,t) –Q is finite or countably infinite set of states –q 0 is initial state –A complete, deterministic, and TM-decidable function t : Q x A  Q x (A U { ● }) current state input action new state action to insert suppress input action

16 Operational Semantics A monitor’s transition function defines how it behaves on individual input actions For the definition of enforcement, we need to consider the way monitors transform entire input executions a1;a2;a2;a3;…a1;a2;a2;a4;… Untrusted input Valid output Monitor Monitors are execution transformers

17 Operational Semantics Judgments Desired judgment: (q 0,s) X  u –Automaton X starting in state q 0 transforms input sequence s into output sequence u Build up to this judgment 1. Single-step judgment (q,s) X  u (q’,s’) 2. Multi-step judgment (q,s) X  u (q’,s’) 3. Transforms judgment (q 0,s) X  u

18 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

19 Effective Enforcement Effective enforcers adhere to two principles Soundness: –Monitors’ outputs (i.e., observable executions) must satisfy the desired property Transparency: –Monitors must not alter the semantics of valid inputs –Conservative definition: on a valid input sequence s, a monitor must output s

20 Enforcing Properties Automaton X starting in q 0 enforces p on a system with action set A iff  s  A ∞ :  u  A ∞ : 1. (q 0,s) X  u 2. p(u)[Soundness] 3. p(s)  (s=u)[Transparency]

21 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

22 Enforcement Powers Related Work Previous work has examined the enforcement bounds of monitors [Schneider ’00; Viswanathan ’00; Hamlen, Morrisett, Schneider ’03; Fong ’04] In the previous models, monitors respond to dangerous actions by halting the target outright Our framework considers more expressive monitors that can accept, suppress, and insert actions

23 Enforcing Properties with Edit Automata Powerful enforcement technique –Suppress (feign execution of) potentially bad actions, and later, if the suppressed actions are found to be safe, re-insert them Limitation: model assumes monitors have same computational power as observing system –In practice, some actions cannot be feigned: –Actions requiring an outside system to execute –Time-dependent actions

24 Renewal Properties Technique of suppressing dangerous actions allows edit automata to enforce any reasonable infinite renewal property Renewal: “Infinite executions are good iff they are good infinitely often”  s  A ω : p(s)  {u≤s | p(u)} is an infinite set

25 Example Renewal Property Check subsystem configuration after rebooting: Whenever reboot is called, it must be followed by a call to checkConf Is a renewal property: –A valid infinite execution has infinitely many valid prefixes (in which checkConf always follows reboot) –An invalid infinite execution has only finitely many valid prefixes There must be some invalid prefix where the next action after a reboot is not checkConf — all successive prefixes are invalid

26 Edit Automata Enforcement (Lower Bound) Theorem:  properties p such that 1. p is a renewal property, 2. p( ● ), and 3.  s  A* : p(s) is decidable,  an edit automaton that enforces p. Edit automata can enforce any reasonable renewal property

27 Edit Automata Enforcement (Lower Bound) Proof idea: Technique of suppressing actions until they are known to be safe causes every valid prefix, and only valid prefixes, of the input to be output –Given a renewal property p, construct an edit automaton X that uses this technique –In all cases, X correctly enforces p If input s has finite length, X outputs longest valid prefix of s Else if  p(s) and s is infinite, X outputs the longest valid (finite) prefix of s (since p is a renewal property) Else if p(s) and s is infinite, X outputs every prefix of s and only prefixes of s

28 Edit Automata Enforcement (Precise Bounds) Edit automata enforce exactly the set of reasonable renewal properties, plus some extra “almost renewal” properties Details are in ESORICS ’05 proceedings

29 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

30 Definition Reminder Safety: “Bad executions cannot be made good” Liveness: “Finite executions can be made good” Renewal: “Infinite executions are good iff they are good infinitely often”

31 Example Renewal Property, Revisited Check subsystem configuration after rebooting: Whenever reboot is called, it must be followed by a call to checkConf Is not a safety property:  p(reboot) but p(reboot;checkConf) Is not a liveness property: reboot;a cannot be “made good” when a  checkConf

32 Safety, Liveness, Renewal SafetyLiveness Renewal All Properties File access control 2 Trivial 3 Eventually audit 4 Check configuration after rebooting 5 Termination 6 Termination + File access control

33 Outline Motivation –Which policies can monitors enforce? Defining “monitor enforces a policy” –Define policies –Define monitors –Define enforcement of policies by monitors Delineating the enforceable policies –Infinite renewal properties –Comparison of renewal, safety, and liveness Summary

34 Summary Monitors modeled by edit automata can enforce any reasonable renewal property Renewal properties include all safety properties, some liveness properties, and some properties that are neither safety nor liveness

35 End Thanks / Questions