1. Enforcing Security Policies using Transactional Memory Introspection Vinod Ganapathy Rutgers University Arnar BirgissonMohan Dhawan Ulfar ErlingssonLiviu Iftode

2. Vinod GanapathyTransactional Memory Introspection/IPAM'082 X server with multiple X clients REMOTE LOCAL

3. Vinod GanapathyTransactional Memory Introspection/IPAM'083 REMOTE Malicious remote X client LOCAL

4. Vinod GanapathyTransactional Memory Introspection/IPAM'084 REMOTE Undesirable information flow LOCAL

5. Vinod GanapathyTransactional Memory Introspection/IPAM'085 Desirable information flow LOCAL REMOTE

6. Vinod GanapathyTransactional Memory Introspection/IPAM'086 X server X server with authorization X client Operation requestResponse Authorization policy Reference monitor Allowed? YES/NO

7. Vinod GanapathyTransactional Memory Introspection/IPAM'087 Server The problem Client Authorization policy Reference monitor Multiple clients Manages resources Likely multithreaded Security enforcement crosscuts application functionality

8. Vinod GanapathyTransactional Memory Introspection/IPAM'088 Outline Enforcing authorization policies Problems with existing techniques Transactional Memory Introspection Implementation and experiments Open questions and future work

9. Vinod GanapathyTransactional Memory Introspection/IPAM'089 Existing enforcement interface dispatch_request ( ) {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); }

10. Vinod GanapathyTransactional Memory Introspection/IPAM'0810 Existing enforcement interface dispatch_request ( ) {... perform_request ( ); } perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1(); };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2(); }; }

11. Vinod GanapathyTransactional Memory Introspection/IPAM'0811 Three problems Violation of complete mediation Time-of-check to Time-of-use bugs Handing authorization failures

12. Vinod GanapathyTransactional Memory Introspection/IPAM'0812 I. Incomplete mediation dispatch_request ( ) { … perform_request ( ); } perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1(); };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2(); }; } Must guard each resource access to ensure complete mediation

13. Vinod GanapathyTransactional Memory Introspection/IPAM'0813 I. Incomplete mediation ssize_t vfs_read (struct file *file,...) {... if (check_permission(file, MAY_READ)) { file->f_op->read(file,...); }... } int page_cache_read (struct file *file,...) { struct address_space *mapping = file->f_dentry->d_inode->i_mapping;... mapping->a_ops->readpage(file,...); } [Zhang et al., USENIX Security ‘02]

14. Vinod GanapathyTransactional Memory Introspection/IPAM'0814 perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } II. TOCTTOU bugs

15. Vinod GanapathyTransactional Memory Introspection/IPAM'0815 perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } II. TOCTTOU bugs Similar race condition found in the Linux Security Modules framework [Zhang et al. USENIX Security ’02] Several similar bugs recently found in popular enforcement tools: [Watson, WOOT ’07] GSWTK Systrace [Provos, USENIX Security ’03] FreeBSD Sysjail [Johnson and Deksters ’07]

16. Vinod GanapathyTransactional Memory Introspection/IPAM'0816 II. TOCTTOU bugs perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } Authorization check and resource access must be atomic

17. Vinod GanapathyTransactional Memory Introspection/IPAM'0817 III. Failure handling perform_request ( ) {... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() };... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } Handling authorization failures is ad hoc and error prone

18. Vinod GanapathyTransactional Memory Introspection/IPAM'0818 III. Failure handling Exception-handling code accounts for a large fraction of server software –Over two-thirds of server software [IBM ’87] –Nearly 46% on several Java benchmarks [Weimer & Necula OOPSLA’04] Exception-handling code itself is error- prone [Fetzer and Felber ’04] SecurityException most often handled erroneously [Weimer & Necula OOPSLA’04]

19. Vinod GanapathyTransactional Memory Introspection/IPAM'0819 Summary of problems Violation of complete mediation –Need to identify all the resources accessed –Example: Bug in Linux Security Modules [Zhang et al., USENIX Security ‘02] Time-of-check to Time-of-use bugs –Examples: [Zhang et al., USENIX Security ‘02] [Watson, WOOT ‘07] Handing authorization failures – Large fraction of server code relates to error handling [IBM survey, ’87, Weimer and Necula, ‘04 ] –Error-handling code is error-prone! [Fetzer & Felber ’04] Security enforcement crosscuts application functionality Our solution: TMI Decouples security enforcement from application functionality

20. Vinod GanapathyTransactional Memory Introspection/IPAM'0820 Outline Enforcing authorization policies Problems with existing techniques Transactional Memory Introspection (TMI) –Programmer’s interface –Mechanics of TMI Implementation and experiments Open questions and future work

21. Vinod GanapathyTransactional Memory Introspection/IPAM'0821 Transactional memory primer Alternative to lock-based programming Reason about atomic sections, not locks TM attempts to guarantee ACID semantics acquire(S1.lock) acquire(S2.lock) value = S1.pop() S2.push(value) Release(S2.lock) Release(S1.lock) transaction { value = S1.pop() S2.push(value) }

22. Vinod GanapathyTransactional Memory Introspection/IPAM'0822 Programmer’s interface to TMI dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); }

23. Vinod GanapathyTransactional Memory Introspection/IPAM'0823 Programmer’s interface to TMI dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } Authorization manager: case (resource=R, access_type=A)  if (!allowed(principal, R, A)) then abort_tx allowed(principal, resource, access)? allowed(principal, resource’, access’)?

24. Vinod GanapathyTransactional Memory Introspection/IPAM'0824 I. Complete mediation for free dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } TMI automatically invokes authorization checks

25. Vinod GanapathyTransactional Memory Introspection/IPAM'0825 II. TOCTTOU-freedom for free dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } Conflicting resource accesses automatically abort transaction

26. Vinod GanapathyTransactional Memory Introspection/IPAM'0826 III. Error-handling for free dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } Unauthorized resource accesses automatically abort transaction

27. Vinod GanapathyTransactional Memory Introspection/IPAM'0827 Decouples functionality and security dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } Authorization manager

28. Vinod GanapathyTransactional Memory Introspection/IPAM'0828 Outline Enforcing authorization policies Problems with existing techniques Transactional Memory Introspection (TMI) –Programmer’s interface –Mechanics of TMI Implementation and experiments Open questions and future work

29. Vinod GanapathyTransactional Memory Introspection/IPAM'0829 TM runtime system The TM runtime maintains per-transaction read/write sets and detects conflicts transaction { value = S1.pop() S2.push(value) } val1 = S1.pop() val2 = S1.pop() S2.push(val2) S2.push(val1) TransactionRead setWrite set Green S1.stkptr Red S1.stkptr, S2.stkptr

30. Vinod GanapathyTransactional Memory Introspection/IPAM'0830 TM runtime system Transaction body Execution Read and Write Sets Validation Contention manager Retry Commit logic Commit

31. Vinod GanapathyTransactional Memory Introspection/IPAM'0831 Transactional Memory Introspection Transaction body Execution Read and Write Sets Validation Contention manager Retry Commit logic CommitAuthorization Auth. checks Auth. Manager Success Failure Abort

32. Vinod GanapathyTransactional Memory Introspection/IPAM'0832 perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } Transactional Memory Introspection dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } Present in read/write set Accesses checked before tx commits

33. Vinod GanapathyTransactional Memory Introspection/IPAM'0833 Outline Enforcing authorization policies Problems with existing techniques Transactional Memory Introspection Implementation and experiments Open questions and future work

34. Vinod GanapathyTransactional Memory Introspection/IPAM'0834 TMI Implementation: TMI/DSTM2 Implemented using Sun’s DSTM2 Object-based software TM system TM system modified to –Trigger authorization checks on additions to read/write set and upon transaction validation –Raise AccessDeniedException upon abort –Integrate transactional I/O libraries Fewer than 500 lines changed in DSTM2

35. Vinod GanapathyTransactional Memory Introspection/IPAM'0835 Porting software to TMI/DSTM2 1.Mark transactional objects with @atomic –Also require @atomic wrappers for libraries: java.util.HashMap, java.util.Vector 2.Reads and writes to fields of @atomic objects replaced with DSTM2 accessors 3.Place transaction{…} blocks around client requests 4.Write an authorization manager

36. Vinod GanapathyTransactional Memory Introspection/IPAM'0836 Dealing with side-effects Problem: –TM provides ACID semantics to memory updates –System calls inside transaction{…} block can violate atomicity and isolation Use transactional I/O packages Integrate with commit logic

37. Vinod GanapathyTransactional Memory Introspection/IPAM'0837 Dealing with side-effects Transaction body Execution Read and Write Sets Validation Contention manager Retry 2-phase commit CommitAuthorization Auth. checks Auth. Manager Success Failure Abort TX I/O

38. Vinod GanapathyTransactional Memory Introspection/IPAM'0838 GradeSheet in TMI/DSTM2

39. Vinod GanapathyTransactional Memory Introspection/IPAM'0839 Evaluation Ported four Java-based servers GradeSheet: A grade-management server FreeCS: A chat server WeirdX: An X window management server –Enforced a simple XACML based policy Tar: A tar archive service –Enforced Java stack inspection policy

40. Vinod GanapathyTransactional Memory Introspection/IPAM'0840 Modifications needed ServerLOCLines modifiedTransactions GradeSheet9003001 Tar service5,000< 501 FreeCS22,00086047 WeirdX27,0004,800108 Authorization managers were approximately 200 lines of code in each case

41. Vinod GanapathyTransactional Memory Introspection/IPAM'0841 REMOTE Example policy enforced in WeirdX LOCAL

42. Vinod GanapathyTransactional Memory Introspection/IPAM'0842 perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } When to enforce policy? dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Eager

43. Vinod GanapathyTransactional Memory Introspection/IPAM'0843 perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } When to enforce policy? dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Lazy

44. Vinod GanapathyTransactional Memory Introspection/IPAM'0844 perform_request ( ) {... perform_access (resource);... perform_access’(resource’); } When to enforce policy? dispatch_request ( ) { transaction [ principal ] {... perform_request ( ); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Parallel

45. Vinod GanapathyTransactional Memory Introspection/IPAM'0845 Performance overheads of TMI 10x -15.8%

46. Vinod GanapathyTransactional Memory Introspection/IPAM'0846 Performance overheads of STM Software transactional memory imposes a significant overhead ServerNativeTMI-portedOverhead GradeSheet395μs451μs14.7% Tar service4.96s15.40s2.1x FreeCS321μs3907μs11.2x WeirdX0.23ms6.40ms26.8x Hardware-accelerated STM will reduce runtime overheads of TM runtime systems

47. Vinod GanapathyTransactional Memory Introspection/IPAM'0847 Outline Enforcing authorization policies Problems with existing techniques Transactional Memory Introspection Implementation and experiments Open questions and future work

48. Vinod GanapathyTransactional Memory Introspection/IPAM'0848 Hardware support for TMI Problem: –STM imposes high runtime overheads –Want to make TMI practical for adoption on real-world servers Solution: Implementing TMI in hardware transactional memory (HTM) systems –HTM-based software as fast (or faster than) as lock-based software.

49. Vinod GanapathyTransactional Memory Introspection/IPAM'0849 Interaction of TMI and I/O Problem: I/O instructions in transactions violate atomicity and isolation Can deal with file and database I/O with transactional libraries Network I/O? Display? Other devices? Possible solution: Combine TMI and virtual machine introspection

50. Vinod GanapathyTransactional Memory Introspection/IPAM'0850 A formal semantics of TMI Problem: –Pathological interactions of TMI with STM implementation details Example: Weak-atomicity, in-place updates –With Lazy enforcement, TMI can leak sensitive information Solution: –Need a formal semantics for TMI

51. Vinod GanapathyTransactional Memory Introspection/IPAM'0851 Summary Transactional Memory Introspection –A new reference monitor architecture –Decouples application functionality from security policy enforcement Benefits –Better guarantees on complete mediation –Freedom from TOCTTOU bugs –Better handling of authorization failures

52. Enforcing Security Policies using Transactional Memory Introspection Reference: Upcoming CCS 2008 paper Vinod Ganapathy Rutgers University vinodg@cs.rutgers.edu http://www.cs.rutgers.edu/~vinodg Thank you!

53. Vinod GanapathyTransactional Memory Introspection/IPAM'0853 This slide intentionally left blank

54. Vinod GanapathyTransactional Memory Introspection/IPAM'0854 Other policies to enforce Prevent unauthorized –Copy and paste –Modification of inputs meant for other clients –Changes to window settings of other clients –Retrieval of bitmaps: Screenshots [Berger et al., ’90] [Epstein et al., ‘90] [Kilpatrick et al., ‘03]

55. Vinod GanapathyTransactional Memory Introspection/IPAM'0855 Contributions Transactional Memory Introspection –Decouples application functionality from security policy enforcement –Ideally suited for multithreaded servers Experiments on four servers: –TMI eases security enforcement –TMI imposes modest runtime overheads

56. Vinod GanapathyTransactional Memory Introspection/IPAM'0856 Stateful authorization policies “A Copy from a High-security window should not be followed by a paste to a low- security window” Need to track order of resource accesses Modify TM system to use read/write lists