1. David Evans http://www.cs.virginia.edu/~evans CS588: Security and Privacy University of Virginia Computer Science Lecture 18: Malcode Countermeasures

2. 19 Nov 2001University of Virginia CS 5882 Menu Reference Monitors Java Security Proof-Carrying Code (time permitting) Monday: Firewalls, Intrusion Detection Movie – make sure I stop by 3:00!

3. 19 Nov 2001University of Virginia CS 5883 Malcode Defenses Constrain program behavior –Reference Monitors In-line Reference Monitors Prevent possibly harmful code from running –Safe Languages –Proof-Carrying Code

4. 19 Nov 2001University of Virginia CS 5884 Program Execution Program Monitor Speakers SuperSoaker 2000 Disk Memory Network

5. 19 Nov 2001University of Virginia CS 5885 Program Execution Program Monitor Speakers SuperSoaker 2000 Disk Memory Network Reference Monitor

6. 19 Nov 2001University of Virginia CS 5886 Ideal Reference Monitor 1.Sees everything a program is about to do before it does it 2.Can instantly and completely stop program execution (or prevent action) 3.Has no other effect on the program or system Can we build this? Probably not unless we can build a time machine...

7. 19 Nov 2001University of Virginia CS 5887 Ideal Reference Monitor 1.Sees everything a program is about to do before it does it 2.Can instantly and completely stop program execution (or prevent action) 3.Has no other effect on the program or system Real most things limited

8. 19 Nov 2001University of Virginia CS 5888 Operating Systems Provide reference monitors for most security-critical resources –When a program opens a file in Unix or Windows, the OS checks that the principal running the program can open that file Doesn’t allow different policies for different programs No flexibility over what is monitored –OS decides for everyone –Hence, can’t monitor inexpensive operations

9. 19 Nov 2001University of Virginia CS 5889 Reference Monitor as Finite State Automaton [Schneider99] 0 1 All other instructions Aim Fire STOP Policy Violation 2 All other instructions Aim All other instructions Aim

10. 19 Nov 2001University of Virginia CS 58810 What policies can be enforced? Assume: –Security Automaton can see entire state of world, everything about instruction about to execute –Security Automaton has unlimited memory, can do unlimited computation Are there interesting policies that still can’t be enforced?

11. 19 Nov 2001University of Virginia CS 58811 What’s a Security Policy? What’s a program? –A set of possible executions What’s an execution? –A sequence of states What’s a security policy? –A predicate on a set of executions

12. 19 Nov 2001University of Virginia CS 58812 More Formally...  : set of all possible executions (can be infinite)  S : set of executions possible by target program S P : security policy set of executions  Boolean S is safe iff P (  S ) is true.

13. 19 Nov 2001University of Virginia CS 58813 Reference Monitors cannot enforce all Security Policies Some policies depend on: –Knowing about the future If the program charges the credit card, it must eventually ship the goods –Knowing about all possible executions Information flow – can’t tell if a program reveals secret information without knowing about other possible executions Reference Monitors can only know about past of this particular execution

14. 19 Nov 2001University of Virginia CS 58814 Safety Policies Reference monitors can only enforce safety policies Safety policy is a predicate on a prefix of states (see Schneider98 for more formal definition) –Cannot depend on future: prefix means once it is false, it is always false –Cannot depend on other possible executions

15. 19 Nov 2001University of Virginia CS 58815 Java Security Real or Decaf?

16. 19 Nov 2001University of Virginia CS 58816 What is Java? A.Island in Indonesia B.A Programming Language (Java  ) C.A Portable Low-Level Language (JVML) D.A Platform (JavaVM) E.A (semi-)successful marketing strategy –JavaScript is not related to Java or Java  F.Work on your projects G.All of the above

17. 19 Nov 2001University of Virginia CS 58817 Java  : Programming Language “A simple, object-oriented, distributed, interpreted, robust, secure, architecture neutral, portable, high-performance, multithreaded, and dynamic language.” [Sun95]

18. 19 Nov 2001University of Virginia CS 58818 What is a secure language? 1.Language is designed so it cannot express certain computations considered insecure. 2.Language is designed so that (accidental) program bugs are likely to be caught by the compiler or run- time environment instead of leading to security vulnerabilities. A few attempt to do this: PLAN, packet filters

19. 19 Nov 2001University of Virginia CS 58819 Safe Programming Languages Type Safety –Compiler and run-time environment ensure that bits are treated as the type they represent Memory Safety –Compiler and run-time environment ensure that program cannot access memory outside defined storage Control Flow Safety –Can’t jump to arbitrary addresses Which of these does C++ have? Not a new idea: LISP had these in 1960!

20. 19 Nov 2001University of Virginia CS 58820 Java  Safety Type Safety –Most types checked statically –Coercions, array assignments type checked at run time Memory Safety –No direct memory access (e.g., pointers) –Primitive array type with mandatory run-time bounds checking Control Flow Safety –Structured control flow, no arbitrary jumps

21. 19 Nov 2001University of Virginia CS 58821 Malicious Code Can a safe programming language protect you from malcode? 1.Code your servers in it to protect from buffer overflow bugs 2.Only allow programs from untrustworthy origins to run if the are programmed in the safe language

22. 19 Nov 2001University of Virginia CS 58822 Safe Languages? But how can you tell program was written in the safe language? –Get the source code and compile it (most vendors, and all malicious attackers refuse to provide source code) –Special compilation service signs object files generated from the safe language (SPIN, [Bershad96]) –Verify object files preserve safety properties of source language (Java)

23. 19 Nov 2001University of Virginia CS 58823 JVML javac Compiler malcode.java Java  Source Code malcode.class JVML Object Code JavaVM Joe User Joe wants to know JVML code satisfies Java  ’s safety properties.

24. 19 Nov 2001University of Virginia CS 58824 Does JVML satisfy Java  ’s safety properties? iconst_2 push integer constant 2 on stack istore_0 store top of stack in variable 0 as int aload_0 load object reference from variable 0 arraylength replace array on top of stack with its length No! This code violates Java  ’s type rules.

25. 19 Nov 2001University of Virginia CS 58825 Bytecode Verifier malcode.class JVML Object Code Java Bytecode Verifier Joe User JavaVM “Okay” Invalid STOP Trusted Computing Base

26. 19 Nov 2001University of Virginia CS 58826 Bytecode Verifier Checks JVML code satisfies Java  ’s safety properties Type safe – stack and variable slots must store and load as same type Memory safe (guaranteed by instruction set) Control flow safe: jumps must be within function, or call/return

27. 19 Nov 2001University of Virginia CS 58827 Are Java Bytecode Verifiers Complicated? ~700 rules to enforce, JVML specification is (not all clearly specified) Emin Gün Sirer found > 100 bugs in commercial bytecode verifiers (using automatic test generation) –At least 15 of them were security vulnerabilities JVML includes jsr instruction (jump to subroutine), can be called with different types in variables and on stack

28. 19 Nov 2001University of Virginia CS 58828 Java javac Compiler malcode.java malcode.class JVML Joe User Java Bytecode Verifier JavaVM “Okay” Invalid STOP Trusted Computing Base

29. 19 Nov 2001University of Virginia CS 58829 JavaVM Virtual machine – interpreter for JVML programs Has complete access to host machine Bytecode verifier ensures some safety properties, JavaVM must ensure rest: –Type safety of run-time casts, array assignments –Memory safety: array bounds checking –Resource use policy

30. 19 Nov 2001University of Virginia CS 58830 JavaVM Policy Enforcment From java.io.File: public boolean delete() { SecurityManager security = System.getSecurityManager(); if (security != null) { security.checkDelete(path); } if (isDirectory()) return rmdir0(); else return delete0(); } [JDK 1.0 – JDK 1.1]

31. 19 Nov 2001University of Virginia CS 58831 java.lang.SecurityManager /** Throws a SecurityException if the calling thread is not allowed to delete the specified file. This method is invoked for the current security manager by the delete method of class File. */ (Some other comments deleted.) public void checkDelete(String file) { throw new SecurityException(); }

32. 19 Nov 2001University of Virginia CS 58832 Security Manager Reference monitor –How well does it satisfy the requirements? Complete mediation Can stop execution/prevent action Limited effect on execution until policy violation User/host application creates a subclass of SecurityManager to define a policy

33. 19 Nov 2001University of Virginia CS 58833 HotJava’s Policy (JDK 1.1.7) public class AppletSecurity extends SecurityManager {... public synchronized void checkDelete(String file) { checkWrite(file); }... }

34. 19 Nov 2001University of Virginia CS 58834 AppletSecurity.checkWrite (some exception handling code removed) public synchronized void checkWrite(String file) { if (inApplet()) { if (!initACL) initializeACLs(); String realPath = (new File(file)).getCanonicalPath(); for (int i = writeACL.length ; i-- > 0 ;) { if (realPath.startsWith(writeACL[i])) return; } throw new AppletSecurityException ("checkwrite", file, realPath); } Note: no checking if not inApplet! Very important this does the right thing.

35. 19 Nov 2001University of Virginia CS 58835 inApplet boolean inApplet() { return inClassLoader(); } Inherited from java.lang.SecurityManager: protected boolean inClassLoader() { return currentClassLoader() != null; }

36. 19 Nov 2001University of Virginia CS 58836 currentClassLoader /** Returns an object describing the most recent class loader executing on the stack. Returns the class loader of the most recent occurrence on the stack of a method from a class defined using a class loader; returns null if there is no occurrence on the stack of a method from a class defined using a class loader. */ protected native ClassLoader currentClassLoader();

37. 19 Nov 2001University of Virginia CS 58837 Recap java.io.File.delete calls SecurityManager.checkDelete before deleting HotJava overrides SecurityManager with AppletSecurity to set policy AppletSecurity.checkDelete calls AppletSecurity.checkWrite AppletSecurity.checkWrite checks if any method on stack has a ClassLoader If not no checks; if it does, checks ACL list

38. 19 Nov 2001University of Virginia CS 58838 JDK 1.0 Trust Model When JavaVM loads a class from the CLASSPATH, it has no associated ClassLoader (can do anything) When JavaVM loads a class from elsewhere (e.g., the web), it has an associated ClassLoader

39. 19 Nov 2001University of Virginia CS 58839 JDK Evolution JDK 1.1: Signed classes from elsewhere and have no associated ClassLoader JDK 1.2: –Different classes can have different policies based on ClassLoader –Explict enable/disable/check privileges –SecurityManager is now AccessController

40. 19 Nov 2001University of Virginia CS 58840 What can go wrong? Java API doesn’t call right SecurityManager checks (63 calls in java.*) –Font loading bug, synchronization ClassLoader is tricked into loading external class as internal Bug in Bytecode Verifier can be exploited to circumvent SecurityManager Policy is too weak and allows damaging behavior

41. 19 Nov 2001University of Virginia CS 58841 Hostile Applets See http://java.sun.com/sfaq/chronology.html (about 1 new vulnerability/month) Easy to write “annoying” applets (policy is too imprecise; no way to constrain many resource operations) http://www.cigital.com/hostile- applets/index.html

42. 19 Nov 2001University of Virginia CS 58842 Proof-Carrying Code

43. 19 Nov 2001University of Virginia CS 58843 (From http://www.cedillasys.com/pages/tech/technotes/) Maze

44. 19 Nov 2001University of Virginia CS 58844 (From http://www.cedillasys.com/pages/tech/technotes/) Proof-Carrying Maze

45. 19 Nov 2001University of Virginia CS 58845 Proof-Carrying Code Guarantee properties of untrustworthy code by checking a proof provided by code producer Creating a proof is hard –Have to make up invariants, choose cases, pick strategies, etc. Checking a proof is easy –Simple mechanical application of rules

46. 19 Nov 2001University of Virginia CS 58846 Have we seen anything like this? Java Bytecode Verifier is a simple instance of PCC: –Bytecodes include extra information on typing, stack use, etc. –Bytecode verifier checks it to enforce low- level code safety properties

47. 19 Nov 2001University of Virginia CS 58847 Fortune Cookie “That which be proved cannot be worth much.” Fortune cookie quoted on Peter’s web page must can True for all users True for all executions Exception: Low-level code safety

48. 19 Nov 2001University of Virginia CS 58848 Reasons you might prefer PCC Run-time performance? –Amortizes additional download and verification time only rarely –SFI Performance penalty: ~5% If you care, pay $20 more for a better processor or wait 5 weeks Smaller Trusted Computing Base? –Not really smaller: twice as big as SFI (Touchstone VCGen+checker – 8300 lines / MisFiT x86 SFI implementation – 4500 lines) You are a vendor who cares more about quality than time to market –Not really PCC (not across a trust boundary)

49. 19 Nov 2001University of Virginia CS 58849 Feedback Slips

50. 19 Nov 2001University of Virginia CS 58850 You are getting a strip of paper with your recorded grades so far (check they are correct), and my recommendation for PS5/project: You’re doing fine (but keep working on your project). Enjoy your Thanksgiving. You’re doing okay, but go easy on the Cranberry Sauce. You should either do PS5 or work a lot on your project. You need to get cracking to avoid a mediocre grade (or worse) in this course. You should do PS5 and work on your project. No one got this. Still plenty of time to do well in the course.

51. 19 Nov 2001University of Virginia CS 58851 Movie