Security: Lessons Learned and Missed from Java Nathanael Paul David Evans University of Virginia ACSAC 2004

Java VM.NET VM Major Security Vulnerabilities (Cumulative)

3 Why the disparity in vulnerabilities? Hypotheses: No one uses/attacks.NET –Windows Update installs.NET framework –Attractive target with over 90% market share Microsoft is smarter than everyone else –Check their profit and market share Learned from past –.NET learned from experience with Java

4 Universal Security Principles [Saltzer and Shroeder, 1974] [McGraw and Viega, 2001] Keep it simple Complete Mediation Least Privilege Secure Weakest Link Defense in Depth

5 Virtual Machines Platforms that allow untrusted code to execute with restrictions enforced by the virtual machine (VM)

6 Source bytecodes Verifier Low-level Code Safety Must ensure programs are type, memory, and control safe using data-flow analysis High-level policy enforcement depends on low-level code safety VM

7 Verifier is (should be) Conservative.NET/Java programs Safe programs Verifiable programs Bug

8 Object Creation and Initialization Virtual machine must ensure object is initialized before use –Security permissions restrict some objects from being created –Improper initialization can create a vulnerability Bug in MSIE 4.0, 5.0, 6.0 [lsd-pl.net] Similar bug in Sun and Netscape Lesson 1: Keep it simple

9 Java –new – create new object reference –dup – duplicate reference –invokespecial – calls constructor.NET –newobj is equivalent to Java’s new, dup, and invokespecial instructions Object Creation Instructions

10 Object Initialization Vulnerability [lsd-pl.net] class LSDbug extends SecurityClassLoader { public LSDbug() { try { LSDbug(5); } catch (SecurityException e) { this.loadClass(…); } } public LSDbug (int x) { super(); // throws Security Exception }

11 Bootstrapping the VM Need to bootstrap the virtual machine Certain classes providing policy enforcement need full trust –Infinite recursion if checks needed on all classes Lesson 2: Least Privileges

12 Bootstrapping the VM Java 1.0 –Fully trusted code on CLASSPATH –Current Java versions have bootclasspath for backwards compatibility.NET’s trusted path is a cache of signed files

13 Location-based Vulnerability [Hopwood, 1996] Netscape cached files on local filesystem Guessing cached file names could allow arbitrary code execution Applet could execute cached files located on CLASSPATH

14 Monitoring Execution Lesson 3: Fail-safe Defaults and Complete Mediation

15 Monitoring Execution Want policy extensible but complicates policy enforcement –Java 1.0 (HotJava) and 1.1 had all or nothing trust for applets Reference monitor should be tamper-proof and always be invoked

16 Reference Monitor’s Enforcement Java’s reference monitor, the SecurityManager may be bypassed SecurityManager sm = System.getSecurityManager(); if (sm != null) { sm.checkListen(21); // listen on port 21? }.NET’s SecurityManager cannot be inherited or instantiated

17 Failure to Monitor Vulnerability [Brumleve, 2000] SecurityManager.checkListen() allows creation of a ServerSocket object Flaw in ServerSocket.implAccept(Socket s) –Accepts connection to get remote address and port number –Calls socket’s close() and throws SecurityException if permissions violated –Subclass of Socket can override close() to keep socket open

18 Principles Review Keep it simple –Object initialization –.jsr/swap vulnerability (see paper) Least privileges –Bootstrapping the VM –Stack Inspection Fail-safe Defaults and Complete Mediation –Brown Orifice –DoS attacks –Union/Intersection in Policy Resolution

19 Conclusions Classic security principles still important today Hard to follow them in real systems –Easier to find complex solutions than simple ones –Tradeoffs between security and other goals Complete Mediation vs. Efficiency (policy expressiveness) Simplicity vs. Backwards compatibility ( bootclasspath ) Fail-safe defaults vs. Usability (Default Policies) Some reasons for optimism

20 Questions ?

21 Conclusions Why do we still have problems today? –Security vs. Efficiency –Defense in Depth vs. Simplicity [McGraw, Viega] –Flexibility vs. Simplicity Evaluate principles in context [McGraw, Viega]

22 Object Initialization Vulnerability [lsd-pl.net] ()LSDbug → (I)LSDbug → com/ms/security/SecurityClassLoader/ ()LSDbug Security exception occurs (caught by ()LSDbug) since code does not have permission to instantiate ClassLoader

23 Granted Permissions in Policies Permissions are granted, not excluded Java’s policy is the union of all granted permissions.Net policy is the intersection of a 4-level hierarchical policy –Enterprise –Machine –User –AppDomain Lesson 3: Fail-safe defaults in Permission Resolution

24 Static/Dynamic Permissions Policy enforcement can be optimized –Need flexibility Static permissions –Must be known before run-time –Faster checking possible Dynamic –Can change on-the-fly –Checks delayed until run-time Lesson 3: Fail-safe Defaults

25 Policy Implementation: Static/Dynamic Permissions Granted in class loaders (e.g., AppletClassLoader) Attached to assemblies and can be checked before run-time Union of all permissions in policy files Intersection of permissions in policy files Static Dynamic Java. Net

26 notes Emphasize overall point (talk of analysis of lessons learned… one sentence – slide 2) Pointer Don’t flip between overall pic Make sure point out vulnerability is on Java Wrap up each section (at end of vulnerability) better Have better transitions Mention a couple more of the s & s principles Look more at audience Point out no significant security vulnerabilities in.Net (double check)

27 notes More principles –Defense in depth –Chain is only strong as weakest link –Secure failure (not seen in Java’s exceptions!) –Compartmentalization –Choke points (narrow interface to system) –Usability –Trust community (open design crytpo) –No security through obscurity –Educate user

28 Object Initialization Vulnerability [lsd-pl.net] LSDbug child class of SecurityClassLoader Call constructor, call constructor, call superclass constructor (exception occurs) new dup invokespecial LSDbug() … invokespecial LSDbug(int) … invokespecial SecurityClassLoader()

29 Object Initialization Vulnerability [lsd-pl.net] MSIE 4.0, 5.0, 6.0 Create object of a security-critical class to escalate privileges Similar bug in Sun and Netscape implementations

30 Verifier is (should be) Conservative.NET/Java programs Safe programs Verifiable programs

31 Complexity Increases Risk.NET/Java programs Safe programs Verifiable programs Bug