6/12/20011 KaffeOS: Isolation, Resource Management and Sharing in Java Godmar Back School of Computing University of Utah Dissertation Defense

Dissertation Defense 6/12/ Motivation l Java is widely used –Examples: applets, servlets, Enterprise Java Beans , mobile agents, active packets, database procedures –Untrusted code: possibly malicious or buggy –Multiple applications on behalf of multiple users l Primary motivation  Want to provide robust environment for these applications l Secondary motivation –Make efficient use of resources; increase scalability –Run on “small” systems (handhelds, embedded systems)

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Java Applications Applet / Agent / Servlet / Database Procedure … JVM Base OS This work is about system structure.

Dissertation Defense 6/12/ Current Options - Single JVM with ad-hoc layer -Efficient sharing App 1 App 2 App 3 Ad hoc layer Base OS JVM App 1 App 2 App 3 JVM Base OS - Multiple JVMs -Strong isolation -Good resource management

Dissertation Defense 6/12/ Java Operating System App 1 App 2 App 3 App 4 Java OS Base OS + Good isolation + Good resource management + Efficient sharing  requires process model

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ KaffeOS Core Ideas To provideKaffeOS uses Isolation and Safe Termination Red Line Memory ManagementSeparate Heaps Interprocess Communication Shared Heaps Flexible Resource Management Hierarchical Resource Framework

Dissertation Defense 6/12/ The Red Line in Hardware-based OS l MMU prevents access to kernel data structures in user mode l System calls enter kernel mode Cheriton 1994

Dissertation Defense 6/12/ The Red Line for Safe Termination l Isolation of applications and system: user/kernel boundary –Kernel classes must not be corrupted –Termination is deferred while inside kernel code –All exceptional situations are handled in kernel code NB: different from built-in synchronize

Dissertation Defense 6/12/ class Queue { synchronized Object get(); synchronized void put(Object); } Queue producer, consumer; void run () { while (!done) { Object obj = producer.get(); consumer.put(obj); } l Producer/Consumer Example –Invariant: all produced items are consumed Guaranteeing Consistency Kernel.enter(); Kernel.leave();

Dissertation Defense 6/12/ Red Line in KaffeOS User/kernel dimension is different from trust dimension: User GC System GC User code (untrusted) Kernel code (trusted) Runtime Libs (trusted) Finalization User Kernel User modeKernel mode TrustedRuntime codeKernel code UntrustedUser code

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Limit memory use of processes 2.Fully reclaim processes’ memory after termination 3.Garbage collect processes individually Memory Management: Goals

Dissertation Defense 6/12/ Single Heap (Conventional JVM) Reference Object

Dissertation Defense 6/12/ Separate Heaps Kernel Heap User Heaps Goal 1: Limit memory use

Dissertation Defense 6/12/ Reclaiming Memory l References can make objects unreclaimable Hardware-based OS MMU maps virtual memory OS manages physical pages Java runtime system No MMU Garbage Collector frees unreachable objects

Dissertation Defense 6/12/ Legal and Illegal References Legal reference Illegal reference Goal 2: Full reclamation

Dissertation Defense 6/12/ Preventing Illegal References l Use GC write barriers –Technique used in incremental and generational collection l Intercept each write, check heaps –Throw SegmentationViolationError if illegal –Only PUTFIELD, PUTSTATIC, and AASTORE bytecode instructions have to be checked l Small cost on legal write (common case)

Dissertation Defense 6/12/ Use of Entry and Exit Items Entry Item n nn x x x x Exit ItemRefCount 1 12 Goal 3: Separate Collection

Dissertation Defense 6/12/ Collecting Cycles l Merge heaps upon termination into kernel heap

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ IPC: Direct Sharing l Alternatives: –No sharing (copy between processes) –Indirect sharing (use proxy objects or surrogates) l Objects contain direct pointers to shared objects –Preserve Java model l Should not compromise accurate accounting or full reclamation!

Dissertation Defense 6/12/ Shared Heaps Legal reference Illegal reference User Heap Kernel Heap Shared Heap User Heap

Dissertation Defense 6/12/ Shared Heaps: Model l Accounting of shared objects –Shared heap’s size is fixed after creation –Sharers are all charged for shared heaps: double charging l Reclamation –As soon as garbage collector detects that nothing on shared heap is referenced l Restrictions on programming model –Cannot allocate new objects after heap is frozen –Cannot have references to user heaps

Dissertation Defense 6/12/ Shared Heaps: Trie Example package shared; class Trie { static class TrieNode { final static int WIDTH = 96; TrieNode [] children = new TrieNode[WIDTH]; boolean eow = false; } private TrieNode root; private int maxlength; Trie(Reader r) throws IOException { … } public void dump(PrintStream s) { … } public boolean lookup(String s) { … } } import shared.Trie; Shared sh = Shared.registerClass("shared.Trie", 1); Trie trie = (Trie) sh.getObject(0); … trie.lookup(“aback”); A A B C A B C A R O N K F T A

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design –Isolation and Safe Termination –Memory Management –Interprocess Communication –Flexible Resource Management l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Hierarchical Resource Framework l Resource API to allow for different policies l Hierarchical management l Work-conserving

Dissertation Defense 6/12/ CPU and Memory Allocation l Memory –Must not require partitioning –Soft limits: Limit overall use, allow temporary use of surplus memory –Hard limits: Provide memory guarantees if desired l CPU –Stride-scheduling for processes as a whole –Priority-based scheduler for threads in process

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation –Performance for well-behaved code –Performance for adverse loads l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Performance for Well-behaved Code l Baseline performance l Write barrier overhead –Microbenchmarks –SpecJVM98 benchmarks l Compared against –Base JVM: Kaffe (Kaffe00 from June 2000, –IBM JDK – fastest industrial 1.1 JVM l Microbenchmarks –Instruction cost: »11 cycles minimum with one word overhead per object, 43 cycles minimum with no memory overhead

Dissertation Defense 6/12/ SpecJVM Performance of KaffeOS

Dissertation Defense 6/12/ Write Barrier Counts Benchmark Dynamic Count (millions) Write Barrier Overhead (Best-case) for No Heap Pointer Conservative upper bound for IBM JDK compress % (0.0%)0.7% jess % (1.0%)10.5% db % (3.8%)8.1% javac % (2.5%)7.3% mpegaudio % (0.8%)-1.7% mtrt % (0.5%)-0.2% jack % (1.9%)14.1%

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation –Performance for well-behaved code –Performance for adverse loads l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Performance for adverse loads l DOS Scenarios against –Memory –CPU –Garbage collector l KaffeOS –is more robust –delivers more effective service –improves performance under adverse loads

Dissertation Defense 6/12/ DoS Scenario l Servlet Engine: Apache , JServ 1.1, JSDK 2.0 l Some number of “good” servlets l Plus one MemoryHog servlet –Allocate lots of memory –Try to crash/tie up JVM l How would we deal with this? –Run 1 servlet per IBM JVM (IBM/1) –Run all servlets on one IBM JVM (IBM/n) –Run each servlet in a KaffeOS process

Dissertation Defense 6/12/ Service Under DoS Attack

Dissertation Defense 6/12/ MemHog Stresstest

Dissertation Defense 6/12/ CPU Denial of Service Scenario l Problem: cannot tell a CPU denial-of-service attack from a process doing useful work l Instead: ensure that processes obtain their share when runnable l Allow for manipulation of shares to reflect user priorities l MD5 Servlet computes MD5 hash sum over /usr/dict/word for each request –Requests/sec is measure of service

Dissertation Defense 6/12/ CpuHog Scenario

Dissertation Defense 6/12/ GarbageHog Scenario

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Related Java Work l Java VMs –Multi-processing JVM [Balfanz 98] –J-Kernel [Hawblitzel et al. 98] –Alta [Tullmann 99] –IBM OS/390 JVM [Dillenberger 00] –MVM [Czajkowski 01] –JSR 121 l Java Language Extensions –Luna [Hawblitzel 99] l Java Realtime Extensions –Realtime Working Group [Sun et al. 00]

Dissertation Defense 6/12/ Related OS Work l Single-address-space OS –Opal [Chase et al. 94] l Single-language OS –Pilot [Redell et al. 80] –Cedar [Swineheart et al. 86] –Inferno [Dorward et al. 97] l Extensible OS –SPIN [Bershad et al. 95]

Dissertation Defense 6/12/ Outline l The Case for KaffeOS l KaffeOS’s Design l Experimental Evaluation l Related Work l Future Work and Conclusion

Dissertation Defense 6/12/ Future Work l JanosVM (ongoing) –Java operating system for active network node l Integration in higher-performing JVM –More accurate evaluation of costs and benefits l Static analysis using meta-level compilation –Ensure properties about user/kernel mode, shared and reloaded classes etc. statically l Automatic user-level sharing

Dissertation Defense 6/12/ Contributions l Operating system principles apply to language runtime systems –Architectural concept: user/kernel boundary l Software mechanisms can be used to implement full isolation of processes in a Java virtual machine –GC mechanisms: write barriers, distributed GC techniques l KaffeOS demonstrates how to reconcile isolation, resource management and sharing –Can stop resource-based denial-of-service attacks with small penalty for well-behaved code

Dissertation Defense 6/12/ Acknowledgments l Wilson Hsieh l Patrick Tullmann l Jason Baker l Tim Stack l Jay Lepreau and rest of Flux group for support and feedback on earlier papers

Dissertation Defense 6/12/ The End l Time for more Questions

Dissertation Defense 6/12/ Soft and Hard Limits l Hard limits: –are debited in full from parent –provide memory guarantees l Soft limits: –only actual use is debited –limit overall memory use –allow siblings to use surplus memory 64 MB 60 MB Limit Use 48 MB 44 MB 48 MB 30 MB 48 MB 14 MB roothardsoft 16 MB 9 MB

Dissertation Defense 6/12/ CPU Scheduling l Stride scheduler for processes l Priority-based scheduler for threads in process Root 100%60%10%20%40% Thread Priority

Dissertation Defense 6/12/ MemHog (Without User/kernel)

Dissertation Defense 6/12/ Summary l Java operating systems are necessary to support untrusted Java applications l KaffeOS –Architectural concepts taken from operating systems –Mechanisms taken from garbage collection »Distributed GC »Write barriers l Big issue: dealing with memory/GC l Resource-based denial-of-service attacks can be stopped

Dissertation Defense 6/12/ The KaffeOS Java Operating System l Separation –Processes do not interfere with each other –Safe termination l Resource management –Resource consumption can be limited –Recover all resources when an application dies –Account for shared resources reasonably l Direct sharing –Interprocess communication can be efficient –Preserve the Java model

Dissertation Defense 6/12/ What Is a Java Operating System? l Execution environment for Java bytecode –Support for multiple applications: operating system –Implemented fully in software: no hardware support »Can be embedded in another application »Does not depend on underlying OS for features l Not a “single-language OS” –OS does not have to be written entirely in Java –Does not need to be standalone

Dissertation Defense 6/12/ J-Kernel l Processes are called tasks –Separate namespaces l Pure Java –portability across JVMs l Indirect sharing –capabilities –bytecode rewriting to support automatic marshaling/unmarshaling for remote calls –thread migration –no direct cross-process references –gc is shared

Dissertation Defense 6/12/ Alta l Modeled after Fluke microkernel [Ford 96] –process hierarchy, nesting l Hierarchical management –parent controls resource consumption of children –Siblings can share objects directly l Common heap –parent must judiciously control childrens communication to avoid cross-heap refs –GC is shared service

Dissertation Defense 6/12/ K0 l Modeled after monolithic kernel l Separation –separate heaps with separate garbage collection –write barrier to avoid cross-heap references l Clear user/kernel separation l Direct sharing of objects –restricted to enable full reclamation

Dissertation Defense 6/12/ Copying  Simple accounting  Full reclamation  Requires serialization of objects  Can be slow P1P1 P2P2

Dissertation Defense 6/12/ Indirect Sharing  Revocable proxies  Full reclamation  Method invocation requires copying P1P1 P2P2

Dissertation Defense 6/12/ Direct Sharing  Separate heaps  Dedicated shared heap  Full reclamation  Direct sharing is restricted  Requires monitoring of store instructions P1P1 P2P2

Dissertation Defense 6/12/ Hybrid  Supports both direct and indirect sharing  More flexible  Direct references can prevent full reclamation  Method invocation may require copying P1P1 P2P2 Parent

Dissertation Defense 6/12/ Comparison RTJ and KaffeOS l Different goals and assumptions l Assumptions: –KaffeOS: user code is untrusted, kernel code is trusted –RTJ: all code is trusted (embedded environment) l Primary Goals: –KaffeOS: guard against malicious or buggy applications –RTJ: provide predictable (bounded, “real-time”) performance

Dissertation Defense 6/12/ ScopedMemory vs. Separate Heaps l Both –control pointer assignments –can be limited: resource limits vs. budgeted allocation –can allow for bounded allocation (KaffeOS hard resources vs. LTMemory) l RTJ ScopedMemory –is not garbage collected, does not allow any cross-heap refs –threads can enter/leave scopes l KaffeOS Heaps –are garbage collected (exception: shared heaps!) –GC time is subject to process scheduling –allows cross-heap refs where needed –threads cannot switch heaps (Janos threads can)

Dissertation Defense 6/12/ Memory and CPU Accounting l Accurate and complete per-process accounting –Minimize number of objects on kernel heap –Minimize amount of time spent in kernel code l Separate garbage collection –Minimize unaccounted resource use –Avoid priority inversion –Maintain isolation

Dissertation Defense 6/12/ Code Structure User GC System GC User code (untrusted) Kernel code (trusted) Runtime Libs (trusted) Finalization User Kernel Java C

Dissertation Defense 6/12/ User/Kernel vs. AIEs l Both –Asynchronously affect a thread’s execution –Are deferred during certain sections of code l RTJ: –Thread decides when it’s safe to receive AIEs –AIEs disabled during synchronized { } and constructors –Limited compiler support l KaffeOS –Only kernel can defer termination –User can be terminated any time – system consistency is unaffected –Compiler support future work

Dissertation Defense 6/12/ l Or: can you have your cake and eat it too? Are AIEs useful? class Queue { synchronized Object get(); synchronized void put(Object); } Queue producer, consumer; void run () throws AsynchronouslyInterruptedException { while (true) { Object o = producer.get(); /* is it safe to terminate here? */ consumer.put(o); /* how about here? */ }

Dissertation Defense 6/12/ Using synchronized (I) l Decreased parallelism l More deadlocks void run () throws AsynchronouslyInterruptedException { while (true) { synchronized { Object o = producer.get(); consumer.put(o); }

Dissertation Defense 6/12/ Using synchronized (II) l Blurs semantics of “synchronized” l Error-prone and inefficient l Synchronization != Deferred termination void run () throws AsynchronouslyInterruptedException { while (true) { synchronized (new Object()) { Object o = producer.get(); consumer.put(o); }

Dissertation Defense 6/12/ void poll() throws AsynchronouslyInterruptedException {}; void run () { while (true) { Object o = producer.get(); consumer.put(o); try { poll(); } catch (AsynchronouslyInterruptedException) { return; } Polling l But: polling is better done explicitly instead of using “magic” – optimize for readability!

Dissertation Defense 6/12/ Better polling l No AIEs needed l Question: is it possible to write large pieces of code that accesses shared state and can cope with asynchronous event notification? l If no: deferred delivery provides a false sense of security void run () { while (!stopped) { Object o = producer.get(); consumer.put(o); }

Dissertation Defense 6/12/ KaffeOS vs. RTJ l Very similar mechanisms, very different goals l KaffeOS implementation base could easily provide ScopedMemory –Mechanisms in place (write barriers, separate allocators etc.) l Question: will RTJ be enough for embedded systems? Or will they too need OS structure to isolate applications?