Jarle Hulaas Swiss Federal Institute of Technology in Lausanne (EPFL) Switzerland (Formerly at CUI, Uni Geneva) Dimitri Kalas University of Geneva Switzerland Monitoring of resource consumption in Java-based application servers 10th HP-OVUA workshop, Geneva, Switzerland July 7, 2003

7/7/ th HP-OVUA workshop, Geneva, Switzerland 2 Overview What is Resource Management ? Our General Approach Some Details on our J-RAF Tool Management of a Servlet Engine Adapting J-RAF to Tomcat Assessment

7/7/ th HP-OVUA workshop, Geneva, Switzerland 3 What is Resource Management ? Physical resources: CPU (Virtual) Memory Network bandwidth Management of resources: Monitoring of consumption Controlling (i.e. limiting) it

7/7/ th HP-OVUA workshop, Geneva, Switzerland 4 Benefits of Resource Management Software development Monitoring and profiling of distributed applications Security Prevention against Denial-of-Service attacks Reliability More robust behaviours in presence of resource shortage Accountability (liability) for resource consumption E-commerce Billing for resource consumption Quality-of-Service guarantees, load balancing

7/7/ th HP-OVUA workshop, Geneva, Switzerland 5 What about Java ? Problem: Java offers no support for resource management ! Our solution: portable resource management through byte-code rewriting schemes The Java Resource Accounting Framework, 2nd edition

7/7/ th HP-OVUA workshop, Geneva, Switzerland 6 Rewrite bytecode General Approach Execute Original Java Application (compiled) Transformed, resource-aware version of Application Per-thread accounting objects Net CPUMem Meta-level Execution hooks

7/7/ th HP-OVUA workshop, Geneva, Switzerland 7 Our Solution Program transformations to expose resource consumption Application classes and libraries, incl. JDK Rewritten programs unknowingly update accounting objects CPU: count the number of executed JVM instructions Memory: redirection of memory allocations Network bandwidth: wrapper libraries

7/7/ th HP-OVUA workshop, Geneva, Switzerland 8 Class Analysis for CPU monitoring bipush 10 2 istore_1 3 iconst_1 6 istore_3 7 goto invokestatic #2 13 pop 14 ldc2_w #3 17 bipush invokestatic #5 22 iinc iload_3 26 iconst_1 27 if_icmple iload_2 31 iconst_1 32 if_icmpl iinc goto astore_2 42 iconst_0 43 istore 3 44 return Exception table: from to target type (size=2) 2 (size=3) 3 (size=6) 4 (size=3) 5 (size=3) 6 (size=2)7 (size=3) 8 (size=1)

7/7/ th HP-OVUA workshop, Geneva, Switzerland 9 Method Rewriting by J-RAF Object f (int x, MemAccObj mem, CPUAccObj cpu) { cpu.usage += 8; if (x < 0) { cpu.usage += 8; return null; } else { cpu.usage += 26; int y = g(x, mem, cpu); mem.checkAllocation(SIZEOF_FOO); Object o = new Foo(y, mem, cpu); mem.register(o); return o; }

7/7/ th HP-OVUA workshop, Geneva, Switzerland 10 Generic Application Structure Bytecode Rewriting Low-level Resource Acc&Control (Memory, CPU) High-level Resource Accounting & Control Resource-aware Application JVM Native Code Resource-aware JDK

7/7/ th HP-OVUA workshop, Geneva, Switzerland 11 J-RAF in Action CPU monitoring for some multi- threaded applet

7/7/ th HP-OVUA workshop, Geneva, Switzerland 12 Validation of the J-RAF approach Tested on J2SE … arbitrary applications, as well as applet and mobile agent environments Sun HotSpot Virtual Machine, IBM JDK … and on J2ME (embedded devices) … dedicated system with Java processor from aJile Systems

7/7/ th HP-OVUA workshop, Geneva, Switzerland 13 Benchmarks for CPU accounting SPEC JVM98

7/7/ th HP-OVUA workshop, Geneva, Switzerland 14 Resource Mgmt for Servlet Engines Reporting Requests HTML, streams Access rights Servlet Engines Providers Database End-user Resource Mgmt for Servlet Engines

7/7/ th HP-OVUA workshop, Geneva, Switzerland 15 Objectives Fine-grained basis for: Usage-based billing (pricing policies are out of our scope) Identification of bottlenecks Badly programmed or malicious apps Load-balancing on clusters of servers

7/7/ th HP-OVUA workshop, Geneva, Switzerland 16 Required J-RAF extensions Define a servlet as a coherent set of threads Flexible concept of client (end-user) context Enhanced detection of bandwidth consumption Remote reporting

7/7/ th HP-OVUA workshop, Geneva, Switzerland 17 Servlets as sets of threads How to detect the start and end of a request processing ? Special rewriting of methods like doGet and doPost, to cope with Tomcats thread pooling Register and account for created sub- threads Add a RequestAccount class for aggregating usage of all these threads

7/7/ th HP-OVUA workshop, Geneva, Switzerland 18 User-level accounting Periodically do: Summarize usage of all individual threads inside a servlet Summarize usage of all servlets run on behalf of a given user Report this usage to a database Problems that had to be solved: Overflow of counters at all levels Calibration of timeslices according to processing power

7/7/ th HP-OVUA workshop, Geneva, Switzerland 19 User context Allow servlet programmers to define their own: Authentication schemes, IP-based or password-based Programmer must implement a ClientID interface Read usage rights from database User data reporting schemes Allow extensible reporting of user data, e.g. session IDs

7/7/ th HP-OVUA workshop, Geneva, Switzerland 20 Resource-aware Servlet Engine Resulting Servlet Structure Bytecode Rewriting Low-level Resource Acc&Control (Memory, CPU) High-level Resource Accounting & Control Resource-aware Servlet JVM Native Code Resource-aware JDK

7/7/ th HP-OVUA workshop, Geneva, Switzerland 21 Sample Log for a Given Client Servlet nameTimestampJVM ops/sIn+Out Bandwidth [Bytes/s]

7/7/ th HP-OVUA workshop, Geneva, Switzerland 22 Performance Testing Setting: Pentium-class PCs in a 100Mb LAN, Tomcat 4.1, MySQL Generate many simultaneous connections with Jmeter Different test servlets: 1. Output of an empty page 2. Streaming of factorials 3. Download, ZIP-compress, and output a web page

7/7/ th HP-OVUA workshop, Geneva, Switzerland 23 Preliminary Performance Results Simultaneous requests Empty page Streaming factorials ZIP page 1700 %84.9 %10.5 % %23.9 %20.7 % %22.8 %30.8 % Overhead due to CPU monitoring Average overhead: same order of magnitude as for standard Java applications (~ 30 %)

7/7/ th HP-OVUA workshop, Geneva, Switzerland 24 Cons Purely a Java-level framework: Difficult to account for resources consumed by native code No real-time / resource reservation capabilities Increase of code size Restrictions on what servlets may do to play with thread priorities to perform introspection to inspect or modify accounting objects

7/7/ th HP-OVUA workshop, Geneva, Switzerland 25 Conclusions Initial work towards fine-grained management of Java servers Reasonable overheads Future work: Finalize management of memory Improve control vs monitoring Support for clusters of servers

7/7/ th HP-OVUA workshop, Geneva, Switzerland 26 Thank you for your attention !