1. © 2007 ADP 1 Java Capacity Planning & Performance Measurements Dr. Carl J. De Pasquale carl_depasquale@adp.com Electronic Numerical Integrator and Computer Electronic Discrete Variable Automatic Computer

2. © 2007 ADP 2 Java Capacity Planning & Performance Measurements Agenda –Capacity vs. Performance –Java capacity planning Java black box –Tools Java application –Tools –Performance Measurements Coding standards Byte code instrumentation –Method performance measurements using management library –Method performance measurements using ARM & management library Tools –Conclusions

3. © 2007 ADP 3 Java Capacity Planning & Performance Measurements Hardware Does Not Always Solve Capacity Problems Capacity of a given hardware configuration is subject to the application constraints Capacity = f(Hardware, Application Performance) Assume a given hardware configuration –is 70% utilized, –Provides a 3 second application response, –When the arrival rate is two hundred transactions per second. –Management upgrades the hardware configuration with one that is twice as fast What could be the affect on the response time?

4. © 2007 ADP 4 Java Capacity Planning & Performance Measurements What could be the affect on the response time? Response Time Could Improve –If there isn’t an application CPU bottleneck –If there isn’t an application threading bottleneck –If there isn’t a network or I/O bottleneck Response Time Could Degrade –Work arrives faster Saturates non CPU peripherals CPU experiences I/O Waits –If there is an application threading bottleneck –If SQL is a poorly written More physical I/O executed than required table scans/small reads –If there is a Network Bottleneck Small packet size

5. © 2007 ADP 5 Java Capacity Planning & Performance Measurements Black box Java Capacity Planning –The Java Process CPU Disk Network Memory –No application visibility –Many java classes running in one JVM Application Capacity Planning –Process java Application Class/Method Data –CPU time by method –Elapsed time by method –Method invocation count –Memory usage heap –Workflow trace –SQL timing

6. © 2007 ADP 6 Java Capacity Planning & Performance Measurements Capacity Planning Black Box (java) - No Application Visibility –Collect process data –Identify busy period –Analyze data –Review reports to identify hot spots Computer CPU utilization Computer summary Disk IO summary Network summary Memory summary Process summary – Java will usually be the top resource consumer

7. © 2007 ADP 7 Java Capacity Planning & Performance Measurements Capacity Planning Black Box (java) - No Application Visibility –Identify and eliminate all bottlenecks (analytically) –Apply and monitor changes to physical infrastructure –Once bottlenecks are eliminated Recollect performance data Use Analytical modeling to help determine Capacity = f(Hardware, Application Performance) –Tools Opnet IT GURU Hyperformix Capacity Manager Team Quest Performance Assurance

8. © 2007 ADP 8 Java Capacity Planning & Performance Measurements Application Capacity Planning (java) - Application Visibility –Coding Standards Application Specific Log –Non standard format –Each application log must be parsed by capacity planner –May or may not contain necessary data –Better than nothing –Tools Byte Code Engineering Library (BCEL) Java ClassLoader

9. © 2007 ADP 9 Java Capacity Planning & Performance Measurements Sample Code public final class test { test () { } public void runtest () { for ( int i = 0 ; i < 9999999 ; i++ ) {} } public static void main(String[] argv) { try { for (int i = 0 ; i < 1; i++){ test t1 = new test(); t1.runtest(); for (int j = 0 ; j < 2 ; j++) new oohrah ( argv[0], argv[1] ) ; } } catch ( Exception e ) { e.printStackTrace(); } Sample Code public class oohrah { public oohrah( String a, String b ) { spin(); } public static void spin() { int i; new hello(); for (i=0; i < 500000000; i++); } public class hello { public hello( ) { printHello(); } public static void printHello() { System.out.println ("HelloWorld (from hello)"); }

10. © 2007 ADP 10 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation - JDK 1.5

11. © 2007 ADP 11 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation - JDK 1.5 Start Class Name=test,Method Name main Elapsed 1180537593616 CPU 515625000 Start Class Name=test,Method Name runtest Elapsed 1180537593632 CPU 515625000 End Class Name=test,Method Name runtest Elapsed 1180537593663 CPU 546875000 ClassName=test runtestCount==> 1 ClassName=test runtestCpu==> 31250000 ClassName=test mainCount==> 1 ClassName=test mainCpu==> 0 Start Class Name=oohrah,Method Name spin Elapsed 1180537593679 CPU 562500000 Start Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000 HelloWorld (from hello) End Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000 ClassName=hello printHelloCount==> 1 ClassName=hello printHelloCpu==> 0 End Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000 ClassName=oohrah spinCount==> 1 ClassName=oohrah spinCpu==> 1328125000 Start Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000 Start Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000 HelloWorld (from hello) End Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000 ClassName=hello printHelloCount==> 2 ClassName=hello printHelloCpu==> 0 End Class Name=oohrah,Method Name spin Elapsed 1180537596366 CPU 3203125000 ClassName=oohrah spinCount==> 2 ClassName=oohrah spinCpu==> 2640625000 End Class Name=test,Method Name main Elapsed 1180537596381 CPU 3218750000 ClassName=test runtestCount==> 1 ClassName=test runtestCpu==> 31250000 ClassName=test mainCount==> 1 ClassName=test mainCpu==> 2703125000

12. © 2007 ADP 12 Java Capacity Planning & Performance Measurements Application Capacity Planning (java) - Application Visibility ARM Instrument –Elapsed time –Invocation counts –Correlation –ArmTransactionsWithMetrics CPU time –Tools Byte Code Engineering Library (BCEL) Java ClassLoader

13. © 2007 ADP 13 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation - ARM

14. © 2007 ADP 14 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:34 06/13/07 transaction id=02000000 00000000 00000000 00000000 parent correlator: none flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.main transaction instance handle=1

15. © 2007 ADP 15 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:34 06/13/07 transaction id=04000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest transaction instance handle=2 arm_stop_transaction() time=03:09:34 06/13/07 transaction instance handle=2 transaction status=GOOD flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest

16. © 2007 ADP 16 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:34 06/13/07 transaction id=06000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin transaction instance handle=3

17. © 2007 ADP 17 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:34 06/13/07 transaction id=08000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition oohrah app handle=3 tran name=TransactionDefinition oohrah.spin tran instance id=3 flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello transaction instance handle=4

18. © 2007 ADP 18 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_stop_transaction() time=03:09:34 06/13/07 transaction instance handle=4 transaction status=GOOD flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello arm_stop_transaction() time=03:09:35 06/13/07 transaction instance handle=3 transaction status=GOOD flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin

19. © 2007 ADP 19 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:35 06/13/07 transaction id=0a000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin transaction instance handle=5

20. © 2007 ADP 20 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_start_transaction() time=03:09:35 06/13/07 transaction id=0c000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition oohrah app handle=5 tran name=TransactionDefinition oohrah.spin tran instance id=5 flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello transaction instance handle=6

21. © 2007 ADP 21 Java Capacity Planning & Performance Measurements Dynamic Byte Code Instrumentation – ARM arm_stop_transaction() time=03:09:35 06/13/07 transaction instance handle=6 transaction status=GOOD flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello arm_stop_transaction() time=03:09:35 06/13/07 transaction instance handle=5 transaction status=GOOD flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin arm_stop_transaction() time=03:09:35 06/13/07 transaction instance handle=1 transaction status=GOOD flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.main

22. © 2007 ADP 22 Java Capacity Planning & Performance Measurements Analyzing the Raw Data

23. © 2007 ADP 23 Java Capacity Planning & Performance Measurements Operational Analysis – Method Raw Data Process

24. © 2007 ADP 24 Java Capacity Planning & Performance Measurements Operational Analysis – Overloading Process  Method Method  Process Valid for Black box, and application capacity planning Denning, and J. Buzen, The operational analysis of queuing network models. Computer Surveys, 10(3), 225 - 261 (1978)

25. © 2007 ADP 25 Java Capacity Planning & Performance Measurements Third Party Performance Analysis Tools -Unable to implement custom application logging -Unable to implement Dynamic Byte Code Instrumentation -Third party alternatives Mercury Opnet Panorama/ACE Quest Perform Assure Wily

26. © 2007 ADP 26 Java Capacity Planning & Performance Measurements Conclusion By adding a bit of sophistication during application development, performance testing, or production deployment the often overlooked non-functional requirement of “application performance” requirement can be addressed

27. © 2007 ADP 27 Java Capacity Planning & Performance Measurements Thank You for Attending – Have a nice rest of the day Questions Comments Discussion