1. uPortal Performance Optimization Faizan Ahmed Architect and Engineering Group Enterprise Systems & Services RUTGERS faizan@rutgers.edu@rutgers.edu

2. 6/15/20152 GC History  Garbage Collection (GC) has been around for a while (1960’s LISP, Smalltalk, Eiffel, Haskell, ML, Scheme and Modula-3)  Went mainstream in the 1990’s with Java (then C#)  JVM implementations have improved on GC algorithms/speed over the past decade

3. 6/15/20153 GC Benefits/Cost  Benefits  Increased reliability  Decoupling of memory mgmt from program design  Less time spent debugging memory errors  Dangling points/memory leaks do not occur (Note: Java programs do NOT have memory leaks; “unintentional object retention” is more accurate)  Costs  Length of GC pause  CPU/memory utilization

4. 6/15/20154 GC Options  Sun’s 1.3 JDK included 3 GC strategies  1.4 JDK includes 6 and over a dozen command line options  Application type will demand strategy:  Real-time – short and bounded pauses  Enterprise – may tolerate longer or less predictable pauses in favor of higher throughput

5. 6/15/20155 GC Phases  GC has two main phases:  Detection  Reclamation  Steps are either distinct:  Mark-Sweep, Mark-Compact  … or interleaved  Copying

6. 6/15/20156 Fundamental GC Property “When an object becomes garbage, it stays garbage.”

7. 6/15/20157 Reachability  Roots – reference to object in a static variable or local variable on an active stack frame  Root Objects – directly reachable from roots  Live Objects – all objects transitively reachable from roots  Garbage Objects – all other objects

8. 6/15/20158 Reachability Example Runtime Stack ref Heap   

9. 6/15/20159 GC Algorithms  Two basic approaches:  Reference Counting – keep a count of references to an object; when a count of zero, object is garbage  Tracing – trace out the graph of objects starting from roots and mark; when complete, unmarked objects are garbage

10. 6/15/201510 Reference Counting  An early GC strategy  Advantages:  can be run in small chunks of time  Disadvantages:  does not detect cycles  overhead in incrementing/decrementing counters  Reference Counting is currently out-of favor

11. 6/15/201511 Tracing  Basic tracing algorithm is known as mark and sweep  mark phase – GC traverses reference tree, marking objects  sweep phase – umarked objects are finalized/freed

12. 6/15/201512 Mark-Sweep Example Root  1. mark-sweep start  Root  2. end of marking Root 3. end of sweeping

13. 6/15/201513 Mark-Sweep  Problem is fragmentation of memory  Two strategies include:  Mark-Compact Collector – After mark, moves live objects to a contiguous area in the heap  Copy Collector – moves live objects to a new area

14. 6/15/201514 Mark-Compact Collector Example  Root  1. end of marking Root 2. end of compacting

15. 6/15/201515 Copy Collector Example Root  1. copying start Root 2. end of copying Unused From To

16. 6/15/201516 Performance Characteristics  Mark-Compact collection is roughly proportional to the size of the heap  Copy collection time is roughly proportional to the number of live objects

17. 6/15/201517 Copying  Advantages:  Very fast…if live object count is low  No fragmentation  Fast allocation  Disadvantages:  Doubles space requirements – not practical for large heaps

18. 6/15/201518 Observations  Two very interesting observations:  Most allocated objects will die young  Few references from older to younger objects will exist  These are known as the weak generational hypothesis

19. 6/15/201519 Generations  Heap is split into generations, one young and one old  Young generation – all new objects are created here. Majority of GC activity takes place here and is usually fast (Minor GC).  Old generation – long lived objects are promoted (or tenured) to the old generation. Fewer GC’s occur here, but when they do, it can be lengthy (Major GC).

20. 6/15/201520 Sun HotSpot Heap Layout YoungOldPermanent EdenFromTo Survivor Spaces

21. 6/15/201521 Before Minor GC  OldPermanent EdenFromTo Survivor Spaces   Unused  Young

22. 6/15/201522 After Minor GC Empty OldPermanent Eden ToFrom Survivor Spaces Unused Young

23. 6/15/201523 GC Notes  Algorithm used will vary based on VM type (e.g., client/server)  Algorithm used varies by generation  Algorithm defaults change between VM releases (e.g., 1.4 to 1.5)

24. 6/15/201524 Young Generation Collectors  Serial Copying Collector  All J2SEs (1.4.x default)  Stop-the-world  Single threaded  Parallel Copying Collector  -XX:+UseParNewGC  JS2E 1.4.1+ (1.5.x default)  Stop-the-world  Multi-threaded  Parallel Scavenge Collector  -XX:UseParallelGC  JS2E 1.4.1+  Like Parallel Copying Collector  Tuned for very large heaps (over 10GB) w/ multiple CPUs  Must use Mark-Compact Collector in Old Generation

25. 6/15/201525 Serial vs. Parallel Collector stop-the-world pause Serial Collector Parallel Collector

26. 6/15/201526 Old Generation Collectors  Mark-Compact Collector  All J2SEs (1.4.x default)  Stop-the-world  Single threaded  Train (or Incremental) Collector  -Xincgc  About 10% performance overhead  All J2SEs  To be replaced by CMS Collector  Concurrent Mark-Sweep (CMS) Collector  -XX:+UseConcMarkSweepGC  J2SE 1.4.1+ (1.5.x default (-Xincgc))  Mostly concurrent  Single threaded

27. 6/15/201527 Mark-Compact vs. CMS Collector stop-the-world pause Mark-Compact Collector Concurrent Mark-Sweep Collector initial mark concurrent marking remark concurrent sweeping

28. 6/15/201528 JDK 1.4.x Heap Option Summary Low Pause CollectorsThroughput CollectorsHeap SizesGeneration Young Old Permanent 1 CPU2+ CPUs Serial Copying Collector (default) Parallel Copying Collector -XX:+UseParNewGC 1 CPU2+ CPUs Copying Collector (default) Parallel Scavenge Collector -XX:+UseParallelGC -XX:+UseAdaptiveSizePolicy -XX:+AggressiveHeap -XX:NewSize -XX:MaxNewSize -XX:SurvivorRatio Mark- Compact Collector (default) Concurrent Collector -XX:+UseConcMarkSweepGC Mark- Compact Collector (default) -Xms -Xmx Can be turned off with –Xnoclassgc (use with care) -XX:PermSize -XX:MaxPermSize

29. 6/15/201529 Heap Tuning  Analyze application under realistic load  Run with –verbose:gc and other options to identify GC information  Select GC engine based on need  Size areas of heap based on application profile – e.g., an app that creates many temporary objects may need a large eden area

30. 6/15/201530 Verbose Minor GC Example 62134.872: [GC {Heap before GC invocations=1045: Heap def new generation total 898816K, used 778761K eden space 749056K, 100% used from space 149760K, 19% used to space 149760K, 0% used tenured generation total 1048576K the space 1048576K, 24% used compacting perm gen total 32768K, used 26906K the space 32768K, 82% used 62134.880: [DefNew Desired survivor size 138018816 bytes, new threshold 16 (max 16) - age 1: 3718312 bytes, 3718312 total - age 2: 5935096 bytes, 9653408 total - age 3: 2614616 bytes, 12268024 total - age 4: 1426056 bytes, 13694080 total - age 5: 2095808 bytes, 15789888 total - age 6: 3996288 bytes, 19786176 total - age 7: 550448 bytes, 20336624 total - age 8: 4058384 bytes, 24395008 total : 778761K->23823K(898816K), 0.1397140 secs] 1035112K->280173K(1947392K) Heap after GC invocations=1046: Heap def new generation total 898816K, used 23823K eden space 749056K, 0% used from space 149760K, 15% used to space 149760K, 0% used tenured generation total 1048576K, used 256350K the space 1048576K, 24% used compacting perm gen total 32768K, used 26906K the space 32768K, 82% used }, 0.1471464 secs]

31. 6/15/201531 Verbose Major GC Example 199831.706: [GC {Heap before GC invocations=9786: Heap def new generation total 898816K, used 749040K eden space 749056K, 99% used from space 149760K, 0% used to space 149760K, 0% used tenured generation total 1048576K, used 962550K the space 1048576K, 91% used compacting perm gen total 32768K, used 27040K the space 32768K, 82% used 199831.706: [DefNew: 749040K->749040K(898816K), 0.0000366 secs]199831.707: [Tenured: 962550K->941300K(1048576K), 3.6172827 secs] 1711590K- >1653534K(1947392K) Heap after GC invocations=9787: Heap def new generation total 898816K, used 712233K eden space 749056K, 95% used from space 149760K, 0% used to space 149760K, 0% used tenured generation total 1048576K, used 941300K the space 1048576K, 89% used compacting perm gen total 32768K, used 27007K the space 32768K, 82% used }, 3.6185589 secs]

32. 6/15/201532

33. 6/15/201533 Demonstrate Heap tuning JVM heap tuning was demonstrated for uPortal running on local machine using JDK 1.5.06. (one hour)

34. 6/15/201534 Programming Tips  No need to call System.gc()  Finalizers must be used with care!  The use of Object pools can be debated  Learn to use ThreadLocal for large Objects (but use with care)  Be cognizant of the number/size of Objects being created  Use caches judiciously (e.g., WeakHashMap’s make bad caches under a heavily loaded system)

35. 6/15/201535 Techniques for production env.  lsof command  File descriptors  Verbose gc  Kill -3  Tune gc

36. 6/15/201536 Conclusions  GC can be your friend (or enemy)  Put your app on an Object diet  Should always monitor an app with -verbose:gc -XX:+PrintGCDetails  Tune your heap only if there is a problem  A large heap may slow an app (more memory is not always good)

37. 6/15/201537 ????????????????????

38. 6/15/201538 References  Garbage Collection in Java - http://www.cs.usm.maine.edu/talks/05/printezis.pdf http://www.cs.usm.maine.edu/talks/05/printezis.pdf  A brief history of garbage collection – http://www-128.ibm.com/developerworks/java/library/j-jtp10283/ http://www-128.ibm.com/developerworks/java/library/j-jtp10283/  Garbage collection in the HotSpot JVM - http://www-128.ibm.com/developerworks/java/library/j-jtp11253/http://www-128.ibm.com/developerworks/java/library/j-jtp11253/  Diagnosing a GC problem - http://java.sun.com/docs/hotspot/gc1.4.2/example.html http://java.sun.com/docs/hotspot/gc1.4.2/example.html