Java 9: The Quest for Very Large Heaps Bernard Traversat VP Software Development, Java SE, Oracle bernard.traversat@oracle.com Antoine Chambille Head of Research & Development, Quartet FS antoine.chambille@quartetfs.com

Oracle and ActivePivot Partnership History 2014 2015 2016 2017 partnership to operate a credit risk application on SPARC server with 384 cores and 16TB ActivePivot engineers and JVM engineers work together in Java 9 to improve garbage collection for large heaps release of Java 9 with very large heap support Oracle and Quartet FS meet through a common customer (investment bank), with the challenge of running ActivePivot calculations in a 10TB JVM presentation at JavaOne 2015 presentation of achievements at JavaOne 2016

The Rise of In-Memory Computing $

Use Cases for In-Memory Computing Finance Pricing Supply Chain

Java for In-Memory Computing Performance JIT State of the art parallel computing (java.util.concurrent) Popularity Runs everywhere Versatile, safe, forgiving 20M developers But garbage collection scalability is not limitless

ActivePivot design for large memory calculations Off-Heap Memory Management Parallel Programming Most database structures are Off-Heap Columns of numbers Vectors of simulations Hash tables Indexes Relieves Garbage Collection Implemented with mmap and a custom malloc implementation Fork/Join Pool Reentrant thread pool Work stealing Lock-Free Data Structures Dictionaries Indexes Queues

Unleashing Many-Cores Parallelism Before After DATA PARTITION DATA CPU

NUMA Architecture Allocated memory A Thread A Thread B Allocated memory B

Garbage Collection (GC) Java Garbage Collection behaviors have the most impact on overall application performance and throughput As the JVM heap size grows and object allocation rate increases so does the amount of time that an application must pause to allow the JVM to perform GC Long and unpredictable GC pauses significantly affect applications: Delaying or deadlocking high-frequency transactions Deteriorating application throughput Causing user-session timeouts Requiring costly over-capacity planning GC pauses have traditionally be proportional to the size of the heap. So, a larger heap means a longer pause. Optimized GCs for predictability or low-pause time are been proposed but they tend to have limited applicability (small heap, throughput degradation and full GC cliff)

Current GC strategy are becoming obsolete GC Background Current GC strategy are becoming obsolete Memory and heap size are getting too large for current GC algorithms to scale Core counts are increasing providing more cycle opportunity for concurrent GCs Increase risk of heap fragmentation due to large heaps (100+ GB) The ratio of read vs write is also drastically changing (immutable big data objects) Scalability and predictability matter as much as low-latency Reducing GC storm jitters on large Cloud deployment Avoid unexpected delay in microservices that can create chained delayed reactions in other dependent microservices

G1 (Garbage First) Architecture O O E E O One large contiguous heap space divided into many fixed size regions Size can be 1MB – 32MB Scale to multi-TB heap Each region can be assigned a unique eviction/compaction policy (Eden region, Survivor region, Humongous or Old region) Per region scalable collection process Segment the heap for future capabilities E O O S E O S O O E E O O S S O O O O O H E H H O E Eden regions S Survivor regions O Old generation regions H Humongous regions Available / Unused regions

G1 (Garbage First) Architecture Segmented regions provide more predictable GC pause-times Non-contiguous Eden, Survivor and Old generation spaces Live objects from Eden and Survivor regions are evacuated to a set of unused regions which become the new Survivor regions Number of regions in Eden and Survivor evolve dynamically per GC requirements After concurrent cycle, old regions are collected based on region object liveness Remembered Set tracks the number of object references into a given region Collection Set tracks the set of regions to be collected on a given GC

G1 Garbage Collector – Overview Young collection (n times) Young collection Initial mark Young collection Mixed collection (n times) G1 Remark Clean up STW STW STW STW STW STW STW APP App threads Marking threads CONCURRENT MARKING

G1 Garbage Collector – Running application App threads 0. Initial state 1. Allocate objects App threads STW Young collection Initial mark Young collection (n times) Remark Clean up Mixed collection (n times) G1 APP CONCURRENT MARKING Marking threads App threads STW Initial mark Young collection Young collection (n times) Remark Clean up Mixed collection (n times) G1 APP Marking threads CONCURRENT MARKING

G1 Garbage Collector – Young collection 1. Evacuate live objects 2. Reclaim memory Survivor region STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Young collection …until threshold is reached (IHOP) STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Initial mark Start of the concurrent marking… STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Concurrent marking Keep going with young collections while marking STW Young collection Initial mark Young collection (n times) Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads STW Marking threads STW STW CONCURRENT MARKING APP Marking threads CONCURRENT MARKING

G1 Garbage Collector – Remark Help marking completion STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Remark Collect statistics about old regions… based on % of live objects 75% 30% 15% STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Clean up Clean-up, select regions to collect STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Young Collection 1. 2. STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

G1 Garbage Collector – Mixed collection 1. 2. STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP

Read-Only Benchmark 2015 GC Time Distribution Short Queries GC Time Distribution Calculation Queries GC Time Distribution Full Scan

Improving Young Collections Objects Thread 1 Thread 2 T1 T2 PUBLIC BUFFER PRIVATE BUFFER

Before After Work Stealing Work Stealing Refill Thread 1 Thread 2 PUBLIC BUFFER PUBLIC BUFFER Work Stealing Work Stealing Refill PRIVATE BUFFER PRIVATE BUFFER

Read-Only Benchmark 2015 vs 2016 GC Time Distribution Short Queries GC Time Distribution Calculation Queries GC Time Distribution Full Scan 2015 2016 2016 2015 2016 2015

Java Heap Live Data Bitmap C D E F 512 bits B C D E F

One at a time Before After LOCK FREE Marking Thread 1 Marking Thread 2 MARK STACK One at a time LOCK FREE SHARED STACK

Read-Write Benchmark 2016 GC Time Distribution Short & Full Scan Queries Impact of Updates on Query Time Short & Full Scan Queries Full scan Read & Write Read & Write Short Read only Updates recycle about one terabyte every 10mn Read only

Do you want to be part of it? Please make the connection Antoine Chambille antoine.chambille@quartetfs.com Bernard Traversat bernard.traversat@oracle.com www.quartetfs.com