Download presentation
Presentation is loading. Please wait.
1
Java 9: The Quest for Very Large Heaps
Bernard Traversat VP Software Development, Java SE, Oracle Antoine Chambille Head of Research & Development, Quartet FS
2
Oracle and ActivePivot Partnership History
2014 2015 2016 2017 partnership to operate a credit risk application on SPARC server with 384 cores and 16TB ActivePivot engineers and JVM engineers work together in Java 9 to improve garbage collection for large heaps release of Java 9 with very large heap support Oracle and Quartet FS meet through a common customer (investment bank), with the challenge of running ActivePivot calculations in a 10TB JVM presentation at JavaOne 2015 presentation of achievements at JavaOne 2016
3
The Rise of In-Memory Computing
$
4
Use Cases for In-Memory Computing
Finance Pricing Supply Chain
5
Java for In-Memory Computing
Performance JIT State of the art parallel computing (java.util.concurrent) Popularity Runs everywhere Versatile, safe, forgiving 20M developers But garbage collection scalability is not limitless
6
ActivePivot design for large memory calculations
Off-Heap Memory Management Parallel Programming Most database structures are Off-Heap Columns of numbers Vectors of simulations Hash tables Indexes Relieves Garbage Collection Implemented with mmap and a custom malloc implementation Fork/Join Pool Reentrant thread pool Work stealing Lock-Free Data Structures Dictionaries Indexes Queues
7
Unleashing Many-Cores Parallelism
Before After DATA PARTITION DATA CPU
8
NUMA Architecture Allocated memory A Thread A Thread B
Allocated memory B
9
Garbage Collection (GC)
Java Garbage Collection behaviors have the most impact on overall application performance and throughput As the JVM heap size grows and object allocation rate increases so does the amount of time that an application must pause to allow the JVM to perform GC Long and unpredictable GC pauses significantly affect applications: Delaying or deadlocking high-frequency transactions Deteriorating application throughput Causing user-session timeouts Requiring costly over-capacity planning GC pauses have traditionally be proportional to the size of the heap. So, a larger heap means a longer pause. Optimized GCs for predictability or low-pause time are been proposed but they tend to have limited applicability (small heap, throughput degradation and full GC cliff)
10
Current GC strategy are becoming obsolete
GC Background Current GC strategy are becoming obsolete Memory and heap size are getting too large for current GC algorithms to scale Core counts are increasing providing more cycle opportunity for concurrent GCs Increase risk of heap fragmentation due to large heaps (100+ GB) The ratio of read vs write is also drastically changing (immutable big data objects) Scalability and predictability matter as much as low-latency Reducing GC storm jitters on large Cloud deployment Avoid unexpected delay in microservices that can create chained delayed reactions in other dependent microservices
11
G1 (Garbage First) Architecture
O O E E O One large contiguous heap space divided into many fixed size regions Size can be 1MB – 32MB Scale to multi-TB heap Each region can be assigned a unique eviction/compaction policy (Eden region, Survivor region, Humongous or Old region) Per region scalable collection process Segment the heap for future capabilities E O O S E O S O O E E O O S S O O O O O H E H H O E Eden regions S Survivor regions O Old generation regions H Humongous regions Available / Unused regions
12
G1 (Garbage First) Architecture
Segmented regions provide more predictable GC pause-times Non-contiguous Eden, Survivor and Old generation spaces Live objects from Eden and Survivor regions are evacuated to a set of unused regions which become the new Survivor regions Number of regions in Eden and Survivor evolve dynamically per GC requirements After concurrent cycle, old regions are collected based on region object liveness Remembered Set tracks the number of object references into a given region Collection Set tracks the set of regions to be collected on a given GC
13
G1 Garbage Collector – Overview
Young collection (n times) Young collection Initial mark Young collection Mixed collection (n times) G1 Remark Clean up STW STW STW STW STW STW STW APP App threads Marking threads CONCURRENT MARKING
14
G1 Garbage Collector – Running application
App threads 0. Initial state 1. Allocate objects App threads STW Young collection Initial mark Young collection (n times) Remark Clean up Mixed collection (n times) G1 APP CONCURRENT MARKING Marking threads App threads STW Initial mark Young collection Young collection (n times) Remark Clean up Mixed collection (n times) G1 APP Marking threads CONCURRENT MARKING
15
G1 Garbage Collector – Young collection
1. Evacuate live objects 2. Reclaim memory Survivor region STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
16
G1 Garbage Collector – Young collection
…until threshold is reached (IHOP) STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
17
G1 Garbage Collector – Initial mark
Start of the concurrent marking… STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
18
G1 Garbage Collector – Concurrent marking
Keep going with young collections while marking STW Young collection Initial mark Young collection (n times) Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads STW Marking threads STW STW CONCURRENT MARKING APP Marking threads CONCURRENT MARKING
19
G1 Garbage Collector – Remark
Help marking completion STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
20
G1 Garbage Collector – Remark
Collect statistics about old regions… based on % of live objects 75% 30% 15% STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
21
G1 Garbage Collector – Clean up
Clean-up, select regions to collect STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
22
G1 Garbage Collector – Young Collection
1. 2. STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
23
G1 Garbage Collector – Mixed collection
1. 2. STW Young collection STW Initial mark STW Young collection (n times) STW Remark STW Clean up STW Young collection STW Mixed collection (n times) G1 App threads Marking threads CONCURRENT MARKING APP
24
Read-Only Benchmark 2015 GC Time Distribution Short Queries
GC Time Distribution Calculation Queries GC Time Distribution Full Scan
25
Improving Young Collections
Objects Thread 1 Thread 2 T1 T2 PUBLIC BUFFER PRIVATE BUFFER
26
Before After Work Stealing Work Stealing Refill Thread 1 Thread 2
PUBLIC BUFFER PUBLIC BUFFER Work Stealing Work Stealing Refill PRIVATE BUFFER PRIVATE BUFFER
27
Read-Only Benchmark 2015 vs 2016
GC Time Distribution Short Queries GC Time Distribution Calculation Queries GC Time Distribution Full Scan 2015 2016 2016 2015 2016 2015
28
Java Heap Live Data Bitmap
C D E F 512 bits B C D E F
29
One at a time Before After LOCK FREE Marking Thread 1 Marking Thread 2
MARK STACK One at a time LOCK FREE SHARED STACK
30
Read-Write Benchmark 2016 GC Time Distribution Short & Full Scan Queries Impact of Updates on Query Time Short & Full Scan Queries Full scan Read & Write Read & Write Short Read only Updates recycle about one terabyte every 10mn Read only
31
Do you want to be part of it? Please make the connection
Antoine Chambille Bernard Traversat
Similar presentations
