„Oracle Code ONE 2018“ JVM CPU & Memory Settings for docker containers in production DEV6060 Andreas Hochleitner 22.10.2018

Starting questions Orientation and/or background Technical, dev, ops, business, sales, management Which JDK are you using in PRODUCTION 11 10 9 8 (<=u121, <=u131) 7 <7 Who is using … in Production Containers: Docker, LXC, .. Orchestration: Kubernetes, Docker Swarm, Rancher, ..

Java Dev Perspective: impact of production platform to the Java Development No impact Slight impact Huge impact

My professional background Studies Informatics at Technical University Vienna (TU Wien) / Austria 30 years in the IT business Development centric > 20 years Java / JavaEE (J2EE) > 10 years independant consultant as Architect

Current projects Consulting JavaEE-Product Development for Startup (development, process, state-of-the-art) Solution Architect in a large component-oriented Landscape (JavaEE, CI/CD, strong integration aspects) Wildfly / Jenkins / Groovy / Docker / Kubernetes

Prerequisites for this talk Basic Docker Knowhow Basic Kubernetes Knowhow Basis Linux Knowhow

Structure 1 Background – Motivation 2 Overview 3 JDK – 7 8 9 10 11 4 Container-Basis: Docker - K8S 5 Different Aspects with examples Memory CPU Threads 6 Summary

1 Background – Motivation Last JavaOne Session „Java inside docker: What you must know to not FAIL“ Rafael Benevides, Burr Sutter Different articles/blogs about „Don‘t use Java in Containers“ or „How to not fail using Java in Containers“ JDK 9 – special parameters -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap Backported to JDK 8 u131 Master-Agent-Jenkins in OnPrem-Kubernetes ==> OOM Killer Problem

2 Overview Production oriented JDK versions are not arbitrarily choosable Moving existing applications into container directions What you must know? Which problems can occur? How to deal with or fix the problems?

3 JDK – 7 8 9 10 11 Examples are based on OpenJDK from DockerHub OpenJDK 8u121 OpenJDK 8 (8u181) OpenJDK 9 OpenJDK 10 (OpenJDK 11)

4 Container-Basis: Docker - K8S Docker 18.03.1-ce on Linux (Ubuntu 16.04) Kubernetes OnPrem (large K8S-Cluster, Minikube)

Docker – Mini Basics What are Docker Containers Linux cgroups & namespaces Dockerfile „docker build“ „docker run“

Kubernetes – Mini Basics Kubernetes Cluster What is a Pod? Kubectl Manifest-File (yaml-file)

Basic Problem JVM in a container sees Host Settings and not Cgroup-Settings Depending on JDK version OOM Killer (out-of-memory killer)

Different Aspects with examples Memory CPU Threads

JDK 10 Ergonomics Process to improve application performance by JVM and Garbage collection heuristics (behavior-based heuristics) Server class 2 or more physical processors 2 or more GB of physical memory

JDK 10 Ergonomics Server class defaults Similar to other JDKs Garbage-first (G1) collector Inital heap size of 1/64 of physical memory Maximum heap size of ¼ of physical memory Tiered compiler, using both C1 andC2 Similar to other JDKs

JDK 9 Improvements (8 backport) -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap (-XX:MaxRAMFraction=1) With these Settings JVM will use Cgroup-infos, and not Host-infos

JDK 10 Container Support (improvements) Improve docker container detection and resource configuration usage JDK-8146115) -XX:-UseContainerSupport -XX:ActiveProcessorCount=count Allow more flexibility in selecting Heap % of available RAM (JDK-8186248) -XX:InitialRAMPercentage -XX:MaxRAMPercentage -XX:MinRAMPercentage attach in Linux should be relative to /proc/pid/root and namespace aware (JDK-8179498) -XX:+UseContainerSupport is default

JVM Settings for the worst case -XX:ParallelGCThreads -XX:CICompilerCount -Xmx -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap

6 Summary Understand your JVM application Memory Cpu usage Thread counts Analyse & test your concrete JVM in realistic production environment Use up-do-date JDK version (11 → 10 → 9 → 8 → 7) as far it is possible Set JDK/Container/Pod-Settings according to availablilty

Q & A

Thank You! Mail: andreas.hochleitner@hochis.at Twitter: @ahochleitner