0. Delivering eBay's CI Solution with Apache Mesos & Docker
Mohit Soni
Ashish Hunnargikar
Software Engineer
Software Engineer

1. Why we did it.... Thousands of VMs dedicated to run Jenkins!
Utilization is less than 5% 
VMs idle most of time (no one runs CI builds frequently)
VM sprawl and poor resource utilization
Polyglot application stacks (Java, C++, Node.js, Python, Scala)
Different OS flavors (Ubuntu & RHEL)
Software version conflicts
Special hardware requirements
Build slave management and maintenance is a nightmare

2. And here’s what we did.... Mesos for cluster management
Marathon to launch Jenkins masters
Jenkins Mesos-plugin to launch build slaves
Build slaves Dockerized as well as regular

5. Why use Docker containers?
Managing varied slave flavors in a large cluster is hard!
Docker effectively isolates dependencies inside the container
Docker allows host OS to remain independent of workload dependencies
Why build Docker-in-Docker?
Run Docker build/push/pull inside a Docker container
Eliminates redundant CI build dependency downloads
Helps to enforce a heterogeneous Mesos slave cluster
Allow different OS flavors
Prevent software version conflicts
Application dependencies
Hardware independence

6. Basic Use Case Build C++ & Java applications on both RHEL & Ubuntu
Mesos slave host server is running Ubuntu
Jenkins job is running builds in a RHEL or Ubuntu container
Application dependencies can be preinstalled in base images OR can be downloaded during build time

7. Advanced Use Case Build RHEL & Ubuntu Docker images
Mesos slave host server is running Ubuntu
Jenkins job is running inside a Docker container
Jenkins job is building a RHEL/Ubuntu Docker image from an application Dockerfile

8. Enabling the Docker CI workflow using Jenkins & Mesos

9. Initial Build Strategy Individual Docker-in-Docker Setup
Complete isolation
Docker daemon running inside the Docker container requires privileged mode
I/O overhead of downloading and duplicating all the AUFS layers

10. Optimized Build Strategy Shared Docker-in-Docker Setup
Docker daemon not required inside the container so no privileged mode
Single reusable AUFS image layer cache
Redundant network and disk I/O eliminated for layer downloads
Lack of isolation for layers

11. For more information visit:
Q&A
For more information visit:
Apache Mesos:
Jenkins Mesos Plugin:
Docker:
Marathon Framework: