1. Scalable containers with Apache Mesos and DC/OS
by Nataliia Tkachenko
Rostyslav Fridman
June, 2016

2. Agenda

3. Apache Mesos introduction
Microservices. Why do we need them?
What is Apache Mesos?
Apache Mesos architecture
Apache Mesos installation
What is Mesosphere DC/OS?
Demo

4. Microservices
And
Why do we need them?

5. Conway's law
“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.”
- Melvin Conway, 1967

6. Siloed functional teams lead to slioed application architectures.
Monolithic application
Siloed functional teams lead to slioed application architectures.
Because Conway’s Law.
UI Specialists
Middleware
Specialists
DBA
Specialists

7. Cross-functional teams. Organized around capabilities.
Microservices
Cross-functional teams. Organized around capabilities.
Because Conway’s Law.

8. Different designs
Monolithic application
Microservices
A monolithic application puts all its functionality into a single process…
A microservices architecture puts each elemnt of functionality into a separate service…
… and scales by replicating the monolith on multiple servers.
… and scales by distributing these services across servers, replicating as needed.

9. Increasing frustration 1
Monolithic applications
Increasing frustration 1
Whole rebuild is necessary every time 2
Hard to isolate code 3
Scaling requires more resources 4

10. Independent deploy and scaling 1
Microservices
Independent deploy and scaling 1
Strict module boundary 2
Modules developed by different teams 3
Services scaled separately 4

11. Life without Apache Mesos
Origins
Life without Apache Mesos

12. Prior practices: Dedicated servers
Low utilization rates
Longer time to bootstrap new servers

13. Prior practices: Virtualization
Even more machines to manage
Substantial performance decrease due to virtualization
VM licensing costs

14. Hard to utilize resources
Static partitioning is BAD
Hard to utilize resources
Hard to scale elastically
Hard to deal with failures

15. What is Mesos?
Mesos
Overview

16. Mesos function
Mesos abstracts the entire datacenter into a single pool of computing resources, simplifying running distributed systems at scale

17. Mesos does not work with machines
Mesos is a level of indirection
Mesos does not work with machines
Mesos works with resources (CPU, memory, disk space)
Mesos is like a “kernel” of datacenter

18. Mesos Frameworks
Mesos is the foundation for a number of distributed systems. These distributed systems are called frameworks.

19. Inner workings
Mesos
architecture
Overview

20. Coordinator Frameworks Allocator Resources Mesos architecture
Jenkins Scheduler
Chronos Scheduler
Frameworks
ZooKeeper Quorum
Mesos
Master
Standby
Master
Standby
Master
Allocator
Mesos Slave
Mesos Slave
Mesos Slave
task
Jenkins executor
task
Chronos executor
task
Jenkins executor
Chronos executor
Resources

21. Example of resource offer
Slave 1 reports to the master that it has 4 CPUs and 4 GB of memory free. The master then invokes the allocation policy module, which tells it that framework 1 should be offered all available resources.
The master sends a resource offer describing what is available on slave 1 to framework 1.
The framework’s scheduler replies to the master with information about two tasks to run on the slave, using <2 CPUs, 1 GB RAM> for the first task, and <1 CPUs, 2 GB RAM> for the second task.
Finally, the master sends the tasks to the slave, which allocates appropriate resources to the framework’s executor, which in turn launches the two tasks (depicted with dotted-line borders in the figure). Because 1 CPU and 1 GB of RAM are still unallocated, the allocation module may now offer them to framework 2.
Job 1
Job 2
FW Scheduler
Framework 1
Job 1
Job 2
FW Scheduler
Framework 2
Slave has 4 cpu, 4 GB RAM 2
Start task1 with 2 cpu and 1 gb
Start task2 with 1 cpu and 2 gb 3
Allocation module
Mesos master
I have 4 cpu, 4 GB RAM 1
Allocate resources for task1
Allocate resources for task2 4
Executor
Task
Slave 1
Executor
Task
Slave 2

22. Installation
Mesos
Installation
Overview

23. $ sudo yum install mesos marathon
Installing Apache Mesos manually
Java
Zookeeper
Mesos Slave
Mesos Master
Marathon
(Optional) Mesos-DNS
(Optional) Docker
Please use the steps described on this link:
$ sudo yum install mesos marathon

24. … or using DC/OS Install on on any of the following platforms:
Local (Vagrant)
Cloud (AWS, Azure, Packet)
Custom Cloud
On-Premises
Please use the steps described on this link:

25. Demo

26. Create dockerized Jenkins master using Marathon
Demo agenda
Set up Mesos with DC/OS
Create dockerized Jenkins master using Marathon
Connect Jenkins master to Mesos as a Framework
Create job to build sample application
Run job
Jenkins framework should automatically create slave on mesos for running this task

27. CONTACTS Rostyslav Fridman Rostyslav_Fridman@epam.com Skype: altor-kun
GitHub: