1. CloudOpting - Hackathon
Technical Speech

2. Introduction to technologies used in CloudOpting

3. Docker

4. Introduction Queue User DB QA server VMs Laptop Data Center
Various services and applications:
Static website
Queue
User DB
API endpoint
Analytics DB
Web frontend
Background workers
Various hardware environment
Can we migrate applications easily and quickly?
QA server
VMs
Production Server
Laptop
Data Center
Production Cluster
Cloud público

5. Introduction API endpoint Analytics DB Queue Web frontend User DB
Static website
Background workers
QA server
Laptop
Docker is an engine that let us to encapsulate any information and make it portable, self sufficient and isolated
So it can be manipulated through estándar operations and can be executed consistently in any hardware
Once it has been containarised, you deal with Docker, every application will be the same. Standard administration.
Dev: Create it once and run wherever you want
Sys: Configure it once and run wherever you want
VMs
Cloud público
Data Center
Production Server
Production Cluster

6. Introduction What is Docker? What are the benefits?
Currently it is the most widespread container technology.
Take advantage of characteristic of Linux Kernel like cgroups, libcontainer (LXC modification), Namespaces…. Therefore containers share part of this Kernel.
Can work in every system where Linux runs (laptops, VMs, Cloud…). Recently also for Windows.
Formed by a set of tools oriented to make easier the way to deal with these containers.
Provide an isolated environment which contains everything needed to run an application.
Provide images which can be easily generated, modified from others, shared and versioned
Lighter than a virtual machine

7. Introduction Difference with a Virtual Machine Hypervisor
Infrastructure
Host OS
Hypervisor
Guest OS
Bin/ Libs
App B
App A’
App A
Host OS
Docker Engine
Infrastructure
Bin/Libs
Binaries/Libraries
App A
App A’
App B
App B’
App B’’
App B’’’
Virtual Machines
Docker containers

8. Introduction Some benefit using Docker
Reduction of number of VMs needed
Better use of resources available in the VM (containers use and share resources dynamically Automatic vertical scaling.
Less time needed to deploy and enable us using horizontal scaling.
Quick and automatic reboot of container after falling down.
Independency of the Cloud.
Lots of images available, therefore developers need less time to prepare and develop applications.
Easy to move, deploy , etc. Once you have the container you deal with Docker, not the application.
Easy deployment even with complex systems with docker-compose. This tool let us to describe how we want to deploy everything at containers level

9. Introduction Components Incremental revolution of the platform:
If we need a runtime (Docker container): Images, container, volumes.
If we need a way to distribute it: Dockerfile, DockerHub, Docker Registry.
If we need to execute it in different machines: Docker machine.
If we need to build complex solutions: Docker compose.
If we need scale application or create cluster of applications: Docker Swarm.

10. Docker
Block of Docker

11. Docker images Storage Dockerfiles: Recipes for construction
Docker Hub: Storage repository (property of Docker).
Save: docker push ImageName:tag
Search: docker search ImageName o acceso web
Download: docker pull ImageName:tag
Docker Registry: Private storage.
Save: docker push RegistryHost:port/ImageName:tag
Download: docker pull RegistryHost:port/ImageName:tag
Docker save/load: Store in .tar.gz (for image persistence with all its layers)
Save: docker save ImageName > ImageName.tar.gz
Load: docker load < ImageName.tar.gz
Docker export/import: Store in .tar.gz (for image container persistence and only one layer)
Save: docker export container > container.tar.gz
Load: cat container.tar.gz | docker import - image:tag

12. Docker images DockerFile Main Instructions: FROM VOLUME MAINTAINER
EXPOSE
ENV
ADD
COPY
VOLUME
USER
WORKDIR
RUN
ENTRYPOINT
CMD …

13. Docker images Main commands Docker build Docker images Docker rmi
Docker tag
Docker push
Docker pull
Docker import
Docker export
Docker commit
Docker load
Docker save …
Tip: docker rmi $(docker images –aq)

14. Docker images How to build images:
Utilizamos el commando docker build [OPTIONS] PATH (path is the address where the docker file is. This is called CONTEXT), Usual options:
-t, --tag valor: usually the nomenclature used is: “organization/ImageName:tag”. By default tag=latest
-f, --file cadena: Name of the docker file which will be used to create the image.

15. Docker Images
Example

16. Docker containers Difference between image and container:
A dockerfile ia a recipe which contains the instructions to create an image. Usually this will be done modifying a master image (parent image).
An image is like a template, a snapshot of the status of a container.
Based on UFS can be built stacking layers. All of them are only reading layers.
A container is an executing image. These are which our application will use.
The container adds a last layer of reading/writing above. The user only works with this upper layer.

17. Docker containers Main commands Docker start Docker run Docker stop
Docker restart
Docker pause
Docker unpause
Docker kill
Docker attach
Docker create
Docker run
Docker rm
Docker ps
Docker logs
Docker inspect
Docker stats
Docker exec …
Tip: docker rm -v $(docker ps -aq)

18. Docker Containers Main options in docker run: --name --env
--publish, -p
--publish-all, -P
--link
--log drive
--log opt
--volume
--restart
--env
--env-file
--entrypoint
--expose
--workdir

19. Docker containers - Networking
Port publishing
The container are connected by default to an intern virtual network. Executing ipconfig we can see the network interface in the host. The name is docker0.
Docker allows the accessing to containers from outside only if it is indicated explicitly. For that purpose the ports will be mapped through NAT
To indicate that we wish to publish a port the following options in the docker run can be used:
-p “port_host:port_container”
-P : this choice will publish the ports which were indicated by EXPOSE in the dockerfile and random ports will be mapped.
A range of ports can also be indicated [96-175]:[ ]
Even any port can be indicated. In this case Docker will choose a random port.

20. Docker containers - Networking
Links among containers
Links can be done among containers through option --link in the docker run command. --link=“container name or id:Alias”
When this command is used, the container (we want to connect to) should be previously created.
Instruction link adds to /etc/hosts (of the container is being created) the IP of the container we want to connect to.
Instruction link also adds some state variables to the created container :
name_PORT
name_PORT_num_protocol
name_PORT_num_protocol_ADDR
name_PORT_num_protocol_PORT
name_PORT_num_protocol_PROTO
name_NAME

21. Docker containers - Networking
Network types
Bridge mode: This mode indicates that after container creation this will connect to the virtual network (host interface docker0). This is the option by default.
Host mode: this mode indicates that the container will share the connection stack with the host.
Container mode: This mode indicates that the container will share the connection stack of other container previously created
None mode: no network interface is indicated to the container, therefore the container does not have any network interface

22. Docker containers - Volumes
Data persistence
Problem: the data generated in a container will be enclosed in this container.
Solution: let share files or data with the outside
To add a volume we will use option -v “dir_host:dir_container” or --volume=“dir_host:dir_container”

23. Docker container
Example

24. Docker Compose Main instructions Build Image Links Ports Net Command
Entrypoint
Environment
Env_file
Expose
Links
Net
Volumes
container_name
log_driver
log_opt
fluentd-address
Tag …

25. Docker Compose Main commands Docker-compose build Docker-compose up
Docker-compose down
Docker-compose start
Docker-compose restart
Docker-compose stop
Docker-compose pause
Docker-compose unpause
Docker-compose scale …

26. Docker compose
Example

27. Logs collecting
Docker considers log all that uses the standard output of the container. Therefore the service must be set up to show this info on the screen
Docker provides several ways to extract logs.
Among these ways, Fluentd (tool in the Docker suite) offers native compatibility with Docker.
Fluentd is a software for logs collection and treatment. This tool allows us to collect, filter and store logs from different services running inside the containers
In CloudOpting platform other complementary technologies are used: ElasticSearch for logs storage and Kibana for logs visualization.

28. Logs collecting
Time for Demo

29. Docker - Hands on code