1. Container technology, Microservices, and DevOps
Digital Innovation Workshop
April 2019
Henrik Bærbak Christensen

2. The DevOps Problem Crossing boundaries, that is, moving executing code
Any code we move, has a lot of dependencies on the env it moves to: OS, java run time, env variables, property files, credentials, network, connections, firewall rules, exsistence of external services, etc.
Source: Torben Haagh, StiboSystems
Henrik Bærbak Christensen

3. Henrik Bærbak Christensen
Was Solved in 1960’ies
… for physical goods…
Henrik Bærbak Christensen

4. Henrik Bærbak Christensen
Docker = Container
Henrik Bærbak Christensen

5. Definition
(Docker web site, 2019)
That is, not just program/code, but the required execution environment
Henrik Bærbak Christensen

6. Containers are Virtual Machines
Henrik Bærbak Christensen

7. Physical vrs. Virtual Machine
Henrik B Christensen

8. Process / System VM Example: Java VM Example: M51
Pro: Small memory footprint
Con: Requires executing environment
Example: M51
Pro: Requires only ‘hardware’
Con: Large memory footprint
(Smith & Nair, 2005)
Henrik B Christensen

9. Henrik Bærbak Christensen
Containers
Mission: To make virtualization lightweight
Traditional VMs
Docker
Henrik Bærbak Christensen

10. Henrik Bærbak Christensen
Classifying Docker
A Same-ISA Process VM
Docker containers provide a virtual Linux OS
Docker containers typically execute a single process
Ex: Run a MySQL database, a TomCat web server, …
Henrik Bærbak Christensen

11. Docker Engine

12. Henrik Bærbak Christensen
Images and Containers
Core concepts in Docker
Image The encapsulation of a VM
I.e. the physical file that contains the VM
Similar to a Java Jar file, a DLL/assembly, .exe
Container The executing instance of an image
Similar to an executing Java system, running the main() from the Jar file, a
Docker Engine The Virtual Machine Monitor program
That handles images and executes containers
Henrik Bærbak Christensen

13. Henrik Bærbak Christensen
Example
Henrik Bærbak Christensen

14. Henrik Bærbak Christensen
Images
Onion file system: Copy-on-Write
Every operation basically creates a new file layer
Changing ‘hans.txt’ in layer N creates a (modified) copy of ‘hans.txt’ in layer N+1
Base images = ‘prebaked file system’
All layers up-till N forms an Image
I.e. henrikbaerbak/cloudarch:e16.1
Ubuntu LTS server base image
Java, Ant, Ivy, Git, … are all layered on top
Layer N+1
Layer N …
Base Layer
Henrik Bærbak Christensen

15. Henrik Bærbak Christensen
Building Images
How do you build a traditional server?
Unbox the machine, power up, install Linux, install application suite and libraries, configure network and dependencies, execute server software – Phew…
Lifecycle - classic
container = instantiate(image1)
Docker run –ti ubuntu:18.04 bash
modify container
Install software, change files, add stuff, …
commit container → image2
Docker commit
Power up
Install your app
‘Freeze’ the machine
Henrik Bærbak Christensen

16. Henrik Bærbak Christensen
Docker Hub
‘Bootstrapping’ – where do I get the base layer from?
Docker Hub is a public repository of a lot of base images
(and a lot of open source images as well)
Henrik Bærbak Christensen

17. Container Lifecycle Management
Docker Engine

18. Henrik Bærbak Christensen
We need to…
Build images
Execute containers
Monitor containers
Kill containers
Henrik Bærbak Christensen

19. Henrik Bærbak Christensen
Images
‘pull’ = pull a named image from Hub
‘push’ = push a named image to Hub
Henrik Bærbak Christensen

20. Henrik Bærbak Christensen
Containers
run = (pull image), and start container (-d = in background)
ps = see all running containers (-a = all, also the dead ones)
images = see all images
rm = remove container (-f =force, even if currently running)
Henrik Bærbak Christensen

21. Henrik Bærbak Christensen
Docker run
Docker run (image) (command parameters)
Acceptable parameters defined by the image!
Zillion parameters, the most important are
docker run –d : -d = daemon mode / server
-ti: terminal interactive, so you can interact!
--name (myname): give it a name
-p (host):(container): port-mapping
--network=(network): define network to use
”host” The host’s network
”container:(containername)” That of a named container
”networkname” That of a named network
Henrik Bærbak Christensen

22. Henrik Bærbak Christensen
Docker Network
Port-mapping
Expose given port on the host’s network
-p 22333:37123
The container’s port is now visible at port on the host network
Exercise:
Why a need for different ports on container and host?
Henrik Bærbak Christensen

23. Henrik Bærbak Christensen
Containers
‘logs’ = See the log (shell output) of running container
‘logs –f’ = Tails the log (keeps running)
Henrik Bærbak Christensen

24. Henrik Bærbak Christensen
Containers
A container is very opaque 
What the heck is going on inside it ???
Where are the files located???
‘exec -ti (container) bash’
Is: execute ‘bash’ interactive TTY in the running container
Henrik Bærbak Christensen

25. Henrik Bærbak Christensen
Find it on the Web 
Henrik Bærbak Christensen

26. Infrastructure-as-Code
Dockerfile
Infrastructure-as-Code

27. Henrik Bærbak Christensen
Building Images
Lifecycle – infrastructure-as-code
You automate the install script: Dockerfile
Example: henrikbaerbak/jdk8-gradle
Henrik Bærbak Christensen

28. Henrik Bærbak Christensen
Anatomy of an Image
State your base image
Identify yourself
Copy your software
WORKDIR set working directory in container
COPY (src) (dest)
copy from src to dest (src: relative to current dir, dest: rel. to workdir)
Configure
EXPOSE expose port to the outside
Execute
CMD state defaults for process to be run
Imhotep
Henrik Bærbak Christensen

29. Henrik Bærbak Christensen
Tech note on RUN / CMD
RUN is executed while the image is created
‘build time’
CMD is executed when the container is started
‘run time’
Preferred format is an array!
CMD [”python3”, ”server.py”, ”argument1”]
Henrik Bærbak Christensen

30. Henrik Bærbak Christensen
Building Image
‘docker build .’
= Build image from Dockerfile in current dir
You will want to name the image
-t (imagename) ‘tag’ the image with a name
Example:
‘docker build –t henrikbaerbak/mcd:version1 .’
Build image using local Dockerfile in this directory, resulting image is called ‘henrikbaerbak/mcd’ and tagged with label ‘version1’
Imhotep
Henrik Bærbak Christensen

31. Multistage Dockerfile
Issue: Avoid image having intermediate steps
Example:
I want to distribute SkyCave daemon without the source code
Step 1: Compile source code and produce a ‘JAR’ file
Step 2: Have an image with ONLY the jar file
Require that I copy the jar file from step 1 to step 2 but nothing else…
Henrik Bærbak Christensen

32. Debugging Dockerfile

33. Henrik Bærbak Christensen
Coding Means Bugs...
Docker files are code that builds infrastructure
And coding means making mistakes that must be found and corrected – aka debugging...
So - How to debug Dockerfiles?
Henrik Bærbak Christensen

34. Henrik Bærbak Christensen
Take Small Steps
Why not
Step 1: Let the Dockerfile only copy the rigth files into the right places – and then no more
Step 2:
‘docker build –t myimage .’
‘docker run --ti myimage bash’
And then test that everything are in the right places?
Step 3: Update the Dockerfile with the EXPOSE and CMD
Step 4: and then run/debug it...
Henrik Bærbak Christensen

35. Henrik Bærbak Christensen
Use CMD
The CMD command defines default command to start the container process
But you can overrule it by issuing another one!
Example:
docker run –d –p 4567:4567 skycave
Start the skycave container, and execute the CMD specified command
docker run –d –p 4567:4567 skycave bash
Start the skycave container, and execute ‘bash’ only!
Now, you can use standard linux bash commands to review the file system and inspect the container...
Henrik Bærbak Christensen

36. Henrik Bærbak Christensen
Use the ‘logs’ feature
If a container is running, you may inspect its ”screen output” using the logs command
docker logs (containername)
Or continously monitor (tail)
Docker logs –f (containername)
Hit ctrl-c to quit...
It is Docker practice to output log messages on the console, instead in log files!
Henrik Bærbak Christensen

37. Inspecting Running Containers
If the container is running, you can also simply ‘log in’ and look what is going on...
docker exec –ti (containername) bash
Start ‘bash’ as a terminal interactively (-ti) in the container
Henrik Bærbak Christensen

38. Inspecting Dead Containers
If a container starts and then dies due to a bug, and you have no idea from the ‘logs’ output, you cannot ‘exec’ into it, as it is not running
Use this trick instead:
docker ps –a
note down the container ID
docker commit (ID) mysnapshot
Create image ‘mysnapshot’ from the container
docker run –ti mysnapshot bash
And start a new container from that image, and run ‘bash’ in it...
May look into local log files, directory structure, etc.
Henrik Bærbak Christensen

39. Nifty Uses of Docker Images

40. Henrik Bærbak Christensen
Besides the Obvious
Running a service in a container is obvious
But you can do a lot of fun things
Example:
My students have to import a large dataset into their own MongoDB database to solve exercises using the (map,reduce) paradigm.
I made ‘ml-load’ which contains the data and the import script to a running mongo db named ‘mldb’ on the network.
Henrik Bærbak Christensen

41. Henrik Bærbak Christensen
Besides the Obvious
I need to compile stuff on a machine without the compiler, I only have the source code
Make a dockerfile which is based upon a base image with compiler and build tool; copies the code into the image and then compiles the compiled stuff to a specific folder that is ‘mounted’ on the host machine.
Here I copy to the host machines ‘ivy’ cache which means stuff on the host can now access the compiled module…
Henrik Bærbak Christensen