Use Docker to Enhance Your Testing Glenn Buckholz E-mail: glenn.buckholz@coveros.com Coveros, Inc.

Agenda Intro Docker Overview How Docker Changes Testing DevOps Approach Demonstration Using Docker to Talk to Developers Speeding Up The Feedback Cycle Questions

Introduction Docker is a popular form of containerization Containerization is a server virtualization method in which the kernel of an operating system allows the existence of multiple isolated user-space instances, instead of just one. Other Container Technologies Jails - BSD Chroot - Filesystem Level Xen paravirtualization - Processes Hypervisor LXC or LXD - Linux Containerization Virtualization vs Containerization Virtualization allows many OS images to use the same hardware. Containerization allows many pieces of software to use the same OS in an isolated fashion.

Docker Overview Docker has 3 major components Images Filesystem of the container. Containers Running instance of an image. Registries Repository of images. Docker Hub is a global public repository. Local Registry on the docker host itself Private Registry Private Registry Image Container

Docker Overview Docker Host Hardware and OS that is shared among containers. Docker Service multiplexes the OS NIC’s Devices Storage CPU limits IO limits Intra Host Communication HA Scaling Load Balancing

Docker Overview Git version control docker commit and docker push Promoting containers When the container is where/what you want docker tag imageid registryhost:port/tagname docker push registryhost:port/tagname Make the environment change around the container. Keep the container constant. Use mount points for configs Use mount points for logs Make the container network agnostic Make the thing that you put in production the thing you had in dev and test.

DevOps Approach Dockerfile creation Automate from known stock images Quality Gates Container “Unit Tests” Not strictly unit tests but you are testing one high level piece of functionality. Container “Integration Tests” Use docker swarm or some other orchestration tool to put together a temporary environment, run minimal testing. Container “Nightly Testing” Run a full set of automated tests overnight. Container “Manual Testing” After nightly testing release the containers to manual testers.

DevOps Approach Hourly (Automatic) Quality Gate Unit, Integration Quality Gate UI testing Nightly Release Candidate Release

DevOps Approach Promote the container through some or all the different stages automatically. Dev and test should have high levels of automation. Each quality gate is automatic and triggered by changes to the Dockerfile OR the underlying application. Failures are preserved for some time (24-72 hours) Dev Test Stage Prod Failure

Demo Tester Developer Interaction Demo “I need a blue seal balancing on its flippers with an eye.” 5 Min max Have test prop with a picture of the ideal behind the projector screen Have an index card with obvious answers under it. Give the volunteer 15 seconds to look at the picture and the answers. Ask him to recite the defects verbally. Argue about the colors. Pause and count the number of agreed defects and disagreed defects. Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item. Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

Demo - What the Developer Made 5 Min max Have test prop with a picture of the ideal behind the projector screen Have an index card with obvious answers under it. Give the volunteer 15 seconds to look at the picture and the answers. Ask him to recite the defects verbally. Argue about the colors. Pause and count the number of agreed defects and disagreed defects. Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item. Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

Demo - What the PO Wanted 5 Min max Have test prop with a picture of the ideal behind the projector screen Have an index card with obvious answers under it. Give the volunteer 15 seconds to look at the picture and the answers. Ask him to recite the defects verbally. Argue about the colors. Pause and count the number of agreed defects and disagreed defects. Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item. Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

Demo - The difference Issues The shade of blue is wrong The eye is on the wrong side They are mirror images of each other The Seal is balanced wrong Problems I had to specifically articulate each issue. Some of these issues don’t translate well in words. 5 Min max Have test prop with a picture of the ideal behind the projector screen Have an index card with obvious answers under it. Give the volunteer 15 seconds to look at the picture and the answers. Ask him to recite the defects verbally. Argue about the colors. Pause and count the number of agreed defects and disagreed defects. Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item. Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

Demo - Or I Can Just Compare the Two 5 Min max Have test prop with a picture of the ideal behind the projector screen Have an index card with obvious answers under it. Give the volunteer 15 seconds to look at the picture and the answers. Ask him to recite the defects verbally. Argue about the colors. Pause and count the number of agreed defects and disagreed defects. Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item. Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

Speeding Up Feedback Cycles Developers can have complete access to containers to look at logs Failed containers can be saved to a private registry Testers can demo the defect on demand. Old Way: New Way: Fight About it Compromise Find Bug Fix it Find Bug Demonstrate Fix it

Demo Technical Demonstration

Demo Sample Application Architecture Tomcat 9 MariaDB

How Docker Changes Testing The operating system is domain of specialists due to the impact of one wrong change. The host should still be maintained by specialists. Containers put barriers between the software stack and the OS. Allow the testers direct access to the containers. Containers can now be tagged with test artifacts. Test case failed. Code branch. Testers can capture the state of the failure. This is more complete than a description. Developers can start the container in failed state and debug.

Why This Matters CI/CD Cannot work in a meaningful way without a quick way to pass information Bug interpretation by the tester takes time Version matching environments takes time Developers understanding the issues takes time Getting the application in the right state takes time Following a CI/CD doctrine can make you fail faster. Docker can tell you why you failed faster.

Using Docker to Talk to Developers Sample of an old style bug report What a bug report could look like. Title: Something didn’t work Description: The thing didn’t work Release number: 1.2.2.1 Expected Behavior: The thing should work Steps to reproduce: 1.Do stuff 2.Do more stuff Expected Behavior: The thing should work and it failed when I tried X Image ID: SOME LONG GUID

Using Docker to Talk to Developers System Version 6030ba23a646dad11c3d9fc7ce7b7adb2fbbc92ffee87e82b8667728a74cec2a Logs System State Description of Event

What do I, as a tester, need to know? High level overview of how docker functions. What a docker commit does. The difference between an image and a container. What is a private repository. Don’t air your dirty code in public. How to do or have someone automate a docker push to a private repo. How this can change your defect reports. More concise, rely on the container

Docker Testing and Agile How does this “fit” into agile By reducing the testing feedback cycle meaningful manual testing can fit within sprints Slight change of story/epic format User Story The test case Container id Sprint planning can demand certain features and branches be tested in a particular container for UAT. Clearly sets deployment expectations Can be part of the definition of done, must deploy and work on container X

Adoption Docker is new and I don’t have it. This is an opportunity, you can make testing be part of the process. Upfront implementation is cheap and you can guide process. We have docker but they didn’t think of testing Show with a private repo it can be a cheap add for great gain We already do what you are suggestion, no actually it's cooler OK, Let me take you out for lunch.

Conclusion As a tester I do not need to know about COW filesystems, orchestration, swarms, or any other advanced containerization topic, just basic concepts. If we implement docker I should be able, as a tester, to get command line access to the container to help get more information for my defect reports. (Software Version, container version, log access) Using an image of a broken container is an extremely effective addition to my defect tracking. Containers can allow me, as a tester, to communicate more information with developers effectively.

Questions?

References Containerization Definition - https://en.wikipedia.org/wiki/Operating- system-level_virtualization