1. Use Docker to Enhance Your Testing
Glenn Buckholz
Coveros, Inc.

2. Agenda Intro Docker Overview How Docker Changes Testing
DevOps Approach
Demonstration
Using Docker to Talk to Developers
Speeding Up The Feedback Cycle
Questions

3. Introduction Docker is a popular form of containerization
Containerization is a server virtualization method in which the kernel of an operating system allows the existence of multiple isolated user-space instances, instead of just one.
Other Container Technologies
Jails - BSD
Chroot - Filesystem Level
Xen paravirtualization - Processes Hypervisor
LXC or LXD - Linux Containerization
Virtualization vs Containerization
Virtualization allows many OS images to use the same hardware.
Containerization allows many pieces of software to use the same OS in an isolated fashion.

4. Docker Overview Docker has 3 major components Images
Filesystem of the container.
Containers
Running instance of an image.
Registries
Repository of images.
Docker Hub is a global public repository.
Local Registry on the docker host itself
Private Registry
Private Registry
Image
Container

5. Docker Overview Docker Host
Hardware and OS that is shared among containers.
Docker Service multiplexes the OS
NIC’s
Devices
Storage
CPU limits
IO limits
Intra Host Communication HA
Scaling
Load Balancing

6. Docker Overview Git version control docker commit and docker push
Promoting containers
When the container is where/what you want
docker tag imageid registryhost:port/tagname
docker push registryhost:port/tagname
Make the environment change around the container. Keep the container constant.
Use mount points for configs
Use mount points for logs
Make the container network agnostic
Make the thing that you put in production the thing you had in dev and test.

7. DevOps Approach Dockerfile creation Automate from known stock images
Quality Gates
Container “Unit Tests”
Not strictly unit tests but you are testing one high level piece of functionality.
Container “Integration Tests”
Use docker swarm or some other orchestration tool to put together a temporary environment, run minimal testing.
Container “Nightly Testing”
Run a full set of automated tests overnight.
Container “Manual Testing”
After nightly testing release the containers to manual testers.

8. DevOps Approach Hourly (Automatic) Quality Gate Unit, Integration
Quality Gate UI testing
Nightly
Release Candidate
Release

9. DevOps Approach
Promote the container through some or all the different stages automatically.
Dev and test should have high levels of automation.
Each quality gate is automatic and triggered by changes to the Dockerfile OR the underlying application.
Failures are preserved for some time (24-72 hours)
Dev
Test
Stage
Prod
Failure

10. Demo Tester Developer Interaction Demo
“I need a blue seal balancing on its flippers with an eye.”
5 Min max
Have test prop with a picture of the ideal behind the projector screen
Have an index card with obvious answers under it.
Give the volunteer 15 seconds to look at the picture and the answers.
Ask him to recite the defects verbally.
Argue about the colors.
Pause and count the number of agreed defects and disagreed defects.
Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item.
Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

11. Demo - What the Developer Made
5 Min max
Have test prop with a picture of the ideal behind the projector screen
Have an index card with obvious answers under it.
Give the volunteer 15 seconds to look at the picture and the answers.
Ask him to recite the defects verbally.
Argue about the colors.
Pause and count the number of agreed defects and disagreed defects.
Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item.
Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

12. Demo - What the PO Wanted
5 Min max
Have test prop with a picture of the ideal behind the projector screen
Have an index card with obvious answers under it.
Give the volunteer 15 seconds to look at the picture and the answers.
Ask him to recite the defects verbally.
Argue about the colors.
Pause and count the number of agreed defects and disagreed defects.
Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item.
Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

13. Demo - The difference Issues The shade of blue is wrong
The eye is on the wrong side
They are mirror images of each other
The Seal is balanced wrong
Problems
I had to specifically articulate each issue.
Some of these issues don’t translate well in words.
5 Min max
Have test prop with a picture of the ideal behind the projector screen
Have an index card with obvious answers under it.
Give the volunteer 15 seconds to look at the picture and the answers.
Ask him to recite the defects verbally.
Argue about the colors.
Pause and count the number of agreed defects and disagreed defects.
Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item.
Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

14. Demo - Or I Can Just Compare the Two
5 Min max
Have test prop with a picture of the ideal behind the projector screen
Have an index card with obvious answers under it.
Give the volunteer 15 seconds to look at the picture and the answers.
Ask him to recite the defects verbally.
Argue about the colors.
Pause and count the number of agreed defects and disagreed defects.
Talk restart the scenario and have the person say “I think there is an issue.” and just bring out the item.
Quickly list all the bugs and point out the paper texture is different and they are mirror images of each other.

15. Speeding Up Feedback Cycles
Developers can have complete access to containers to look at logs
Failed containers can be saved to a private registry
Testers can demo the defect on demand.
Old Way:
New Way:
Fight About it
Compromise
Find Bug
Fix it
Find Bug
Demonstrate
Fix it

16. Demo
Technical Demonstration

17. Demo
Sample Application Architecture
Tomcat 9
MariaDB

18. How Docker Changes Testing
The operating system is domain of specialists due to the impact of one wrong change.
The host should still be maintained by specialists.
Containers put barriers between the software stack and the OS.
Allow the testers direct access to the containers.
Containers can now be tagged with test artifacts.
Test case failed.
Code branch.
Testers can capture the state of the failure.
This is more complete than a description.
Developers can start the container in failed state and debug.

19. Why This Matters
CI/CD Cannot work in a meaningful way without a quick way to pass information
Bug interpretation by the tester takes time
Version matching environments takes time
Developers understanding the issues takes time
Getting the application in the right state takes time
Following a CI/CD doctrine can make you fail faster.
Docker can tell you why you failed faster.

20. Using Docker to Talk to Developers
Sample of an old style bug report
What a bug report could look like.
Title: Something didn’t work
Description: The thing didn’t work
Release number:
Expected Behavior: The thing should work
Steps to reproduce: 1.Do stuff 2.Do more stuff
Expected Behavior: The thing should work and it failed when I tried X
Image ID: SOME LONG GUID

21. Using Docker to Talk to Developers
System Version
6030ba23a646dad11c3d9fc7ce7b7adb2fbbc92ffee87e82b a74cec2a
Logs
System State
Description of Event

22. What do I, as a tester, need to know?
High level overview of how docker functions.
What a docker commit does.
The difference between an image and a container.
What is a private repository.
Don’t air your dirty code in public.
How to do or have someone automate a docker push to a private repo.
How this can change your defect reports.
More concise, rely on the container

23. Docker Testing and Agile
How does this “fit” into agile
By reducing the testing feedback cycle meaningful manual testing can fit within sprints
Slight change of story/epic format
User Story
The test case
Container id
Sprint planning can demand certain features and branches be tested in a particular container for UAT.
Clearly sets deployment expectations
Can be part of the definition of done, must deploy and work on container X

24. Adoption Docker is new and I don’t have it.
This is an opportunity, you can make testing be part of the process.
Upfront implementation is cheap and you can guide process.
We have docker but they didn’t think of testing
Show with a private repo it can be a cheap add for great gain
We already do what you are suggestion, no actually it's cooler
OK, Let me take you out for lunch.

25. Conclusion
As a tester I do not need to know about COW filesystems, orchestration, swarms, or any other advanced containerization topic, just basic concepts.
If we implement docker I should be able, as a tester, to get command line access to the container to help get more information for my defect reports. (Software Version, container version, log access)
Using an image of a broken container is an extremely effective addition to my defect tracking.
Containers can allow me, as a tester, to communicate more information with developers effectively.

26. Questions?

27. References
Containerization Definition - system-level_virtualization