1. Revision Control, Automated Testing and Docker RSE Conference 2017
Chris Richardson

2. Revision Control and Testing
FEniCS/dolfin software project - on bitbucket - using git revision control
Test suite (using pytest, google test etc.)
test folder in repository - can run with pytest
Who runs the tests: where and when?
Ideally, developers run the tests when they are working on the code, but...

3. Revision Control with git and Bitbucket

4. pytest def test_UnitCubeMeshDistributed():
"""Create mesh of unit cube."""
mesh = UnitCubeMesh(mpi_comm_world(), 5, 7, 9)
assert mesh.size_global(0) == 480
assert mesh.size_global(3) == 1890
def test_UnitCubeMeshDistributedLocal():
mesh = UnitCubeMesh(mpi_comm_self(), 5, 7, 9)
assert mesh.num_vertices() == 480
assert mesh.num_cells() == 1890

5. so… let’s run the tests make runtests
build 24-Aug :13:50 adaptivity/test_error_control.py::test_error_indicators Solving linear variational problem. build 24-Aug :13:50 PASSED build 24-Aug :13:50 adaptivity/test_time_series.py::test_retrieve_all_connectivities PASSED build 24-Aug :13:50 adaptivity/test_time_series.py::test_subdirectory PASSED build 24-Aug :13:50 ale/test_harmonic_smoothing.py::test_HarmonicSmoothing Current CG solver produced error in boundary coordinates 0.0 build 24-Aug :13:50 PASSED build 24-Aug :13:50 ale/test_harmonic_smoothing.py::test_ale PASSED build 24-Aug :13:50 common/test_timer.py::test_context_manager_named PASSED build 24-Aug :13:51 common/test_timer.py::test_context_manager_anonymous PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun0-1b6f94ab458b b7dff] PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun1-be92f892d82d4002b3f1e03ff1a1f030] PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun2-a2080c717cf94d4788c8e477682f4a9d] PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun c83eae4411bd2da394badff39e] PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun4-a2080c717cf94d4788c8e477682f4a9d] PASSED build 24-Aug :13:51 common/test_timer.py::test_decorator_functionality[fun c83eae4411bd2da394badff39e] PASSED build 24-Aug :13:52 common/test_timer.py::test_decorator_timer_scope PASSED build 24-Aug :13:52 common/test_timer.py::test_decorator_timing_correctness PASSED build 24-Aug :13:52 fem/test_assembler.py::test_cell_size_assembly_1D PASSED build 24-Aug :13:52 fem/test_assembler.py::test_cell_assembly_1D PASSED build 24-Aug :13:52 fem/test_assembler.py::test_cell_assembly PASSED build 24-Aug :13:52 fem/test_assembler.py::test_facet_assembly PASSED build 24-Aug :13:52 fem/test_assembler.py::test_ghost_mode_handling PASSED build 24-Aug :13:52 fem/test_assembler.py::test_functional_assembly[mesh_factory0-4.0] PASSED build 24-Aug :13:52 fem/test_assembler.py::test_functional_assembly[mesh_factory1-6.0] PASSED build 24-Aug :13:52 fem/test_assembler.py::test_functional_assembly[mesh_factory2-4.0] PASSED build 24-Aug :13:52 fem/test_assembler.py::test_functional_assembly[mesh_factory3-6.0] PASSED build 24-Aug :13:53 fem/test_assembler.py::test_subdomain_and_fulldomain_assembly_meshdomains[mesh_factory0] PASSED build 24-Aug :13:53 fem/test_assembler.py::test_subdomain_and_fulldomain_assembly_meshdomains[mesh_factory1] PASSED build 24-Aug :13:53 fem/test_assembler.py::test_subdomain_assembly_form_1 PASSED build 24-Aug :13:53 fem/test_assembler.py::test_subdomain_assembly_form_2 PASSED build 24-Aug :13:53 fem/test_assembler.py::test_nonsquare_assembly PASSED build 24-Aug :13:54 fem/test_assembler.py::test_reference_assembly PASSED build 24-Aug :13:54 fem/test_assembler.py::test_ways_to_pass_mesh_to_assembler PASSED build 24-Aug :13:54 fem/test_dirichlet_bc.py::test_instantiation PASSED

6. Automated Testing Run every time a change is made to “master” branch
Run on some development branches
Build code with all optional dependencies (and without)
Run tests with MPI (and without)
Run Python tests with Python2 (and Python3)
Run C++ tests
Run demos (as well as tests)
Check code coverage

7. A framework to run tests in: docker containers
Docker containers encapsulate an environment.
Create a “development” environment, with all required compilers, libraries and 3rd party software needed to build and test our package
Dockerfile:
Push to online “docker registry”
Linux based (so does not cover e.g. MacOS, Windows)
Ubuntu based (so not covering other distributions)
docker build --tag fenicsproject/dev-env:latest . && docker push
FROM ubuntu:16.04
RUN apt-get install cmake g++ libboost-dev-all etc.

8. How it all fits together
git repositories
docker registry
FEniCS/docker
Dockerfile
build hook
dev-env
dev-env
CI system
Build
FEniCS/dolfin
run hooks
master
Run tests
feature branch
Produce report

9. How it all fits together
FEniCS/docker
Dockerfile
build hook
dev-env
dev-env
Build
FEniCS/dolfin
run hooks
master
Run tests
feature branch
Produce report

10. Hooking in to a CI system
Simply add a .yml file to your repository, and sign up to the service
version: 2 jobs: build: docker: image: quay.io/fenicsproject/dev-env:latest working_directory: /home/fenics/working steps: checkout run: name: Configure DOLFIN command: mkdir -p build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Debug -DDOLFIN_ENABLE_TESTING=ON run: name: Build DOLFIN and install command: cd build && make install run: name: Run Python unit tests (serial) command: python3 -m pytest test/unit/python

11. What it looks like on the CircleCI website...

14. Price comparison: online services
There are a lot of services out there… most offer a free tier, free for open-source, or academic discount. But it is never enough!!!
CircleCI: free for Open-Source (up to 4 nodes) then $$
Bitbucket Pipelines: free 500 mins per month, then $10+ per 500 minutes
Shippable: free for one node, then $25/month per node
Codeship: $75/month (50% educational discount)
Codefresh: 5 nodes, 200 builds per month for free
Distelli
Travis
Vexor: $1.5¢ per minute
They are multiplying like bacteria… Google will find many more…

15. Another option: DIY hosting
Could use Jenkins to orchestrate, but...
We are using Bamboo (Atlassian Product, free to Open Source projects)
Install docker and bamboo on a server
Configure to poll your git repository
Integration with bitbucket website
Extra complexity - maintenance burden
Extra complexity - more flexibility
Unlimited build hours/nodes
Can add dependencies between different modules

16. How it all fits together (again)
build hook
FEniCS/docker
Dockerfile
dev-env
dev-env
Build
FEniCS/dolfin
run hooks
master-build
master
branch-build
Test
feature branch
Report

18. Summary git + docker + CI is a powerful combination
I am not advertising or endorsing any particular product!
Free online services are great for small projects 😃
More complex projects may benefit from running on a local server
Pushing back to a docker registry after the “build” stage has benefits:
We can run all the tests in parallel
If the tests fail 😓, we can docker run the build image on another machine and reproduce 😌
This is not all my own work. Special thanks to Johannes Ring in Oslo.