Tools and software process for the FLP prototype B. von Haller 9. June 2015 CERN

Overview ▶ Scope ▶ FLP prototype ▶ O2 ▶ Software development ▶ Software quality ▶ Software deployment ▶ Continuous Integration ▶ Specific to FLP proto : B. von Haller | CWG11 |

▶ Operating Systems ▶ SLC6, CC7, Ubuntu, Mac ▶ Compilers ▶ Gcc, llvm, icc ▶ Languages ▶ Low level : C ▶ Middle level : C++11, stl, boost libraries, ROOT6 ▶ Open to higher level languages (e.g. python) ▶ Expose interfaces using generators such as SWIG ▶ Web : PHP or Python with Django. To be decided Ecosystem B. von Haller | CWG11 | Only Limited to these 2

Software development ▶ Tools ▶ Version control system: Git with GitLab/GitHub ▶ Software build system: CMake ▶ Issue tracking system: JIRA ▶ Code formatter: clang-format ▶ Procedures ▶ Workflows  A la GitHub (pull request, see next slides) B. von Haller | CWG11 | GitLab to be gradually introduced

Software development GitHub development workflow B. von Haller | CWG11 | ▶ Collaborators have a full remote clone of the main repo, called a fork ▶ Full rights on their fork, none on the main repo ▶ Anything in the master branch is always deployable. ▶ Owners of the main repo (librarians) comment and accept/reject the pull requests ▶ Open source clone of GitHub: GitLabs

Software development GitHub/Gitlab B. von Haller | CWG11 | ▶ Advantages of GitHub/GitLab and the related development workflow ▶ Collaborative and intuitive web interface ▶ Code always reviewed and commented before being accepted ▶ Code can be checked automatically by tools before human reviews ▶ Feature-rich api

Software development ▶ Procedures (continued) ▶ Workflows  A la GitHub (pull request, see next slides) ▶ Coding conventions, naming, formatting -> CWG2 ▶ Code access policy  Open (license still under discussion) -> CWG2 ▶ Repositories ▶ Neither monolithic nor excessively fragmented ▶ Group components that share same dependencies and/or similar functionalities ▶ Packages ▶ Does not always match with repositories (also more flexible) ▶ Release pace or common dependent packages to take into account ▶ Dependency handling for developers ▶ External : original + patch -> recompile (tool to be decided) ; packages ▶ Internal : packages (avoid recompiling what does not need to) ▶ CernVM-FS B. von Haller | CWG11 |

Software quality ▶ Tools ▶ Static analysis -> Clang Static Analyzer ▶ Dynamic analysis -> Valgrind ▶ Tool for code reviews -> GitLab ▶ Code conventions and formatting enforcer -> clang-format B. von Haller | CWG11 | Not initially

Software quality ▶ Testing ▶ Unit tests, usually at the function level ▶ CTest and boost.test ▶ Functional tests ▶ Integration tests, software modules are combined and tested as a group. Full chain. ▶ Requires the code to be designed for it and to provide mock systems at the interfaces ▶ Environment must be controlled B. von Haller | CWG11 |

Software deployment ▶ Packaging ▶ CPack or fpm (to be decided, evaluation ongoing) ▶ Dependency manager for users ▶ Packages using native system offered by OS ▶ CernVM-FS B. von Haller | CWG11 |

Continuous Integration (CI) ▶ Actually wider than pure CI ▶ Inter connects development, quality and deployment ▶ Jenkins ▶ Extensible, open source CI server ▶ Hundreds plugins, wide industry support ▶ Can do many type of tasks (e.g. build, test, release) ▶ On scheduled time or following a trigger (e.g. pull request) ▶ API to build our own tools in front or make other tools collaborate (e.g. JIRA) B. von Haller | CWG11 |

Continuous Integration ▶ Master hosted in IT for ALICE on a VM ▶ Use VMs with Docker on Openstack ▶ Authentication using CERN SSO and egroups ▶ Mesos deployment containers to deal with non-homogeneous tasks and load ▶ Integration with JIRA and GitHub/GitLab ▶ Time scale : infrastructure ready this summer Jenkins for O 2 B. von Haller | CWG11 |

Conclusion ▶ Tools have been chosen and procedures defined to ensure ▶ Code quality, readability and maintainability ▶ Products delivery, stability and performance ▶ The FLP prototype development can start in July directly within this frame B. von Haller | CWG11 |