EMI INFSO-RI EMI Quality Assurance Tools Lorenzo Dini (CERN) SA2.4 Task Leader

EMI INFSO-RI Task Goals Initial situation Build, Integration, Packaging Build and Test Infrastructure Software Repositories Continuous Testing Quality Assurance – Metrics Workflow – QA Reports 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 2 Outline

EMI INFSO-RI EMI QA tools not only for Quality Assurance Single tool-chain and workflow for – Build, integration, Packaging, Testing, QA Unified project builds in a single box Single release tool Single dependency management Uniform and compliant packaging Tests stored together with configuration QA metrics created in builds 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 3 SA2.4 - EMI QA Tools

EMI INFSO-RI LanguagesRelease Integration Dependency Management PackagingTestingBug Tracker 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 4 Initial Situation

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 5 Survey Results 1

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 6 Survey Results 2

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 7 Survey Results 3

EMI INFSO-RI Different cultures, environments, requirements and priorities led to different tools difficult to merge Uniform the build environment to provide a common software engineering workflow without affect developer efficiency Provide a comprehensive infrastructure able to satisfy the totality of needs Introduce testing, QA metrics generation and reporting in the same tool-chain Support everybody on a new system 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 8 Challenges

EMI INFSO-RI Taken the versatility of the gLite ETICS system as a general framework Adopted the ARC dependency management with EPEL and packaging with Mock (Debian with PBuilder) Configured minimal worker nodes as dCache with packages installed as privileged user at build start and build in user space Kept Maven to uniformly and effectively build Java components as UNICORE 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 9 Build, Integration and Packaging

EMI INFSO-RI Every EMI software component is configured in a single ETICS project and organized in releases The whole release is built from source in a single minimal clean node 4 times a day ETICS checks-out the source code, installs the dependencies using YUM/APT and triggers the build systems (Autotools, Ant, Maven) in user space ETICS creates missing SPEC files and produces RPMs and DEBs ETICS triggers Mock/PBuilder in a pristine environment for the packaging 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 10 Build, Integration and Packaging

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 11 Build, Integration and Packaging

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 12 Build, Integration and Packaging

EMI INFSO-RI Virtual infrastructure based on CERN Microsoft HyperV and Condor >50 VMs in >10HVs available at any time for builds and tests VMs are automatically scratched after jobs to ensure clean environments Some high performance nodes have been dedicated to project builds to minimize the build cycle 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 13 Build and Test Infrastructure

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 14 Build and Test Infrastructure

EMI INFSO-RI EMI software repositories are provided for each release and release candidate These repositories are automatically populated after builds An EMI external repository is also provided for packages not available in the OS (SL, EPEL, Debian) Automatic ETICS repositories are also generated with the contents of the builds 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 15 Software Repositories

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 16 Continuous Testing “The gLite Data Management Continuous Integration and Testing Process” – 14:30

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 17 Quality Assurance The goal of Process Metrics is to identify improvement opportunities in the way the software is developed and tested, from a process point of view. In our context the main sources of process metrics are the Defect Tracking tools used in EMI. Product (or Internal) Metrics measure certain characteristics of the product like complexity, changeability and testability. At the moment we rely on static code analyzers to measure these characteristics. ”Software Metrics Defines, Reports and Analysis in EMI” – 12:00

EMI INFSO-RI QA Metrics Workflow school.scc.kit.edu/img/dCache_logo.gif XSLT XSD

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 19 Bug Tracker exporters “Software Quality Assurance in EMI” – 11:00 XSLT XSD The middleware distributions decided not to merge their tracking systems Agreements on fields, states, workflow and interfaces to track the process Exporters query each system and provide a standard XML document with the data This documents are stored and backed up on servers provided by distributions

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 20 QA Plug-ins QA Plug-ins are ETICS modules that run during the build and trigger common static analysis tools for QA SLOCCount, FindBugs, PMD, Checkstyle, PyLint, RPMLint, CPPCheck, etc... Product QA Metrics are produced and stored permanently in the ETICS repository

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 21 QA Report Generator Queries the XML for the process metrics and the ETICS repository for the product metrics creating aggregated metrics. Generates charts, plots and tables with QA information and fills in QA templates with the dynamic information. Reports can be produced daily for Product Teams and Release Manager Produces drafts for EU QA Deliverables

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 22 QA Reports: Process Metrics

EMI INFSO-RI /04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 23 QA Reports: Product Metrics

EMI INFSO-RI A very heterogeneous starting point Uniform software engineering tool-chain Successfully integrated EMI 0 and EMI 1 New integrated testing and QA process All performed improvements benefit all teams at once. Specific solution can make a single team happier but are globally more expensive and not maintainable. 12/04/2011 EGI User Forum - EMI Technical Forum - April Vilnius 24 Conclusions

EMI INFSO-RI Thank you 12/04/ EGI User Forum - EMI Technical Forum - April Vilnius EMI is partially funded by the European Commission under Grant Agreement INFSO-RI