Ref SW & Testbed Mini Workshop June 2000 Software Process & Environment in Atlas Back-end Doris Burckhart CERN ATLAS DAQ/EF-1 Back-end software

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 2 Content Introduction Why a software process The Back-end DAQ Software The Back-End SW Development Process Organisation Phases and Deliverables Software Management Review and Inspection Summary Lessons learnt Summary

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 3 Why a Software Process Define the obvious, plus... define and meet user requirements define structured but flexible work-plan defining responsibilities define interfaces between components define interfaces to external systems Closely follow-up on progress avoids unrealistic goals tracks problems early improves communication provides basis for iterative development Some initial points

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 4 Software Development Version Control Software Release Tool Software Development Process Common Working Culture Standards Agreements Rules Basic Framework Software Development Environment checklists Documentation Configuration Management Testing Tools Development Tools

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 5 Back-End Integration in Atlas online

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 6 Back-end Components

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 7 Back-End Software Process ‘ Light’ Software Process We are not Gurus, just concerned developers based on ‘best effort’ approach - not perfect not formally described in a document but it is on the Web ! -> easy to be updated and to work with Structured Organization - Helpful Framework Process relies on division into component and on a development structured into phases Define and agree on common goals -> community feels responsible for the results -> increased communication Collaborate with Atlas offline where possible coding rules

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 8 SW Development Process identify needs & common issues, define priorities and work-plan, define components perform pre-design investigations -> identify candidate technologies/techniques develop high-level design refine design, implement code 180,000 lines of C++ 40 % generated unit test components integrate with other subsystems Total lines incl. external sw Collect Requirements Pre-design Investigations High-level Design Testing & Integration Detailed Design Implementation Training, Programming and Testing Tools, FrameMaker, html, SRT Configuration Management, StP/OMT CASE TOOLS Software Development Environment

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 9 Organization /1 Work is organized around components small groups dedicated to each component (up to 4 developers) one coordinator for each component prefer one institute per component -> clear boundaries of responsibilities most developers follow a component all the way through its lifecycle look for commonalties between components - don’t duplicate functionality take / reuse as much as possible from colleagues: code, ideas, documentation, templates, examples

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 10 Organization /2 Components developed according to an agreed priority started with core components (run control, config. Databases,...) Then working on additional components (I.e. Online Book- keeper) Component independent parts Software Management (I.e. SRT, CVS) Use of external software (Corba/ILU, Rogue Wave Tool.h++, CmdLine, CHSM) - one developer responsible for each package Supported platforms: those agreed on in the Atlas DAQ community constraint: minimize number of platforms because each platform represents a lot of work for the build and releases of the SW

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 11 Templates and Guidelines Doc. Templates developed within the project generic technical note (FrameMaker) test plan - test report Doc. Templates taken from IP/IPT group user requirements users guide Check-lists and guidelines brief requirements, design and general doc.check-lists C++ coding standards pointers to short easy-to-read ideas about design and testing by Gurus on the Web Simple “how-to’ instructions for most commonly used tools (e.g.SRT)

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 12 Software Management Back-end software grouped into components components map on to SRT packages all packages together form a software repository Test + Build Environment Tools + Release Tools -> Release nightly or official

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 13 Tools: CVS & SRT CVS : Concurrent Version System Used as Code Management System Includes Version Control and Management Keeps track of iterations Control and track changes in our development environment while we are working SRT: Software Release Tools Organizes the development effort at the level of the software as a whole Builds and releases products such as libraries and executables Groups our software as a collection of specific versions of package files - for day to day work and for official release

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 14 Nightly builds Automatic, regular - early problem finding developers commit modified sw to cvs every night new version of the sw in the repository is build on each platform. runs from a batch job each working day during night the latest version of each package is checked out and compiled and linked on each platform. check target is executed. authors of a package will receive an automatically if the build process failed for their package - to be corrected a.s.a.p. The results of the build process are held in temporary log files visible through the web page all binaries, objects and libraries produced by the nightly build will be removed before the next nightly build.

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 15 Official Releases Release every 2 months - time-consuming package developer: steps as for nightly, plus identification (tag) release notes thorough integration testing (2- 3 weeks) represents lots of work for the software librarian each release is given a unique identifier distributed on CD-ROM

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 16 Iterative Development Cycle Document Inspection Code Inspection Document Inspection Applying Testing Tools Code Inspection Requirements Design Test Implementation Documentation Test Plan Implementation Unit Test Use, Maintenance of a Component Integration Test Feedback other components

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 17 Review - Inspection Review: Presentation of each SW Component to the Group in each Development Phase Discussion and Coordination with other components Goal: Clarification and Accept/Reject Decision Goal: Clarification and Accept/Reject Decision Inspection: Quality Improvement Process to the software project Goal: Defect Detection & Defect Prevention Goal: Defect Detection & Defect Prevention

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 18 Informal Review From the start of the project document preparation and monthly open meetings present status, results, proposals inform colleagues - receive feedback suggestions -> enhancements -> acceptance Results: Coherent set of end-product components Increased communication Drawback: Lack of time of reviewers No code review

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 19 Inspection - Objectives Defect Detection documents are checked for cleanness and consistency against rules Defect Prevention learning from defects found suggesting improvements On the Job Training education in standards and rules apply creativity

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 20 Sources Inspection Process Map Inspection Team: Inspection Leader Authors Inspectors Planning & Entry Kick-off Meeting Checking Logging Meeting Brainstorming Edit Exit Follow-up Inspection Plan Issue log tables Action Lists Exit Product Data Summary Change Requests Product Rules Checklists Based on Tom Gilb’s Inspection method

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 21 Results 3 Requirement Inspections 2 Design Inspections 5 Code Inspections Total : ~ 200 pages of documents, lines of code Each issue is logged, discussed, checked Emphasis on non-trivial issues Per Inspection : 10 to 200 issues found Number of recorded issue logs depends on : Type of Inspection, Phase of Project, Entry conditions, Experience of Inspectors, Counting system -> Improved Code -> Improved Documentation

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 22 Inspection Process Results Change Requests To Rules for Requirements, Design or Coding To the Inspection Procedure Action Lists Actions to be performed outside the Inspection Process Questions to be clarified, i.e. beyond the scope of Inspection Observations: Requirements: most important, the first in the chain: a bug may propagate to the end -> unwanted results even with perfect code Design: the hardest to inspect, difficult to provide a good set of guidelines Code: most time consuming: Code & Documentation, mother documents & reference documents, Need good set of rules, Use of automatic checking tools

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 23 Adaptation to Environment Everything is allowed which helps improving process product communication cooperation, education integration, coherence while keeping Consistency and improving Efficiency HEP: geographically distributed no specialized expertise little formal training little hierarchical power participation by conviction

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 24 Efficiency - Flexibility Efficiency Inspection is time consuming - don’t waste time and effort of inspectors Careful planning and Clear Instructions Solid Process Framework Inspect Samples, ‘light’ Inspections Motivation of peers Flexibility Build in change Management Well defined procedure - but each inspection to be handled individually

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 25 Experience Inspection is Take fears seriously Explain aims Respect privacy Demonstrate helpfulness Participation Trust amongst colleagues Constructive criticism Integration Common working culture Fear to be criticized and judged

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 26 Lessons Learnt: Organization Component model is effective Clear interfaces allows parallel development by multiple groups Small group dedicated to each component (max. 4 developers) One coordinator for each component One institute per component works best Component development based on agreed priority (core components first) Look for commonality between components - avoid duplication Centralised integration is best Need people and tests dedicated to this task Must have good collaborative tools Code repository, release management, web site, mailing lists etc. External software (Corba/ILU, Tools.h++, CmdLine, CHSM etc.) One developer responsible for each external package

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 27 Lessons Learnt: Software Process Software process is important do start gently (can’t go from chaos to Nirvana in one step) do tune your software process according to your project do keep it simple and stick with it (don’t abandon ½ way through) do review/inspect every deliverable (requirements, design, code etc.) do provide templates, checklists and examples for all deliverables do get a non-author to perform component testing do encourage developers to share their work and ideas Things to avoid don’t ask developers for a deliverable unless it will be used don’t underestimate time & effort for SW mgmt, integration & test.

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 28 Lessons Learnt: Software Releases Each one must be better than the last Iterative development model reduces errors and risks Use feedback to drive/focus work Nightly on all supported platforms - build & test Official - coincide with Back-end meetings: discuss status of last release and contents of the next More platforms == more work Implies every developer has access to every platform Try to keep the list short - agree within experiment Software librarian != developer Librarian is not there to fix faults in the software Web page gives access to build log for each component A release is an important milestone and a lot of work for everybody

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 29 Summary We need this working SDP & SDE as a helpful framework and to assure quality to hold software efforts from Institutes together, collect fractions of manpower to keep development of components in line to keep track of modification and versions to always know where we are, keep documents Reviews prepare the ground and stabilize SDP Adaptation of the Inspection method to the Environment is necessary Use Review and Inspection through entire SW life cycle Gain in quality and experience, appreciated by authors and peers Help for team building in a distributed environment

June 2000 Software Process and Engeneering Environment in Atlas Back-End - Doris Burckhart 30 References Atlas DAQ technical notes Software Development Environment -> pointers to all relevant topics mentioned in this presentations, like Component Development Procedure how to use SRT introduction to CVS and more...