1. Geant4 Software Process 3rd LCB Computing Workshop, Marseilles 29 September, 1999 John Apostolakis (CERN) for Geant4 with thanks for material and suggestions to Katsuya Amako, Simone Giani

2. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19992 Overview of the talk Aspects of Geant4 software process Context: Worldwide Collaboration Software Methodology  Macro and Micro Processes  OOA&D Testing, Quality Assurance Software Tools

3. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19993 Geant4 Overview What is Geant4? Toolkit to simulate the passage of particles through the matter. It’s for Monte Carlo simulations of detectors in high energy physics. But also for space and cosmic rays applications, nuclear and radiation computations, heavy ions and medical applications. How has it been developed? Development based totally on the object-oriented software technology. Benefit from experience and the algorithmic techniques accumulated in GEANT. Worldwide collaboration of ~100 physicists, engineers, computer scientists from 15 countries.

4. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19994 Software Methodology Why software methodology? For development of a complex software like Geant4, introduction of an engineering discipline is essential OO methodology à la Booch employed because it provides a very pragmatic approach has easy-to-understand models with rich notations. provides a way to design a system into independent subsystem  allows efficient work sharing in design and implementation stages.

5. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19995 OO Methodology a la Booch: Benefits Common-sense approach: Common-sense approach: with an incremental and iterative process Clear models with rich notations fill the gap between design and implementation. Clear models with rich notations fill the gap between design and implementation. these models/notations help to exchange idea of design between people dispersed over the world Seperation into independent subsystems Seperation into independent subsystems allows efficient job sharing scheme can be defined both in design and implementation stages.

6. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19996 Software development in Booch method Structure of the software process: Macro process  This provides the framework for a global software process.  Borrowing from the waterfall and spiral model Micro process  This provides the framework for a fine structure of the software process, which consist of an iterative and incremental approach.  Similar to spiral model

7. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19997 The Macro Development Process OO Analysis Requirement Maintenance OO Design Evolution Establish core requirement Develop a model of the system’s desired behavior Create an architecture for the implementation Manage post-delivery evolution Evolve the implementation through successive refinement

8. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19998 Application of Booch method to Geant4 Basic principle Don’t blindly follow the method.  There is no “silver bullet” in constructing a complex software system. Judge ourselves which elements of the method are important and applicable to our project.  Adapt not adopt  Booch method is flexible enough to allow this.

9. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 19999 Macro Process: OO Analysis Goal: To identify all major objects/classes in the problem domain. To produce a central model (class diagram) using objects/classes already found. To identify clusters of classes that are themselves cohesive, but are loosely coupled relative to other clusters. Major Products: Requirements document (ESA PSS-05: URD v.06) Class diagram - analysis phase  Scenario (Object/Interaction) diagram - analysis phase Scenario (Object/Interaction) diagram - analysis phase Class category diagram (preliminary)

10. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199910 Macro Process: OO Analysis - continued OO analysis and the worldwide collaboration: Close collaboration of people dispersed worldwide from the beginning stage of OOA. The fundamental design of Geant4 was totally defined in this phase. In Geant4 a core team was 6 / 7 people played an essential role.  As the number was small, the core people could work very closely together even from various countries.  Yet the number was large enough to bring together different expertise (and was 1/3 of Geant in 1994)  no clear rule of thumb

11. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199911 Macro Process: OO Design Goal: To refine the diagrams produced in the OOA phase using the computing domain classes and objects (for example, manager objects, interface objects, etc) so that objects and classes can be coded and executed. [Note]  OOA and OOD progressed concurrently in most category developments.  The 2/3 of the first year was spent for OOA and OOD. Major Products: Object diagram - design phase Scenario (Object/Interaction) diagram - design phase

12. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199912 Macro Process: OO Design - continued Set of classes that are closely coupled make a Class Category.Class Category. Class categories are cohesive and have looser couplings with each other.  We used class category as a unit to share tasks in OOD (and also in the implementation phase).  Loose couplings enabled each category to work relatively independent.  Essential for worldwide collaboration Class Category Diagram had a fundamental importance in the worldwide collaboration. Class Category Diagram  Much larger number of people could work together under each category. [Note]  Class category is NOT a functional separation of the software system - well defined interface to other category.  Class category is not a class wrapping “FORTRAN”.

13. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199913 Class Category Diagram Run Geant4 Readout Event Event Gen. Digi Hit Magnetic Fields Magnetic Fields Tracking Detector Physics Track Material Particle Definition Particle Definition Visualization ODBMS Utilities Geometry CAD Interface CAD Interface

14. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199914 The Micro Development Process Identify classes and objects Specify class and object interface and implementation Identify class and object relationships Identify class and object semantics

15. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199915 The Micro Development Process It deals with the OO analysis & design at the small, detailed level In contrast with large scale macro development process Great similarities in its process with macro process analysis and design detailed class diagrams Tools (micro & macro) CASE Rose/C++  extensive use for OOA/OOD stage

16. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199916 Macro Process: Evolution Goal: Construct the production version via incremental unofficial and official releases. Provide usage examples and user’s documents. Major Products: Refined OOA/OOD via prototype coding. Regular incremental releases of the product. Official release of the product/examples/documents. [Note]  Micro process played a major role in this phase.  Regular incremental releases of the prototype provided a clean milestone in the project.

17. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199917 Testing Procedures Different levels of testing: unit testing of classes by developers  required for each class - exercising it thoroughly  testing single physics model against data system integration testing  tests developed together by developers & test team  continuously refined as components are added  run regularly (see following slide) system testing  continuous comparisons of “full setups” with selected data  under discussion

18. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199918 Testing procedures - continued Category coordinators create tags submit them to system testing team Testing team does incremental testing, category by category  following dependency order of class-category diagram accepts or rejects tags  if they succeed or fail to pass (global) system tests So we avoid “big bang” integration of parts by regular incremental releases.  Loosely couplings of class categories prevent wide- spread influence caused of a design change in a class.

19. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199919 Macro Process: Maintenance Goal: No final goal in maintenance! Major Products: Major upgrade release Minor bug fixed release Improved examples and documents [Note] The maintenance phase is the new big challenge that Geant4 has faced this year.

20. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199920 Code Management Current status of Geant4 codes  Over a thousand classes, plus associated data. Code management and class category Class category is a unit of a code management and code release.  Careful design to avoid circular dependencies is crucial.  The dependency structure sets the release order.  The release order dictates the working group schedule. One coordinator per class category.  A coordinator has a total responsibility to manage codes. Coding rule  Few coding rules -- global guide lines Tools: CVS, remote CVS

21. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199921 Quality assurance Software reliability and metrics: Insure++ and Logiscope Coding guidelines automatic checking by scripts Code inspection within class categories Code and design inspection for categories interfaces Test to ensure that categories respect dependencies  on other categories - using our own tool

22. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199922 Collaboration management Distributed management Code release coordinator - SLAC Example w.g. coordinator - Hiroshima Document coordinator - KEK Progress review and planning General workshops: KEK/CERN 95, TRIUMF 96, SLAC 97, Niigata-JP 98 Specific local workshops: per sub-domain or group Formal reports & milestones C ERN /D RDC 94, C ERN /L CRB 95, C ERN /L CB 97 & 98 Geant4 collaboration milestones (1999)

23. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199923 GEANT4: Conclusions Worldwide software development allows us to exploit the expertise of world authorities on physics (& more) distributed resources (people) and results in a greatly improved product (“quantum-leap”) To successfully manage worldwide software development, the engineering discipline of a rigorous software process is an absolute necessity. Key parts user requirements OO Analysis and design according to recognised method a detail design before coding is vital OO design had fundamental importance in the Geant4 construction, offering clear tools e.g.  The concept of class category provided a excellent scheme of sharing the code development.

24. THE END

25. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199925 Atlas Geometry - 1

26. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199926 Atlas Geometry - 2

27. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199927 Hadronic Interaction: G3 vs. G4 Preliminary results reported in CERN/LCB report ‘97 CERN/LHCC/97-40, 10 June 1997 Distributions obtained by G3 and G4 are overlaid.  Can’t distinguish Neutron Capture Coherent elastic scattering Neutron induced fission

28. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199928 VRML: Walk Through Atlas Detector

29. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199929 DAVID: Detector Overlap Viewer Example: Overlap of a box and parameterized volumes.

30. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199930 Geant4 Collaboration Map Member country Member institute

31. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199931 The Macro Development Process OO Analysis Requirement Maintenance OO Design Evolution Establish core requirement Develop a model of the system’s desired behavior Create an architecture for the implementation Manage post-delivery evolution Evolve the implementation through successive refinement

32. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199932 The Micro Development Process Identify classes and objects Specify class and object interface and implementation Identify class and object relationships Identify class and object semantics

33. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199933 Collaboration management Management Technical Steering Board (TSB) Collaboration Board Working groups Responsible for design and implementation work in specific class categories: Event/Digi/Hits - CERN, HiroshimaGeometry - CERN Tracks+Tracking - KEK, KyotoPersistency - SLAC, KEK Hadronics - TRIUMF, Aachen E.Mag. - TRIUMF, LAPP FastMC - PNHE, SLAC GUI/Visualisation - Naruto, Manchester QA/Tools - CERN, KEK, SLAC, TRIUMF

34. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199934 Collaboration management - continued Other distributed management Code release coordinator - SLAC System Testing Team coordinator - Manchester Example codes working group coordinator - Hiroshima Document coordinator - KEK Progress review and planning General workshops: KEK/CERN 95, TRIUMF 96, SLAC 97, Niigata-JP 98 Specific local workshops: per sub-domain or group Formal reports CERN/DRDC 94, CERN/LCRB 95, CERN/LCB 97

35. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199935 Software tools Basic principle No special tools for the distributed development.  Employ commonly available ones. List of tools RationalRose/C++, C++/g++ RW-Tools.h++, STL, LHC++ OODBMS (CERN/ RD45) CVS, GNUmake gdb, prof, gprof Fukui Graphic System, OpenGL/OpenInventor,OX, VRML Purify, Logiscope

36. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199936 Object Diagram - Tracking

37. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199937 Scenario Diagram - Tracking

38. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199938 Class Category Diagram Run Geant4 Readout Event Event Gen. Digi Hit Magnetic Fields Magnetic Fields Tracking Detector Physics Track Material Particle Definition Particle Definition Visualization ODBMS Utilities Geometry CAD Interface CAD Interface

39. 3rd LHC Workshop, Marseilles J.Apostolakis 29th September 199939 Infrastructure Network Without INTERNET, we couldn’t make it - it’s obvious! Bandwidth has been improved much better since we started the project - though we need more. Tools for everyday life telnet, ftp, WWW, e-mail Video conference  CODEC for most cases Telephone and teleconference Face-to-face Meeting Still most efficient and critical for concentrated works