Michela Piergentili, INFN Genova F. P. Brooks, “No Silver Bullet - Essence and Accidents of Software Engineering”, IEEE Computer 20(4):10-19, April, 1987 As we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity.... Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any - no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware. We cannot expect ever to see twofold gains every two years.... disciplined, consistent effort Although we see no startling breakthroughs - and indeed, I believe such to be inconsistent with the nature of software - many encouraging innovations are under way. A disciplined, consistent effort to develop, propagate, and exploit these innovations should indeed yield an order-of-magnitude improvement. There is no royal road, but there is a road.

Michela Piergentili, INFN Genova Experience with Software Process in Physics Experiments S. Guatelli, B. Mascialino, L. Moneta, I. Papadopoulos, A. Pfeiffer, M.G. Pia, M. Piergentili CERN - INFN Genova Monte Carlo 2005 Topical Meeting Chattanooga, April 2005

Michela Piergentili, INFN Genova Challenges Complexity of physics and experiments Geographically distributed teams and computing resources Mission critical applications Quality and reliability for sensitive applications

Michela Piergentili, INFN Genova Trends in software engineering discipline Develop a discipline for software engineering Shift attention from computing technology to software process as the way to achieve quality, lower costs Define quantitative standards to evaluate software capability maturity: SEI’s Software Capability Maturity Model ISO The culture of software process is not widely spread in the physics research environment yet Software is still perceived as “writing code” The success of software projects is mainly left to individual efforts (CMM: “heroic”, level 1) Widely used in software industry

Michela Piergentili, INFN Genova Process framework incrementaliterative An incremental and iterative software life-cycle Based on the Unified Process (Booch, Jacobson and Rumbaugh) Rational Unified Process: UP + practical guidance and tools Customized Customized to the specific development environment Equivalent to CMM / ISO Level 3 at least

Michela Piergentili, INFN Genova Projects adopting the RUP Pilot project: Anaphe (Data Analysis Tools project, CERN/IT Division) –a playground to learn the RUP and to adapt it to the physics research environment RUP in projects Geant4 Low Energy Electromagnetic Physics Geant4 Electromagnetic Physics Validation Project Statistical Toolkit Brachytherapy Simulation & Dosimetry IMRT Geant4-based Dosimetry System REMSIM Simulation (radioprotection, Aurora Programme) Bepi Colombo Simulation (planetary astrophysics) Solar System Radiation Environment Generator RUP in an organization RUP in an organization (encompassing several projects) Geant4 Advanced Examples (in progress)

Michela Piergentili, INFN Genova Retain all the best practices suggested by the RUP –Iterative development –Management of the requirements –Component-based architecture –Visual modeling with UML –Continuously verify quality –Change management Identify essential activities, artifacts and key roles –for every RUP activity: is it justified in our environment? What are the benefits? How much does it cost? Justify all artifacts to be produced in relation to the project objectives Guidelines for process tailoring

Michela Piergentili, INFN Genova Essential artifacts Vision Risk List Development Plan Development Case CVS repository User Requirements Architecture Model Design Model Test Plan Test Suite Prototype Tools Guidelines The system Documentation + refinement of artifacts from previous phase The product Release Notes Training Material + refinement of artifacts from previous phases Other optional artifacts may be produced in some projects …much more than just the code! A minimal subset identified within the large set of RUP artifacts

Michela Piergentili, INFN Genova Metrics Quantitative process control Objective evaluations are the key to software process improvement Requirements Number or user requirements / use cases Requirement status Traceability Analysis & Design Number of subsystems, packages, classes Number of methods in a class Inheritance tree depth Implementation Code size Method size Fraction of design implemented Deployment New features implemented per release Released defect levels Documentation/examples coverage Schedule Estimated vs. actual task duration Estimated vs. actual duration between milestones Schedule estimating accuracy Process Process capability level Process deviations Number of metrics collected Test Number of unit/system tests Fraction of code covered by unit testing # of defects found by unit testing # of defects found by integration testing # of defects found by system testing Fraction of deliverables reviewed Number of defects found by reviews Defect status Defect recurrence Defect source count Project management Number of tasks planned and completed Number of reviews, walkthroughs Staff size and profile Staff turnover Support Number of support requests Support request aging Average resolution time Re-opened support requests

Michela Piergentili, INFN Genova Some metrics 95% projects delivered on time, on budget and with all the features originally planned –CHAOS Report 2000 (Standish Group): 28% software projects failed 49% challenged 23% successful 5% failure due to neglect of applying the process –no way to enforce the application of the process onto independent researchers

Michela Piergentili, INFN Genova A case study Development of simulation + analysis of a dosimetric system for IMRT –Software requirements: functionality + high quality for medical application –Master thesis work, duration: 1 year –Manpower: 1 developer + 1 process specialist –Developer: no prior software knowledge D = 0.005; p-value = 1 Results –on-time delivery of code, documentation, physics results –all high and medium priority requirements satisfied –~30 billion simulation events produced and analysed –high quality physics results from the software (paper in IEEE-NSS 2004) –355 page MSc. thesis –fraction of code discarded during the development period: 0

Michela Piergentili, INFN Genova Conclusion A rigorous software process is the key technology enabling development teams to achieve success The RUP can be effectively adapted to the peculiar physics research environment –a powerful, but flexible process framework A RUP-based software process has been adopted by several physics projects in may research fields (fundamental physics, space science, astrophysics, medical physics, statistics) Success rate is 95% (to be compared to CHAOS 23%)