Architecture and Control Framework David R. Quarrie Lawrence Berkeley National Laboratory DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National Laboratory NOVEMBER 14-17, 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 2 Overview  Progress since May  Milestones and deliverables  Architecture Review Status  Ongoing activities  December Milestone deliverables  Future Milestones  Personnel  Summary  Additional Background Material

November 14-17, 2000 David Quarrie Architecture & Control Framework 3 Some Background  Definitions  Architecture: General structure of a software system, comprising components, their properties and the relationships between them  Framework: Concrete realization of an Architecture  ATLAS Control Framework (Athena)  Spans simulation, reconstruction, event filter, physics analysis  Prototype based upon GAUDI Framework from LHCb  Parallel Unified Software Development Process (USDP)  Reverse engineer GAUDI  Forwards engineering from user requirements  Identify and work on missing functionality  Significant ATLAS-developed functionality already in place  Good collaboration with LHCb

November 14-17, 2000 David Quarrie Architecture & Control Framework 4 May 2000 Prototype Milestone  Essential - feature set must include:  Read Physics TDR data   Execute sequence of multiple user modules   Desirable - feature set should include:  Generate Event Display   Dynamic loading of user modules   Sequences with branches/filters   Optional - feature set may include:  Rudimentary interactive user interface   Limited data persistence - HBOOK only   Additional items not part of formal deliverables  Event Generator framework prototype (LBNL)   Liquid Argon reconstruction (BNL)   Xkalman++ Inner Detector Tracking (LBNL)   Two Fast Simulation efforts begun (UCL, CERN)

November 14-17, 2000 David Quarrie Architecture & Control Framework 5 Feedback from May Milestone  Identified deficiencies in access to event data  Event Data Model Workshop on 31 st May  Discuss issues and requirements  Second Event Data Model Workshop on 11 th July  Present design  Third Event Data Model Workshop on 23 rd October  Feedback from prototype and start 2 nd design iteration  Detector Description Workshop on 28 th August  Requirements for alignments, calibrations etc. from simulation & reconstruction etc.  Review deliverables for September Milestone  Review scheduled for 10 th July  Architecture Review Committee (ARC) formed  Assign tasks based on updated deliverables & feedback

November 14-17, 2000 David Quarrie Architecture & Control Framework 6 Tutorials & Workshops  Tutorials  Two in parallel at LBNL for May milestone on 9 th May (35 attendees)  At CERN on 31 st May (18 attendees)  At CERN on 6 th July (17 attendees)  Good attendance and feedback  Intention to repeat at regular intervals with updated contents  Architecture Workshops  9-11 March 2000  Plans and deliverables  April 2000  Focus on user requirements  23 rd Jan 2001  Progress to date  Identify missing functionality  Revisit deliverables & milestones  Encourage feedback from developers & users

November 14-17, 2000 David Quarrie Architecture & Control Framework 7 Architecture Review Committee  Mandate  Review the approach taken by the A-Team  Conduct a design review  Review priorities for milestones and deliverables  Comment on database milestones and deliverables  Membership  Daniel Froidevaux (chair) Marzio Nessi  Dario Barberis Pavel Nevski  Chris Bee Alan Poppleton  Laurent Chevalier Rene Brun (ALICE)  Stephen Haywood Jim Kowalkowski (D0)  Julius Hrivnac David Stickland (CMS)  Fred Luehring

November 14-17, 2000 David Quarrie Architecture & Control Framework 8 ARC Meetings  10 th July  24 th August  25 th September  6 th -7 th December  5 th -7 th March 2001  Ongoing, detailed review  First phase of design review completed  Useful design feedback  E.g. Accelerated development of a data dictionary  Goal is significant feedback from users as well as developers  Hence protracted duration  Concern about change of priorities and new deliverables as a result of ARC “real-time” feedback

November 14-17, 2000 David Quarrie Architecture & Control Framework 9 September 2000 Milestone (1)  Essential  Prototype/USDP Merged [US]   Athena in ATLAS release [US]   Event Data Model prototype [US]   Access to transient event store from Algorithm  Interactive Scripting prototype [US]   Event Generator framework deployed [US]   Fast Simulation prototype   Wrapped FORTRAN Fast Simulation prototype   Physics Analysis output to ROOT prototype [US]   Ability to output histograms & ntuples to ROOT files [US] US Deliverables Prototype  Available to developers Deployed  Available to end users in ATLAS software release

November 14-17, 2000 David Quarrie Architecture & Control Framework 10 September 2000 Milestone (2)  Essential (continued)  TileCal testbeam input from Objectivity [DB]  G3 Digits read in from Objectivity [DB]  Objectivity event I/O prototype [DB]  Demonstration of ability to read & write event data (HepMC)  ROOT event I/O prototype [DB]  Demonstration of ability to read & write event data (HepMC)  Desirable  Auditors prototype [US]   Cpu/memory monitors  Physics Analysis binding to ROOT prototype [US]   Ability to switch to ROOT & back directly within Athena  Partial ATLFAST event data I/O to Objectivity & ROOT [DB] [DB] deliverables not discussed here

November 14-17, 2000 David Quarrie Architecture & Control Framework 11 Summary of September Milestone  Most deliverables met  Typically missed by less than 2 weeks  Scripting delivered but still being integrated  Physics Analysis binding to ROOT was major missed deliverable  Athena in ATLAS release of 26 th Oct 2000  Dual Gaudi releases available  Vanilla Gaudi 5  Used for ATLAS release  “Gaudi 5.5” release incorporating our code changes  Used for ATLAS release  ATLAS developed changes and extensions in Gaudi 6  Released on 10 th November 2000  Joint LHCb/ATLAS release  To be used for ATLAS release 1.2.2

November 14-17, 2000 David Quarrie Architecture & Control Framework 12 Developments not in Formal Milestone  Improved Remote Site Support [US]  Goal is easy installation on a laptop with no network access  Solaris 7 support [US]  Not yet an officially supported ATLAS platform  Improved Sequencer for Event Filter [US]  Explicit branching and inverted filter logic  Data Dictionary [US]  Automatic generation of converters & persistent classes  Transient Store Data Browser  Multi-language support  Improved Service Management [US]  Gaudi internals (USDP feedback)  Improved documentation [US]

November 14-17, 2000 David Quarrie Architecture & Control Framework 13 Other Developments  Many ongoing developments based on Athena  Event Filter  Fast simulations  Level 2  Liquid Argon  Muon  TileCal  Xkalman++  Etc.  Active developer community  Support  Mailing list (atlas-sw-architecture)  Craig Tull (LBNL) at CERN for 12 months

November 14-17, 2000 David Quarrie Architecture & Control Framework 14 December 2000 Milestone  Version supporting the Trigger TDR  Essential  ATRECON equivalent reconstruction prototype  Reconstruction deliverable  Pile-up support prototype [US]  Event Data Model deployed [US]  ATLAS release on 10 th Nov 2000   Detector Description (Calibration/Alignment) prototype  Interactive Scripting deployed [US]  Note late delivery of prototype  Fast Simulation deployed (but not validated)  Physics deliverable

November 14-17, 2000 David Quarrie Architecture & Control Framework 15 December 2000 Milestone (2)  Desirable  Auditors deployed [US]  ATLAS release on 10 th Nov 2000   Particle Properties Service prototype [US]  In principle from CLHEP, but concerns about timescale  Physics Analysis output/binding to ROOT deployed [US]  ATLAS release on 10 th Nov 2000   Physics Analysis output/binding to JAS prototype  Objectivity event I/O partially deployed  Solaris 7 support? [US]  Driven by ATLAS Computing schedule

November 14-17, 2000 David Quarrie Architecture & Control Framework 16 April 2001 Milestone  Essential  Geant4 integration prototype  Objectivity Event I/O fully deployed  Desirable  Particle Properties Service deployed [US]  Detector Description partially deployed  Physics Analysis output/binding to JAS deployed  Visualization prototype  Algorithm based already exists (May 2000 prototype)  Full interactive service  Statistics and Monitoring prototype [US]  Bookkeeping prototype [US]  Java/C++ support? [US]

November 14-17, 2000 David Quarrie Architecture & Control Framework 17 September 2001 Milestone  Ready for Mock Data Challenge 0  Continuity test  Essential  Geant4 integration deployed (but not validated)  Detector Description deployed  Visualization deployed  Statistics and Monitoring Tools deployed [US]  Desirable  Bookkeeping deployed [US]  Full multi-language support (services & algorithms in Java)? [US]

November 14-17, 2000 David Quarrie Architecture & Control Framework 18 December 2001 Milestone  First “production” version  Ready for Mock Data Challenge 1  Essential  Bookkeeping deployed [US]  Desirable  GRID enabled prototype [US]

November 14-17, 2000 David Quarrie Architecture & Control Framework 19 Longer Term Milestones  Driven by overall Computing Schedule  Continue model of Athena releases every 3-4 months  Mock Data Challenge I  Production starts Dec 2001  Computing TDR  Writing starts Jun 2002  Mock Data Challenge II  Production starts Nov 2002  Physics Readiness Document  Writing starts Feb 2004  Start commissioning  Jul 2005

November 14-17, 2000 David Quarrie Architecture & Control Framework 20 Software Engineers (LBNL) I  Paolo Calafiura (50%)  Framework (general) (WBS ) 25%  EDM (WBS ) 25%  Chris Day  USDP (WBS ) 50%  Framework (scripting) (WBS ) 50%  Charles Leggett (80%)  Framework (ROOT I/O) (WBS /11) 80%  Massimo Marino  Framework (re-engineering) (WBS ) 75%  EDM (WBS ) 25%

November 14-17, 2000 David Quarrie Architecture & Control Framework 21 Software Engineers (LBNL) II  David Quarrie (80%)  Architecture (WBS ) 50%  Framework (general) (WBS ) 30%  Craig Tull  Framework (scripting, data dictionary) (WBS ) 90%  Framework (GRID) (WBS ) 10%  FTEs  Framework (3.6), EDM (0.5), USDP (0.5), Architecture (0.5)  FY01 Funding  US-ATLAS 4.3 FTE  LBNL 0.8 FTE  Shortfall of 0.5 FTE in FY02

November 14-17, 2000 David Quarrie Architecture & Control Framework 22 Summary  Met most of the September Milestone Deliverables  Typically missed by <2 weeks  All apart from Physics Analysis to ROOT binding delivered by mid- November  Intense Architecture Review underway  Very active developer community  Trying to encourage user involvement  Trying to maintain the momentum since May  Athena has demonstrated support for simulation, event filter, reconstruction, physics analysis  Much work remains to convert prototypes to production and increase the scope

November 14-17, 2000 David Quarrie Architecture & Control Framework 23 Background Material  The following slides provide additional background material

November 14-17, 2000 David Quarrie Architecture & Control Framework 24 Prototype/USDP Merged [US]  A-Team deliverable - Chris Day and Massimo Marino (LBNL)  My definition of deliverable  Reverse engineer GAUDI  Done for Gaudi 5  Forwards engineering from user requirements  All available user input entered and procedures setup to accept more  Creation of formal requirements from user input still incomplete  Identify and work on missing functionality  Done for scripting and sequencing (filtering & branching)  Starting to re-engineer some of core design  Rational product suite & web page   ARC considers this deliverable incomplete  Nov 2000?

November 14-17, 2000 David Quarrie Architecture & Control Framework 25 Athena in an ATLAS Release [US]  A-Team deliverable – Paolo Calafiura & David Quarrie (LBNL)  Everything in place  Gaudi 5 available as external package  Developers Gaudi installation installed in public (project) area  Only used by beta-testers of prototypes  GaudiInterface SRT interface package  Several Athena examples & common executable  CVS tags are available  Includes dynamic loading functionality  Minimal common executable  Run-time dynamic loading on demand  In ATLAS software release of 26 th Oct 2000  Milestone met on 4 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 26 Event Data Model Prototype [US]  Paolo Calafiura (LBNL), Srini Rajagopalan (BNL) et al.  Framework level support for EDM prototype  Access to data from Algorithms  Base classes, smart pointers/handles  Keys & selectors, etc.  StoreGate SRT package available  Design documentation available  One change to Gaudi/Athena base code required  Generalized service accessor for Algorithm  Incorporated into a developers Gaudi/Athena installation  Milestone met on 4 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 27 Interactive Scripting Prototype [US]  Chris Day & Craig Tull (LBNL)  Requirements based on USDP feedback  Prototype functional since 24 th October  Based on PYTHON  Best choice  Abstractions support alternative scripting languages  Single implementation planned  Significant integration issues still being tackled  Extensions to base classes  Integration completed by mid November 2000  Available in updated developers release

November 14-17, 2000 David Quarrie Architecture & Control Framework 28 Event Generator Framework Deployed [US]  Ian Hinchliffe & Marge Shapiro (LBNL)  Several generators available  Isajet, Pythia, etc.  Herwig underway - ready by end of October  Generator to use selectable at run-time  Properties of generator modifiable at run-time  SRT packages and tags in place  Dependent on Athena in an ATLAS release deliverable  In ATLAS Release of 26 th Oct 2000  Milestone met on 4 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 29 Fast Simulation Prototype  UCL (Peter Clarke et al.) deliverable  SRT Packages all available  Documentation available  Generation of histograms demonstrated  Compatibility with Generator framework demonstrated  Physics validation started  Testing against EDM (StoreGate) prototype started  2 nd generation design following experience and review  Milestone met on 4 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 30 Wrapped FORTRAN Fast Simulation Prototype  Elzbieta Richter-Was (CERN)  Two variants delivered  Wrapped FORTRAN  ATLFAST++ (C++)  Primary motivation is validation of new OO design  SRT Packages available and tagged  Timing studies available  Documentation available  Histogramming & Ntuples demonstrated  Milestone met on 15 th August 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 31 Physics Analysis Output to ROOT Prototype [US]  Charles Leggett (LBNL)  Output of histograms and n-tuples to ROOT files  May prototype only supported HBOOK files  Prototype design and implementation complete  Extension to Gaudi base code  Changes incorporated into Gaudi CVS repository and developer installation  Ntuple API limitations overcome  Mapping of variables into blocks/trees  Available 19 th October  Milestone met on 13 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 32 Auditors Prototype [US]  Marge Shapiro & David Quarrie (LBNL)  Algorithm monitoring (before/after execution etc.)  CPU usage and memory usage etc.  All code incorporated into Gaudi CVS repository and into developer Gaudi installation  No SRT tags since Gaudi base functionality  Three concrete auditors delivered  Print Name  CPU Time  Memory growth  Draft documentation fed back to Gaudi User Guide  Milestone met on 4 th October 2000

November 14-17, 2000 David Quarrie Architecture & Control Framework 33 Physics Analysis Binding to ROOT Prototype [US]  Charles Leggett (LBNL)  Ability to switch at run-time between Athena/Gaudi & ROOT and view histograms & n-tuples  Not yet available  Physics Analysis Output to ROOT Prototype took longer than expected  Milestone not yet met  Desirable milestone  Readjustment of priorities  Raise ATLFAST Event I/O to ROOT above this

November 14-17, 2000 David Quarrie Architecture & Control Framework 34 Improved External Site Support [US]  Iwona Sakrejda (LBNL) working on developing procedures  Fred Luehring (Indiana) acting as guinea pig  Site-specific configurations being fed back into Gaudi CVS repository  The ATLAS GaudiInterface package has been updated to support site-specific locations  Goal is to demonstrate stand-alone (i.e. network independent) laptop installation  Currently feasible, but requires too much manual labor  E.g. Modifications to Gaudi & ATLAS packages

November 14-17, 2000 David Quarrie Architecture & Control Framework 35 Solaris 7 Port [US]  Early look at Gaudi/Athena and Solaris CC 6.0  Not yet clear when ATLAS will require this as officially supported platform  End of this year?  Essential Kernel packages compile (with some problems) but don’t yet link  Changes being fed back into the CVS repository  Very low priority since we don’t have a schedule for ATLAS  Good exercise since it’s pointed out a few inconsistencies in the core code base

November 14-17, 2000 David Quarrie Architecture & Control Framework 36 Sequencer [US]  Enhanced sequencer supports explicit branching as well as filtering Sequencer.Members = { “First”, “Second”, “Third” }; Sequencer.BranchMembers = { “Second”, “Forth” };  Algorithm “Third” will be executed if event passes the “Second” filter, otherwise Algorithm “Forth” will be executed.  Inverted logic Sequencer1.Members = { “First”, “Second”, “Third” }; Sequencer2.Members = { “First”, “Second:invert”, “Forth”};  Goal is better support for Event Filter  Requirements based on USDP feedback

November 14-17, 2000 David Quarrie Architecture & Control Framework 37 Data Dictionary [US]  Automatic generation of code for:  Generation of persistent classes and converters  Browsing of transient event store  Multiple language support  Relevant experience within ATLAS  Priority raised as a result of feedback from ARC  LHCb also interested in this topic  Previous experience within US-ATLAS  Paolo Calafiura (CDF), David Quarrie (BaBar), Craig Tull (STAR)  Working group being formed  Annecy group interested in this area

November 14-17, 2000 David Quarrie Architecture & Control Framework 38 Gaudi Service Management [US]  Feedback from USDP  Reverse engineering of Gaudi 5  Gaudi service management and event loop are too tightly coupled  ApplicationMgr handles both  Services are instantiated even if they aren’t needed  Not easy to override default implementations  Massimo Marino (LBNL) reworking this area  Did reverse engineering last FY while funded by LBNL  Also looking into reducing couplings