CMS Software Architecture Software framework, services and persistency in high level trigger, reconstruction and analysis Vincenzo Innocente CERN/EP

CMS (offline) Software Quasi-online Reconstruction Environmental data Slow Control Online Monitoring store Request part of event Store rec-Obj Request part of event Event Filter Objectivity Formatter Request part of event store Persistent Object Store Manager Object Database Management System Store rec-Obj and calibrations store Request part of event Data Quality Calibrations Group Analysis Simulation G3 and or G4 User Analysis on demand Software Architecture Vincenzo Innocente, CERN/EP

Requirements (from the CTP, dec.`96) Multiple Environments: Various software modules must be able to run in a variety of environments from level 3 triggering, to individual analysis Migration between environments: Physics modules should move easily from one environment to another (from individual analysis to level 3 triggering) Migration to new technologies: Should not affect physics software module Software Architecture Vincenzo Innocente, CERN/EP

Requirements (from the CTP) Dispersed code development: The software will be developed by organizationally and geographically dispersed groups of part-time non-professional programmers Flexibility: Not all software requirements will be fully known in advance Not only performance Also modularity, flexibility, maintainability, quality assurance and documentation. Software Architecture Vincenzo Innocente, CERN/EP

Use Cases Simulated Hits Formatting Digitization of Piled-up Events Test-Beam DAQ & Analysis L1 Trigger Simulation Track Reconstruction Calorimeter Reconstruction Global Reconstruction Physics Analysis Software Architecture Vincenzo Innocente, CERN/EP

Subject of T.Todorov & A.Vitelli talks Track Reconstruction Local measurements belongs to detector element and are affected by the detector element state (calibrations, alignments) Pattern recognition navigates in the detector to associate local measurements into a track Subject of T.Todorov & A.Vitelli talks Software Architecture Vincenzo Innocente, CERN/EP

Global Reconstruction Global reconstruction is performed in an absolute reference frame 4-vector-like objects are built out of trajectories and localized energy deposits A wide range of particle identification, jet, vertex, etc algorithms can be applied to produce others 4-vector-like objects Access to the original detector data maybe required Software Architecture Vincenzo Innocente, CERN/EP

Reconstruction Scenario Reproduce Detector Status at the moment of the interaction: front-end electronics signals (digis) calibrations alignments Perform local reconstruction as a continuation of the front-end data reduction until objects detachable from the detectors are obtained Use these objects to perform global reconstruction and physics analysis of the Event Store & Retrieve results of computing intensive processes Software Architecture Vincenzo Innocente, CERN/EP

Components Reconstruction Algorithms Event Objects Physics Analysis modules Other services (detector objects, environmental data, parameters, etc) Legacy not-OO data (GEANT3) The instances of these components require to be properly orchestrated to produce the results as specified by the user Software Architecture Vincenzo Innocente, CERN/EP

CARF CMS Analysis & Reconstruction Framework Application Framework Physics modules Reconstruction Algorithms Event Filter Physics Analysis Data Monitoring Calibration Objects Visualization Objects Event Objects The task of a framework should be that of a modern manager which - delegate responsibilities - keep tight control of the process It defines how things should be done, not what has to be done… The basic principles in the design of the framework are: - framework should not change if an application class is added or modified - persistency should be accounted for from the real begin: Transparent to the end user . - dependencies between concrete classes directly drive the flow of control: No central steering - No global actions no external synchronization The framework is a collection of loosely coupled mechanisms (specific implementations of some design patterns which implement in an abstract way the typical tasks of a HEP reconstruction and analysis software Tasks to be handled by the CMS framework are 1) management of the event objects 2) event filtering 3) definition of the detector set-up 4) access to the raw data 5) access to the asynchronous data (calibrations and alignment) 6) Reconstruction Policy Utility Toolkit LHC++ ODBMS Geant4 CLHEP PAW Successor C++ standard library Extension toolkit Software Architecture Vincenzo Innocente, CERN/EP

Architecture structure An application framework CARF (CMS Analysis & Reconstruction Framework), customisable for each of the computing environments Physics software modules with clearly defined interfaces that can be plugged into the framework A service and utility Toolkit that can be used by any of the physics modules Nothing terribly new, but... Traditional architecture can not cope with LHC-collaboration complexity Software Architecture Vincenzo Innocente, CERN/EP

Problems with traditional architectures Traditional Framework schedules a-priori the sequence of operations required to bring a given task to completion Major management problems are produced by changes in the dependencies among the various operations Example 1: Reconstruction of track type T1 requires only tracker hits Reconstruction of track type T2 use calorimetric clusters as seeds If a user switches from T1 to T2 the framework should determine that calorimeter reconstruction should run first now Example2: The global initialization sequence should be changed because, for one detector, some condition changes often than foreseen Software Architecture Vincenzo Innocente, CERN/EP

Framework Basic Dynamics Avoid monolithic structure Collection of loosely coupled mechanisms which implements in abstract the tasks of a HEP reconstruction and analysis software. Implicit Invocation Architecture No central ordering of actions, no explicit control of data flow: only implicit dependencies External dependencies managed through an Event Driven Notification to subscribers Internal dependencies through an Action on Demand mechanism Software Architecture Vincenzo Innocente, CERN/EP

Event Driven Notification Observers are instantiated by static factories residing in shared libraries. These are loaded on demand during application configuration Dispatcher Detector elements observe physics events Factories observe user requests Obs1 Obs2 Obs3 Obs4 Observers clients or providers Software Architecture Vincenzo Innocente, CERN/EP

Action on Demand Compare the results of two different track reconstruction algorithms Rec Hits Rec Hits Rec Hits Detector Element Hits Event Rec T1 T1 Analysis is a “user object” where tracks (T1 and T2) built by two different algorithms (RecT1 and RecT2) are compared. Both T1s and T2s are reconstructed out of intermediate HITS built by a common RecHits algorithm. For each new event Analysis is notified. It instantiates an iterator over tracks T1 which will ask Event for the T1s. Event will not find any T1 and will ask RecT1 to reconstruct them. RecT1 will ask Hits to Event (instantiating an iterator). Hits are not there and Event will ask RecHits to reconstruct them. RecHits return Hits to Event which hands over them to RecT1 which reconstructs T1s, gives them to Event which hands over them to Analysis. Analysis will then ask Event about T2. T2 are not there and event ask RecT2 to reconstruct them. RecT2 ask Event for Hits, which are there, and are immediately returned to it. T2 get reconstructed and given to Event will will hands over to analysis which will finally perform the comparison. CaloCl Rec T2 Analysis Rec CaloCl T2 Software Architecture Vincenzo Innocente, CERN/EP

Reconstruction Object Model All persistent objects are managed by CARF. Physics Modules access them through standard C++ pointers Software Architecture Vincenzo Innocente, CERN/EP

Framework Main Services Define the events to be dispatched and link them to the their actual source Allow the selection among available resources (user plug-in’s) Integration with the DBMS Transparent use of persistent objects in physics modules Manage the “not yet removed” sequential components (coming from legacy code & data) The combination of Implicit Invocation and Run-Time Dynamic Loading allows a CARF application to configure and build by itself Software Architecture Vincenzo Innocente, CERN/EP

Framework middle layer A given application specializes a certain set of generic CARF mechanism to provide specific services dispatch given events, loads specific libraries A middle layer implements generic clients to such specific services simplified API uniform detailed design uniform use of ancillary services shield developers and users from CARF technical implementations Requires synergy with detectors’ sub-systems Software Architecture Vincenzo Innocente, CERN/EP

CARF layered Structure Core mechanisms and “data structures” Simulation Generic Application TestBeam G3 H2 T9/X5 Raw Data Raw Data Raw Data Generic Clients Software Architecture Vincenzo Innocente, CERN/EP

CARF in Production CMS Analysis&Reconstruction Framework is used in the present ORCA “functional prototype” supports both detector and event reconstruction provides a fully integrated persistency services based on a commercial ODBMS (Objectivity/DB) is being validated by several applications ranging from test-beam (Daq and analysis) to high level trigger studies (digitization, reconstruction, event selection) Software Architecture Vincenzo Innocente, CERN/EP

Conclusions Traditional software architectures (main&subroutines, pipes&filters) have been found not to be adequate to CMS (multiple environments, evolving requirements, a long time-scale) An “implicit invocation” architecture is a flexible software solution which can scale with the complexity of the CMS project. ODBMS, integrated into the framework, provides a coherent management of persistent objects coupled with run-time dynamic-loading, allows to automatically configure an application The framework can effectively shield physics modules from the underlying technology without penalizing performances Software Architecture Vincenzo Innocente, CERN/EP