1. CMS Software Architecture
Software framework, services and persistency
in high level trigger, reconstruction and analysis
Vincenzo Innocente
CERN/EP

2. CMS (offline) Software
Quasi-online
Reconstruction
Environmental data
Slow Control
Online Monitoring
store
Request part
of event
Store rec-Obj
Request part
of event
Event Filter
Objectivity Formatter
Request part
of event
store
Persistent Object Store Manager
Object Database Management System
Store rec-Obj
and calibrations
store
Request part of event
Data Quality
Calibrations
Group Analysis
Simulation
G3 and or G4
User Analysis
on demand
Software Architecture
Vincenzo Innocente, CERN/EP

3. Requirements (from the CTP, dec.`96)
Multiple Environments:
Various software modules must be able to run in a variety of environments from level 3 triggering, to individual analysis
Migration between environments:
Physics modules should move easily from one environment to another (from individual analysis to level 3 triggering)
Migration to new technologies:
Should not affect physics software module
Software Architecture
Vincenzo Innocente, CERN/EP

4. Requirements (from the CTP)
Dispersed code development:
The software will be developed by organizationally and geographically dispersed groups of part-time non-professional programmers
Flexibility:
Not all software requirements will be fully known in advance
Not only performance
Also modularity, flexibility, maintainability, quality assurance and documentation.
Software Architecture
Vincenzo Innocente, CERN/EP

5. Use Cases Simulated Hits Formatting Digitization of Piled-up Events
Test-Beam DAQ & Analysis
L1 Trigger Simulation
Track Reconstruction
Calorimeter Reconstruction
Global Reconstruction
Physics Analysis
Software Architecture
Vincenzo Innocente, CERN/EP

6. Subject of T.Todorov & A.Vitelli talks
Track Reconstruction
Local measurements
belongs to detector element
and are affected by the detector
element state (calibrations, alignments)
Pattern recognition navigates
in the detector to associate
local measurements into a track
Subject of T.Todorov & A.Vitelli talks
Software Architecture
Vincenzo Innocente, CERN/EP

7. Global Reconstruction
Global reconstruction is performed in an absolute reference frame
4-vector-like objects are built out of trajectories and localized energy deposits
A wide range of particle identification, jet, vertex, etc algorithms can be applied to produce others 4-vector-like objects
Access to the original detector data maybe required
Software Architecture
Vincenzo Innocente, CERN/EP

8. Reconstruction Scenario
Reproduce Detector Status at the moment of the interaction:
front-end electronics signals (digis)
calibrations
alignments
Perform local reconstruction as a continuation of the front-end data reduction until objects detachable from the detectors are obtained
Use these objects to perform global reconstruction and physics analysis of the Event
Store & Retrieve results of computing intensive processes
Software Architecture
Vincenzo Innocente, CERN/EP

9. Components Reconstruction Algorithms Event Objects
Physics Analysis modules
Other services (detector objects, environmental data, parameters, etc)
Legacy not-OO data (GEANT3)
The instances of these components require to be properly orchestrated to produce the results as specified by the user
Software Architecture
Vincenzo Innocente, CERN/EP

10. CARF CMS Analysis & Reconstruction Framework
Application Framework
Physics modules
Reconstruction
Algorithms
Event Filter
Physics
Analysis
Data
Monitoring
Calibration
Objects
Visualization
Objects
Event Objects
The task of a framework should be that of a modern manager which
- delegate responsibilities
- keep tight control of the process
It defines how things should be done, not what has to be done…
The basic principles in the design of the framework are:
- framework should not change if an application class is added or modified
- persistency should be accounted for from the real begin: Transparent to the end user .
- dependencies between concrete classes directly drive the flow of control: No central steering
- No global actions no external synchronization
The framework is a collection of loosely coupled mechanisms (specific implementations of some design patterns which implement in an abstract
way the typical tasks of a HEP reconstruction and analysis software
Tasks to be handled by the CMS framework are
1) management of the event objects 2) event filtering 3) definition of the detector set-up 4) access to the raw data 5) access to the asynchronous data (calibrations and alignment) 6) Reconstruction Policy
Utility Toolkit
LHC++
ODBMS
Geant4
CLHEP
PAW
Successor
C++
standard library
Extension toolkit
Software Architecture
Vincenzo Innocente, CERN/EP

11. Architecture structure
An application framework CARF (CMS Analysis & Reconstruction Framework),
customisable for each of the computing environments
Physics software modules
with clearly defined interfaces that can be plugged into the framework
A service and utility Toolkit
that can be used by any of the physics modules
Nothing terribly new, but...
Traditional architecture can not cope with
LHC-collaboration complexity
Software Architecture
Vincenzo Innocente, CERN/EP

12. Problems with traditional architectures
Traditional Framework schedules a-priori the sequence of operations required to bring a given task to completion
Major management problems are produced by changes in the dependencies among the various operations
Example 1:
Reconstruction of track type T1 requires only tracker hits
Reconstruction of track type T2 use calorimetric clusters as seeds
If a user switches from T1 to T2 the framework should determine that calorimeter reconstruction should run first now
Example2:
The global initialization sequence should be changed because, for one detector, some condition changes often than foreseen
Software Architecture
Vincenzo Innocente, CERN/EP

13. Framework Basic Dynamics
Avoid monolithic structure
Collection of loosely coupled mechanisms which implements
in abstract the tasks of a HEP reconstruction and analysis software.
Implicit Invocation Architecture
No central ordering of actions, no explicit control of data flow: only implicit dependencies
External dependencies managed through an Event Driven Notification to subscribers
Internal dependencies through an Action on Demand mechanism
Software Architecture
Vincenzo Innocente, CERN/EP

14. Event Driven Notification
Observers are instantiated
by static factories residing
in shared libraries. These are
loaded on demand during
application configuration
Dispatcher
Detector elements
observe physics events
Factories
observe user requests
Obs1
Obs2
Obs3
Obs4
Observers
clients or providers
Software Architecture
Vincenzo Innocente, CERN/EP

15. Action on Demand
Compare the results of two different track reconstruction algorithms
Rec Hits
Rec Hits
Rec Hits
Detector
Element
Hits
Event
Rec T1 T1
Analysis is a “user object” where tracks (T1 and T2) built by two different algorithms (RecT1 and RecT2) are compared.
Both T1s and T2s are reconstructed out of intermediate HITS built by a common RecHits algorithm.
For each new event Analysis is notified.
It instantiates an iterator over tracks T1 which will ask Event for the T1s.
Event will not find any T1 and will ask RecT1 to reconstruct them.
RecT1 will ask Hits to Event (instantiating an iterator).
Hits are not there and Event will ask RecHits to reconstruct them.
RecHits return Hits to Event which hands over them to RecT1 which reconstructs T1s, gives them to Event which hands over them to Analysis.
Analysis will then ask Event about T2.
T2 are not there and event ask RecT2 to reconstruct them.
RecT2 ask Event for Hits, which are there, and are immediately returned to it.
T2 get reconstructed and given to Event will will hands over to analysis which will finally perform the comparison.
CaloCl
Rec T2
Analysis
Rec
CaloCl T2
Software Architecture
Vincenzo Innocente, CERN/EP

16. Reconstruction Object Model
All persistent objects are managed by CARF.
Physics Modules access them through standard C++ pointers
Software Architecture
Vincenzo Innocente, CERN/EP

17. Framework Main Services
Define the events to be dispatched and link them to the their actual source
Allow the selection among available resources (user plug-in’s)
Integration with the DBMS
Transparent use of persistent objects in physics modules
Manage the “not yet removed” sequential components (coming from legacy code & data)
The combination of Implicit Invocation and Run-Time Dynamic Loading allows a CARF application to configure and build by itself
Software Architecture
Vincenzo Innocente, CERN/EP

18. Framework middle layer
A given application specializes a certain set of generic CARF mechanism to provide specific services
dispatch given events, loads specific libraries
A middle layer implements generic clients to such specific services
simplified API
uniform detailed design
uniform use of ancillary services
shield developers and users from CARF technical implementations
Requires synergy with detectors’ sub-systems
Software Architecture
Vincenzo Innocente, CERN/EP

19. CARF layered Structure
Core mechanisms
and “data structures”
Simulation
Generic
Application
TestBeam G3 H2
T9/X5
Raw
Data
Raw
Data
Raw
Data
Generic Clients
Software Architecture
Vincenzo Innocente, CERN/EP

20. CARF in Production
CMS Analysis&Reconstruction Framework is used in the present ORCA “functional prototype”
supports both detector and event reconstruction
provides a fully integrated persistency services based on a commercial ODBMS (Objectivity/DB)
is being validated by several applications ranging from test-beam (Daq and analysis) to high level trigger studies (digitization, reconstruction, event selection)
Software Architecture
Vincenzo Innocente, CERN/EP

21. Conclusions
Traditional software architectures (main&subroutines, pipes&filters) have been found not to be adequate to CMS (multiple environments, evolving requirements, a long time-scale)
An “implicit invocation” architecture is a flexible software solution which can scale with the complexity of the CMS project.
ODBMS, integrated into the framework,
provides a coherent management of persistent objects
coupled with run-time dynamic-loading, allows to automatically configure an application
The framework can effectively shield physics modules from the underlying technology without penalizing performances
Software Architecture
Vincenzo Innocente, CERN/EP