1. Argonne National Laboratory ATLAS Core Database Software U.S. ATLAS Collaboration Meeting New York 22 July 1999 David Malon malon@anl.gov

2. Argonne National Laboratory A Brief Survey of ATLAS Core Software: Data Storage and Access l Official position is to use technologies supplied by LHC-wide common projects (RD45): Objectivity/DB commercial database l 1 TB Objectivity database was built in 12/98 with digitizations of selected parts of simulated events, each event replicated 10 times--a write-once, read-never database (some useful lessons were learned, though) R.D. Schaffer proposes to redo the terabyte milestone l ATLAS has not yet tried to employ an Objectivity database for useful physics but see Pilot Project report l Review committee has recommended a strategy of separating transient and persistent data representations--ATLAS has not yet tried this but see Pilot Project report

3. Argonne National Laboratory The Problem of Content l With limited personnel available, people who want to provide database capabilities quickly get sucked into what are not purely database issues. l Example: providing access to detector description data leads to issues like –articulation of an ATLAS-wide model for detector description –questions of detector description language, common representation, exchange format –content: Someone actually has to provide the description –(only then:) representation in a database l Providing an event store leads similarly to related issues of architecture –more so in the “low impedance mismatch” view of use of object databases--the early ATLAS thinking

4. Argonne National Laboratory A Brief Survey of ATLAS Core Software: Event Definition l Events are fundamental: What is their role in the global architecture (e.g., are they the units of data into and out of reconstruction software components)? l What is the shape of an event as seen, for example, by a reconstruction program? Everyone will care how this question is answered. l How is this view of an event related to how event data are stored (crucial to scalability and performance)? None of these questions has yet been answered. Some should be answered by the Architecture Taskforce. Portions of a raw event definition were prototyped for the 1 TB milestone. Effort is ongoing (involves Simona Rolli (U.S.)).

5. Argonne National Laboratory A Brief Survey of ATLAS Core Software: Detector Description l Detector descriptions are the province of the detector communities, but –the language and architecture for these definitions are shared –tools to convert from one application view to another (GEANT4 simulation to reconstruction to visualization to …) are common to all of them l Work to date: –pattern-oriented generic model for detector description –pieces have been implemented (Identifier, Range) –partially tested in subdetector geometry definition –no application mapping strategies have yet been tested

6. Argonne National Laboratory A Brief Survey of ATLAS Core Software: Distributed Data l Procedures and tools for distributed database development have not yet been defined –database work to date has been done essentially only at CERN –US has begun to work with CERN database domain architect on these issues l Wide-area distributed access to data for physics analysis has not yet been addressed –official policy for data distribution is to use the LHC-wide common solutions (RD45: wide-area distributed Objectivity federations) –MONARC (with US involvement) is undertaking related work

7. Argonne National Laboratory Database Near-term Priorities l Domain leadership and organization l Development infrastructure l Marshalling of subsystem database resources –almost all content-related database work should be done in this context l Control/database interface l Transient/persistent mappings l Strategy for reaching a decision about datastores l Other strategic issues (e.g., risk management) l Connection to ATLAS global computing plans l (interim(?) infrastructure for distributed data access) l (US: connection to HENP-wide--and broader CS--scalable distributed data handling, and scalability more generally)

8. Argonne National Laboratory Domain Leadership and Organization l Task definition and prioritization (UNDERWAY) l Meeting during next Software Week; workshop in October l Organization with respect to subsystem database responsibilities l Position with respect to RD45 (CERN project to address LHC- wide data handling needs) l Database platform and mass store requirements l Database strategic planning, e.g., –What is our strategy for reaching a decision about datastores (2001)? –What is our implementation strategy with respect to Objectivity? –What is our strategy and timeframe for exploring alternatives storage solutions? –What is our strategy for risk assessment, and for risk minimization?

9. Argonne National Laboratory Infrastructure for Database Development l Database coding rules and endorsed practices (some work underway) l Strategy with respect to products like HepODBMS –as an insulation strategy –as a value-added layer l Most of the infrastructure needed for distributed development is not there; sample issues: –SRT (the ATLAS configuration and build management tool) cannot at present provide some features needed to support database development –some ATLAS proposed strategies for distributed Objectivity development have been rejected by BaBar

10. Argonne National Laboratory Marshalling Subsystem Database Resources Almost all content-related database work should be done in this context. l Subsystem database representatives MUST be involved in the specification of the content of physics objects--transient and persistent--and provide the content for geometry, calibration, alignment databases, and so on. l They must also understand and at least assent to the architecture for access to that content. –Individual subsystems are also likely to be the testbeds for such architectural strategies.

11. Argonne National Laboratory Control/Database Interface Sample Issues: l Is there on-demand access to persistent data? If so, how is this coupled to control? l Must all communication with the datastore pass through control? l Maintenance of database and transaction context as control flows from component to component (a physics code may not need even to be aware that there IS a database, but SOMETHING must handle these issues) We must enable the flow of data into and out of components that use the control framework; this capability must be provided in the same timeframe as any release of the control framework.

12. Argonne National Laboratory Transient/Persistent Mappings l Develop a plan for choosing among them (or for supporting more than one?), consonant with Architecture taskforce recommendations and control framework design. l Survey of data architectures –Current and near-term experiments like BaBar, RHIC, Fermilab Run II –Other LHC experiments –non-HEP approaches to large-scale data handling and distributed data access l Testbeds for mapping strategies –One of the express purposes for tilecal pilot project –Event delivery for alpha release of control framework will be another l Architecture to support alternative datastores

13. Argonne National Laboratory Connection to ATLAS Global Computing Plans l Database domain should be providing ATLAS input to MONARC NOW. l Must also help formulate ATLAS global computing plans with respect to data for next Computing Technical Proposal (12/99 had been proposed due date) l Need (interim?) infrastructure for wide are distributed data access –If a university wants to use the tilecal testbeam database, should they copy it? (How is consistency maintained?) Use an autonomous partition? Access it directly over the net? –How does such experience inform ATLAS long-term global computing plans?

14. Argonne National Laboratory Connection to Broader Scalable Distributed Data Handling Initiatives l ATLAS members have coauthored a (successful) HENP-wide Next Generation Internet proposal (Particle Physics Data Grid) –kickoff meeting tentatively 6 August at Caltech l U.S. ATLAS institutions are also strongly connected to broader computational science initiatives in large-scale distributed data handling It would be an opportunity squandered if the U.S. did not invest some effort to ensure that this work is relevant to and integrable into ATLAS database efforts.