1. EGEE is a project funded by the European Union under contract IST-2003-508833 ARDA Project Status Massimo Lamanna ARDA Project Leader NA4/HEP Cork, 19 April 2004 www.eu-egee.org

2. NA4/HEP Cork, 19 April 2004 - 2 Contents ARDA Project ARDA Mandate ARDA Activities Relations with other EGEE activities Conclusions and Outlook

3. NA4/HEP Cork, 19 April 2004 - 3 ARDA Project LHC Computing Grid (LCG) project  Direct report line to LCG PEB  One of the main NA4 HEP contributors  Main effort provided by EGEE Resources:  LCG resources: Project leader (Massimo Lamanna/CERN) 4 LCG staff (100% at CERN) matching the 4 EGEE staff 1 more staff from LCG (100% at CERN) About 4 FTEs from other sources (not always at CERN)  EGEE resources: 4 NA4 staff (2 persons started on April 1 st, 2 on May 1 st ; all 4 already active) LHC Experiments (ALICE, ATLAS, CMS, LHCb):  4 experiments interfaces (4 physicists) Represent the experiments in project definition, implementation and evaluation Identify and coordinate the experiment contributions –analysis groups in the experiments with whom the middleware people can work to specify the services and validate the implementations –“upper middleware” teams (experiment-specific MW)

4. NA4/HEP Cork, 19 April 2004 - 4 ARDA Mandate Provide a fast feedback to the EGEE MW development team (JRA1)  Avoid uncoordinated evolution of the middleware  Coherence between users expectations and final product Experiments ready to benefit from the new MW as soon as possible  Frequent snapshots of the middleware available  Expose the experiments (and the community in charge of the deployment) to the current evolution of the whole system Move forward towards new-generation real systems (analysis!)  Prototypes should be exercised with realistic workload and conditions (LHC experiments absolutely required for that!) No academic exercises or synthetic demonstrations  A lot of work (and useful software) is involved in current experiments data challenges: this will be used as a starting point Adapt/complete/refactorise the existing: we do not need another system!

5. NA4/HEP Cork, 19 April 2004 - 5 Experiment End-to-End Prototypes Every experiment has already at least one system  Analysis/Production typically distinct entities  Using a variety of back-ends (Batch systems, different grid systems) ARDA will put its effort on the experiment (sub)system the experiment chooses  EGEE MW as foundation layer Multigrid interfaces outside our current scope (more in JRA1) Experiments do know how to deal with this  By default, we expect 4 systems ARDA role  Adapt/complete/refactorise the existing (sub)system!  Collaborative effort (not a parallel development)  Commonality is not ruled out, but it should emerge and become attractive for the experiments. Anyway not imposed “from outside”  First important checkpoint = functional prototypes (LCG Level 1 milestone): Dec 2004

6. NA4/HEP Cork, 19 April 2004 - 6 Experiment End-to-End Prototypes The initial prototype will have a reduced scope  Components selection for the first prototype  Experiments components not in use for the first prototype are not ruled out (and used/selected ones might be replaced later on)  Not all use cases/operation modes will be supported Attract and involve users  Many users are absolutely required The Use Cases are still being defined  Example: A physicist selects a data sample (from current Data Challenges) With an example/template as starting point (s)he prepares a job to scan the data The job is split in sub-jobs, dispatched to the Grid, some error-recovery is automatically performed, merged back in a single output The output (histograms, ntuples) is returned together with simple information on the job-end status

7. NA4/HEP Cork, 19 April 2004 - 7 Example of current activity Metadata catalogues  Every experiment has different implementations (not only schemas)  Used mainly in production environments Few expert users Coordinated update and read actions  ARDA Interface with the EGEE middleware Verify (help to evolve to) analysis environments –Many users »Robustness –Concurrent read actions »Performances  A Common Application Layer might emerge in future ARDA emphasis is to enable each of the experiment to do its job SOON NOW

8. NA4/HEP Cork, 19 April 2004 - 8 ARDA @ LCG Regional Centres Stress and performance tests could be ideally located in some RCs  This is for experiment-specific components  Leverage on RC local know how Data base technologies Web services … Good contacts with MSU-Dubna and Taipei  Many others interested  Pilot non-EGEE sites might enlarge the resources available and give fundamental feedback in terms of “deployability” to complement the EGEE SA1 activity

9. NA4/HEP Cork, 19 April 2004 - 9 ARDA as a Forum The coordination activities would flow naturally from the fact that ARDA will be open to provide demonstration benches  Since it is neither necessary nor possible that all projects could be hosted inside the ARDA experiments’ prototypes, some coordination is needed to ensure that new technologies can be exposed to the relevant community Transparent process  ARDA should organise a set of regular meetings (one per quarter?) to discuss results, problems, new/alternative solutions and possibly agree on some coherent program of work.  The ARDA project leader organises this activity which will be truly distributed and lead by the active partners Special relation with LCG GAG  Experiments representatives coincide with the NA4 experiments representatives ARDA will channel this information to the appropriate recipients

10. NA4/HEP Cork, 19 April 2004 - 10 Relation with other EGEE activities JRA1  EGEE middleware prototypes = sets of services  We expect a series of prototypes during 2004 The first prototype is expected by May 2004 This will evolve to the Dec 04 release candidate with a series of several prototype ARDA is exposed to these systems as soon as possible (several week cycle)  Individual services will evolve in both interface and implementation Interface –Starting point: ARDA document –Evolution: ARDA feedback taken into account –ARDA understands that interfaces have still to be consolidated Implementation –Early services replaced by completely new ones or evolving the existing ones (new versions) –Evolution: ARDA feedback taken into account (bug fixes, performances issues, priority list) –ARDA does not compete on implementing services with JRA1 SA1  SA1 deploys the prototype on a parallel line as LCG2 ARDA will be one of the main pilot user Fundamental to have coherence between deployed resources and data sets

11. NA4/HEP Cork, 19 April 2004 - 11 Conclusions and Outlook ARDA is starting  Main tool: experiment prototypes for analysis  Detailed project plan being prepared  Good feedback from the LHC experiments  Good collaboration with EGEE NA4 Look forward to contribute to the success of EGEE  Helping JRA1 to deliver a fully functionally solution ARDA main focus  Collaborate with the LHC experiments to set up the end-to-end prototypes  Aggressive schedule First milestone for the end-to-end prototypes is Dec 2004