HPC resources LHC experiments are using (as seen from ATLAS - CMS lens) erhtjhtyhy Doug Benjamin Argonne National Lab

Thanks and apologies Talk is seen through ATLAS – CMS lens Thanks : based on feed back and information from Andrej Filipcic (ATLAS) Dirk Hufnagel (CMS) Alexei Klimentov (ATLAS) Daniele Spiga (CMS) Apologies to ALICE and LHCb. ALICE successful in using several HPC centers for a while NERSC and OLCF for example. LHCb – HPC talk from 2018. https://indico.cern.ch/event/651338/contributions/2871262/attachments/1594114/2523997/preGDB-HPC_at_LHCb.pdf

ATLAS HPC Sites/ PanDA queues

CMS HPC sites

ALOCATION Process

European HPC’s (non – pledged) Typically negotiations directly with HPC center Centers allocate most of their time through the local allocation process HPC’s provide a fraction of resources to PRACE for HPC scientific communities, and the approval procedure is much more complicated and stricter when a project is approved, the PRACE assigns the allocation to the HPC center which best fits the requirements of the application. For the LHC experiments this would create additional technical issues.

US DOE LCF process Source: Talk given by Katherine Riley – ALCF science director. In 2018-2109 ATLAS ALCC: 80M hrs OLCF 80M hrs ALCF CMS: startup allocation at ALCF

US DOE – NERSC allocation program Most time comes through DOE production DOE Production time – managed through DOE program managers- Total ATLAS CMS time essentially constant 2018 ATLAS: 175 M MPP hours CMS: 25M MPP hours Total: 200 M hrs. 2019 ATLAS: 120 M MPP hours CMS: 82M MPP hours Total: 202 M hrs.

National Science Foundation (NSF) XSEDE allocation program https://portal.xsede.org/allocations/research CMS’ success Q2 2019 – Q1 2020 PSC/Bridges - 4.78 M hours TACC/Stampede2 - 0.73 (node) hours SDSC/Comet - 3.3M hours

Going forward Next few months - LHC experiments should think if ALCF – ADSP program would really help us in short term. As more HPC’s with accelerators come on-line – challenge will be to get time if our needs are predominantly CPU only

Acknowledgement Policy - CSCS NERSC Acknowledgements Users must quote and acknowledge the use of CSCS resources in all publications related to their production and development projects as follows: "This work was supported by a grant from the Swiss National Supercomputing Centre (CSCS) under project ID ###"

Conclusions ATLAS and CMS both use a variety of HPC sites CMS has done a lot of work to cvmfs available at sites they use. ATLAS should make use of all CMS’ hard work Should come up with common criteria for cvmfs where possible use containers a places that can not or will not use cvmfs. We will discuss the Operational model evolution later in this meeting. Need to keep applying for yearly allocations through various programs. Need multi-year allocations. NERSC ATLAS allocation - 2018 (175 M MPP hrs) to 2109 (120 M MPP hrs) Notified just before the yearly allocation started. If HPC’s are to be come part of the pledged resources then we need have multi-year allocations. This is a discussion that should happen at the Funding Agency levels.