Bologna Workshop Summary and Latest Software Progress at CEPC Weidong Li Computing Center, IHEP 2019-08-16

Motivations of the workshop A common software framework would have to be versatile as to encompass the different needs of the various experiments It should take into account the use of new processors (parallel processing, GPUs, FPGAs, etc.) It should allow to “simply” define different detector geometries Could we have a common event data model? We are here to discuss these things and more and possibly come out of the workshop with a working plan on how to develop together a common solution

Workshop Agenda https://agenda.infn.it/event/19047/ Introduction and motivation (Paolo Giacomelli, INFN Bologna) Software status of the LHC experiments (Tommaso Boccalli, CERN) ILC software (Frank Gaede, DESY) FCC software (Gerardo Ganis, CERN) CEPC software framework (Xingtao Huang, SDU) Common software tools (Graeme Stewart, CERN) Turnkey software stack vision (Andre Sailer, CERN) Round table discussions General discussion on a common software framework Towards a common software stack

FCC Software

Detector Description (FCC)

Event Model: PodIO (FCC)

CEPC SW Framework Needs Huge Data Volume O(~EB) for CEPC running at Z pole Management of Event Data and even non-Event Data Parallelization Levels: algorithm, intra-event and inter-event Technologies: OpenCL, CUDA, TBB, MPI… Re-use of existing and successful serial algorithms Heterogeneous Architectures CPU,GPU, FPGA, HPC, Cloud… Portable and flexible Interfaces to novel tools and software Application of Machine learning, Deep Learning and Big Data into HEP Friendly user interfaces Hiding new techniques from physicists Support flexible analysis with/without framework , interactive web analysis 2019-06-12

CEPC Software (1) Event Data Model LCIO defines a common Event Data Model for Collider Experiments, including Monte Carlo Data, Raw Data Digitization, Reconstruction and Analysis Data Relations between different data objects Will be adopted for CEPC with ROOT extension Combine LCIO with ROOT Put LCIO object inheriting from TObject into Data Store Write Data Store objects into ROOT files Convert Data Store objects into other format (such as NumPy for machine learning) 2019-06-12

CEPC Software (2) Detector Simulation Integration with generators (for example Whizard and Pythia) Adopt DD4hep for detector geometry description Integrate DD4hep with framework Support running different detector designs and comparison among them. Mixing of fast Simulation and full Simulation 2019-06-12

CLiC Software Future plan

iLCSoft Current Future Plan

Common Software Stack (CSS)

Workshop Outcome (1) Development of a Common Turnkey Software Stack (Key4HEP), contains: HEP standard and new libraries / packages, such as ROOT, Geant4, HepMC, VecCore, VecMath, VecGeom, ... Externals, such as Boost, GSL, Eigen, ... EDM and Geo libraries, such as DD4Hep, PODIO Rec/Tracking libraries, such as ACTS, PandoraPFA, ...   One   framework MARLIN framework will continue to be used by the LC community while the Key4HEP is commissioned.

Workshop Outcome (2) Have a simple management scheme (Facilitators) for collaborating on a common software stack F. Gaede (ILC, DESY) W. Li (CEPC, IHEP) X. Huang (SCTF, Shandong) G. Ganis (FCC, CERN) A. Sailer (CLIC, CERN) D. Maximov (SCTF, Budker) D. Lange, X. Lou (USA) G. Stewart (HSF, CERN) T. Boccali (LHC,INFN)

Progress since Bologna Workshop

International Collaboration Expression of Interest (EoI) for participating in the H2020 Innovation Pilot on detector technologies at accelerators: Turnkey software stack for detector studies Definition and adoption of the common event data model (EDM4HEP) Adaptation of geometry package (DD4HEP) Selection of the data processing framework with upgraded set of features Packaging, building, deploying, and continuous integration Compete the integration of all the components Automation of the procedures to ease maintenance and evolvement Participants and contacts Participating institute Main contact person E-mail CERN Andre Sailer andre.philippe.sailer@cern.ch DESY Frank Gaede frank.gaede@desy.de IHEP Weidong Li liwd@ihep.ac.cn INFN Tommaso Boccali Tommaso.Boccali@cern.ch

CEPC SW Prototype: CEPCSW The goal is the development of a software prototype to integrate existing software components for event generation, simulation, reconstruction and data analysis. to demonstrate the capabilities to meet future requirements from detector design and optimization, testbeam experiments, and various studies with M.C. data to understand both detector’s limitation and physics sensitivities. to be able to follow the main trends of computing development of HEP experiments e.g. support of multithreading, GPU, FPGA, HPC etc. to support continuous integrations of new software components, leading towards the final production system. Major components of the prototype Underlying framework: a software skeleton with well defined interfaces for various types of software components Event Data Model (EDM): management of data related to a physics event Detector description: management of detector geometry and material information Infrastructure: tools for version control, compiling, testing and deployment etc.

Underlying Framework For the latest releases, a lot has changed (Graeme A Stewart, CERN) External libraries: the minimal set is only 9 (and only ROOT is HEP), the rest are optional. Whiteboard: the requirement of objects in the store to inherit from Data Object has been gone Converter Persistency Service Data Files Transient Event Store Application Manager PODIO Data Files Message Service Event Data Service JobOptions Service Algorithm Algorithm Algorithm DD4HEP Data Files Data Files Persistency Service Transient Detector Store Particle Prop. Service Detec. Data Service Other Services Data Files Transient Histogram Store Persistency Service Histogram Service The core part of the framework is small, key components are: Application Manager, Algorithms, Tools, Services The version of v29r2 was used in CEPCSW (latest version v32r1)

EDM PODIO developed in the context of FCC studies creates all C++ and Python code based on a description of the EDM structures in a YAML file. the POD layer: holds the arrays of the actual data structures the object layer: consists of transient objects the user layer: introduces handles to the EDM objects and collections of object handles relations between objects: every object is uniquely identified by its ObjectID and object pointers are converted into indices and are saved into the ROOT file. pLCIO https://stash.desy.de/projects/IL/repos/plcio/browse/config/lcio.yaml Implementing almost a complete LCIO EDM based on PODIO Transition from LCIO to pLCIO is feasible at reasonable cost Event (EDM) was created from the YAML file for CEPCSW

Detector Description and Simulation Unified Geometry Service Interfaced to DD4HEP Used by simulation and reconstruction Simulation tool Integrated with physics generator & Geant4 Simulation Reconstruction GeoSvc DD4hep Physics List Detector Construction GeoSvc DD4hep WHIZARD Generator GenAlg Primary Generator Action DetSimAlg DetSimSvc Run Manager Run Action Stepping Action StdHep Files MCParticle SimTrackerHit Event Action Tracking Action Data flow Invoke SimCalorimeterHit User Actions

Infrastructure and External Libraries Common tools CMake: Build & deployment Gaudi cmake macros Git: version control http://cepcgit.ihep.ac.cn/cepc-prototype CVMFS: software distribution CEPC specific: /cvmfs/cepcsw.ihep.ac.cn/prototype External libraries Gaudi, ROOT, Geant4, DD4HEP, LCIO, PODIO, pLCIO etc. Generator CEPCSW Simulation Reconstruction Event Gaudi Framework Core Software layer LCIO PODIO pLCIO ROOT Geant4 DD4hep gcc python cmake External Libraries & Tools

CEPCSW Prototype Tasks Develop components: LCIO compatible reader During the migration, reading existing simulated samples. Unified geometry system for Sim/Rec/Ana Evaluate DD4hep Also integrate and migrate existing Sim/Rec/Ana algorithms. Evaluate and adopt pLCIO EDM Compatible with LCIO EDM ROOT-based Event Data Model & I/O Compatible with LCIO EDM. CEPC workshop at Oxford April, 2019 After Bologna workshop June, 2019

CEPCSW Timeline Date Component Task Manpower 2018.11 General Design, requirements discussion Jiaheng, Xingtao, Ziyan, Tao Core+Sim 2019.04 Oxford Workshop (prototype proposal) Core Event Data Model & ROOT I/O Jiaheng, Xingtao 2019.05 Sim Integration with DD4hep 2019.06 Release Initial prototype (Core+Sim) Tutorial & Documentation 2019.07 Rec Migration of algorithms Chengdong, Yao 2019.11 Demonstration with sim+track rec Tutorial Event Data Model & I/O，读入原有数据，进行分析 模拟 年底完成 Already 1-2 months lagged behind the schedule set at Oxford ! Hope migration of reconstruction algorithms will start as soon as possible.

Summary and plans A well designed data processing chain is important in short term (R&D) and long term (Operation). To fulfill the requirements, a CEPCSW prototype is being developed Developing based on Gaudi framework DD4HEP, PODIO/pLCIO were adopted Easy to migrate and reuse existing algorithms Have joined the international collaboration on Common Software Stack for future HEP experiments Updated plans: August 2019: initial prototype with core software and simulation. Nov 2019: demo with simulation and track reconstruction.

Thank You ! 谢谢