A cloud-based architecture for the Cherenkov Telescope Array observation simulations. Optimisation, design, and results Marco Landoni (INAF – OA Brera) marco.landoni@inaf.it We acknowledge contribution from the grant INAF CTA–SKA, ‘Probing particle acceleration and g-ray propagation with CTA and its precursors’

Publications/Presentations ApJ paper submitted (minor revision received two days ago) Poster @ AGN 13 Case Study (technological transfer between AWS and INAF)

The problem Suppose you want to simulate a source (using CTOOLS) with fluxes that range from 1 Crab to 1mCrab (and 50hrs of observation)

The problem Same applies for the confidence level for the detection (according to the TS statistics). For faint sources (or for few hours of observations) results could be very different, as in the case of the determination of the flux.

We need to run many simulations… Many sources Many different spectral type to test Background IRFs Ctobssim + ctlike could easly made the computation time divergent (hundred of thousands of CPU hours..) So…

How to improve the running time ? Deploy the software on a cluster with many CPU cores and adeguate amount of storage to run as many simulation as possible. But.. Software maitenance could be problematic (shared libraries, calibration databases, path variables, ecc.) A solution could be the containerisation of the software, a method to build an executable files that contains ctools, shared libraries, calibrations (IRFs), … Dire anche che puoi testare il SW in locale. E poi non va sul cluster

Containerisation of the software We used Docker to containerise the CTOOLS with all the libraries and calibrations needed. MyPortableSoftware Docker -> light virtual machine CTOOLS (ctobssim, ctlike,..) Shared library (e.g. gammalib) Docker run MyPortableSoftware ctobssim <params> Operating System (minimal)

Deploy on AWS cluster Main ingredients: - CPU power (basically, computer around the world): - Storage (a distributed Filesystem to store results from simulations) Orchestrator (a service that create the cluster and tells each computer what simulation shall be performed in a queue)

The architecture Con questa architettura siamo riusciti a scodare migliaia di simulazioni complete , tra ctobssim e ctlike, per i papers di cui parlerò dopo

Where our approach has been used ? Many different (peer reviewed) publications in the context of PRIN activities (mainly) Landoni et al 2018, A cloud based architecture for the CTA simulations (ApJ, minor review) Romano et al 2018, Prospects for gamma-ray observations of narrow-line Seyfert 1 galaxies with the Cherenkov Telescope Array (MNRAS published) Tavecchio et al 2018, Putting the hadron beam scenario for extreme blazars to the test with the Cherenkov Telescope Array (MNRAS published). Lamastra et al 2018, Game of winds: unveiling the origin of the gamma-ray emission in NGC 1068 with the Cherenkov Telescope Array (submitted to MNRAS) Dire poi che ci sono ongoing i lavori legate ad ALP + Lorentz Invariance Violation (KSP, etc) Papers in the context of CTA collaboration at large Chernyakova et al 2018, CTA perspective of gamma-rays binaries (A&A) Tight collaboration in the context of ASTRI (e.g. Bottcher et al 2018, see P. Romano talk)

How much does it cost ? The average cost for each publication has been estimated around 300-400 EUR, a budget that is comparable with the price of a couple of printed colour figure on major journal. We are able to smartly use resources without impact on clusters and grid mainly dedicated to long, frequent and network bounded simulations (for example in the context of cosmology).

Thanks!