EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Large Simulations using EGEE Grid for the Pierre Auger Observatory Jiří Chudoba 1,2, Jaroslava Schovancová 1,2, Petr Trávníček 1 1) Institute of Physics, AS CR and 2) CESNET , ASPERA meeting, Paris

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Pierre Auger Collaboration More than 250 PhD scientists from more than 60 institutions from 16 (+2) countries > 400 authors Participating countries: Argentina, Australia, Bolivia *, Brazil, Czech Republic, France, Germany, Italy, Mexico, Netherlands, Poland, Portugal, Slovenia, Spain, United Kingdom, USA and Vietnam * * - associated countries

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Pierre Auger Observatory Southern observatory almost completed Some results already presented Southern hemisphere: Malargüe, Mendoza province, Argentina Northern hemisphere (planned): Lamar, Colorado, USA Lifetime of the observatory: yrs

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Auger Computing Model Data recorded at the experiment site are copied to Lyon - CCIN2P3 –slow line from Malargue, transfer via recorded DVD –EU funded project AUGERACCESS will improve connectivity All Auger members have accounts at CCIN2P3 Data is analyzed remotely at CCIN2P3 or copied and processed on PCs or small computing farms –users must install new versions of the offline sw –users check for availability of new data CPU intensive simulations done at various sites and results copied to CCIN2P3 –some sets are not at CCIN2P3, available to only some users

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Expectations from Grid more computing resources available to more collaboration members uniform way for simulation productions easier access to data tests with simulations were done using regional VOs (central Europe - VOCE, SouthWest)

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris VO AUGER Created in 2006 by the Czech group in cooperation with CESNET (Czech EGEE partner) CESNET provides and maintains central services –LCG RB, gLite WMS, LB, UI, LFC, registration portal, VOMS server, JP server VOMS server replica at FNAL 10 sites from 8 countries support VO AUGER EELA project considers AUGER as supported application –E-infrastructure shared between Europe and Latin America – IST www.eu-eela.org –UNAM (National Autonomous University of Mexico) is the 1 st EELA site supporting VO AUGER

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Registration to the VO AUGER

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Resources, Usage Production of libraries of cosmic ray showers Access to the simulated data files Resources for individual users golias25.farm.particle.czPrague, ASCR156 TB skurut17.cesnet.czPrague, CESNET201 TB apcpc79.in2p3.frGRIF (APC), Paris50 TB grid10.lal.in2p3.frGRIF (LAL), Paris150 TB ce02.lip.ptLIP, Lisboa20 TB tbn20.nikhef.nlNIKHEF, Amsterodam200 TB lcgce.ijs.siIJS, Ljubljana102 TB UNAM40 TB CCIN2P3, Lyon FZK, Karlsruhe10050 TB Aachen8515 TB Wuppertal5025 TB Number of CPUs and available storage for the VO Auger are rough estimates !! They fluctuate by a large factor

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris CPU Challenge in April 2007 –number of really usable CPUs varies

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Corsika simulations Offline software: CORSIKA –(v for tests, v for proton and iron production) Model: –epos, fluka Primary particle –proton (epos_gr01, 3, 7) or iron (epos_gr04, 6) Energy of primary particle –log(E/eV)=17.5, 18.0, 18.5, 19.0, 19.5, 20.0, 20.5, 21.0 Zenith angle –q=0, 18, 26, 38, 45, 53, 60 deg 50 jobs per each [E, q] pair –different random number seed –seed generated using srand(time(NULL)) and rand() functions implemented in standard math.h library

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris CORSIKA Parameters

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Shower libraries epos_gr01proton, EPOS1 – 2800, test (deleted) epos_gr02proton, EPOStest (deleted) epos_gr03proton, EPOS epos_gr04iron, EPOS epos_gr05proton, EPOSJPtest epos_gr06iron, EPOS epos_gr07proton, EPOS

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Library productions 2800 jobs per production Test production (epos_gr01) –August 28 – September 10, 2007 (14 days) 1 st usable proton production (epos_gr03) –October 3 – 24, 2007 (major part in 21 days) –2745 jobs finished, < 2.5 % crashes 1 st iron production (epos_gr03) –October 22 – December 15, 2007 –2200 jobs finished, < 2.5 % crashes Job outputs stored on SE –Primarily at golias100.farm.particle.cz, later also at dcache.ijs.si

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Running jobs – proton showers CE epos_gr03 (proton) CE JobsCPU timeAvg. CPU [#][hr]time [hr] ce02.lip.ptLIP, Lisbon golias25.farm.particle.czFZU, Prague grid10.lal.in2p3.fr+apcpc79GRIF, Paris lcgce.ijs.siIJS, Ljublan skurut17.cesnet.czCESNET, Prague tbn20.nikhef.nlNIKHEF, Amsterdam JOBS COMPUTED (DONE) SHOWERS DONE TOTAL (SHOWERS) 2800

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Running jobs – iron showers CE epos_gr04 (iron) JobsCPU timeAvg. CPU [#][hr]time [hr] ce02.lip.ptLIP, Lisbon golias25.farm.particle.czFZU, Prague grid10.lal.in2p3.fr + apcpc79GRIF, Paris lcgce.ijs.siIJS, Ljublan skurut17.cesnet.czCESNET, Prague tbn20.nikhef.nlNIKHEF, Amsterdam JOBS COMPUTED (DONE) SHOWERS DONE TOTAL (SHOWERS) 2800 example of a logical file name: lfn:/grid/auger/prod/epos_gr04/en20.000/th18.000/DAT part

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Site ratios

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Walltime per site

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Used walltime per library

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Summary production of CORSIKA libraries for AUGER files are accessible by VO auger members plans to copy (part of) libraries to CCIN2P3 current productions limited by available storage working on the detector simulation on the grid expect at least double CPU capacity soon challenge to keep these resources busy

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Backup slides

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris gr06, gr07 CE epos_gr06 (iron) Jobs CPU time Avg. CPU [#][hr]time [hr] ce02.lip.ptLIP, Lisbon golias25.farm.particle.czFZU, Prague gridka...FZK, Karlsruhe00-- grid10.lal.in2p3.frLAL, Paris lcgce.ijs.siIJS, Ljublan skurut17.cesnet.cz CESNET, Prague tbn20.nikhef.nl NIKHEF, Amsterdam JOBS COMPUTED (DONE) SHOWERS DONE TOTAL (SHOWERS) 2800 CE epos_gr07 (proton) Jobs CPU time Avg. CPU [#][hr]time [hr] ce02.lip.ptLIP, Lisbon golias25.farm.particle.czFZU, Prague gridka...FZK, Karlsruhe17 1 grid10.lal.in2p3.frLAL, Paris lcgce.ijs.siIJS, Ljublan skurut17.cesnet.czCESNET, Prague tbn20.nikhef.nl NIKHEF, Amsterdam JOBS COMPUTED (DONE) SHOWERS DONE TOTAL (SHOWERS) 2800

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Jobs per day

Enabling Grids for E-sciencE EGEE-II INFSO-RI ASPERA meeting, Paris Walltime