1. Bulk production of Monte Carlo
MICE Collaboration
Bulk production of Monte Carlo
Dimitrije Maletic
Institute of Physics, University of Belgrade
MICE Project Board, 7th of March 2017.

2. MICE Project Board, 7th of March 2017.
Outline
Introduction
MCproduction on the grid information pages
Information about finished MCproductions
Resources available for MCproduction
Conclusions
MICE Project Board, 7th of March 2017.

3. MICE Project Board, 7th of March 2017.
Introduction (1/3)
MCproduction is regularly discussed at Grid and Data Mover meetings, as a part of Software and Computing Infrastructure project. Talks are also given on Analysis meetings and MICE CMs.
The MC production on the grid includes G4BeamLine simulation and G4 Cooling Channel simulations using MAUS.
MAUS MC production, using already made G4BL libraries, was done upon request with no delays, since March last year.
As of last month, the G4BeamLine production was restarted on the grid, and the grid production of Version 4 of the G4BeamLine libraries started.
MICE Project Board, 7th of March 2017.

4. MICE MC production on the grid block schema
Introduction (2/3)
MICE MC production on the grid block schema
Image by Henry Nebrensky
MC production running on sites supporting MICE VO.
MAUS or G4BL output (or replica) copied to Imperial SE for http access.
Also, copy of aggregated output files should be on RAL T1 tape.
MICE Project Board, 7th of March 2017.

5. MICE Project Board, 7th of March 2017.
Introduction (3/3)
The MC production on the grid starts with the request on the request page.
The production manager (me) should be informed about the request. I discuss the request with Durga.
Then I insert the entry about the MC production into the CDB, and submit the grid jobs.
In order to be able to continue with the MC production, production manager has to have a valid grid certificate and to be included into MICE VOMS.
The MCproduction is using the MAUS software installed on CVMFS on the grid.
Necessary information for MC simulation are http/srm list of G4Beamline chunks, MAUS SW Version, and a simulation datacard details.
MAUS accesses the CDB to get appropriate configuration and calibrations, defined by reading of the datacard.
Each request/start of MCProduction, is tagged with unique MCSerialNumber (row number in CDB table).
Information about MC production and output links are placed at the MCproduction page, linked from MICE Software home page.
MICE Project Board, 7th of March 2017.

6. MCproduction on the grid information pages (1/2)
Information about finished and running MCproductions on the grid:
Information ( and also as examples ) of MCproduction Requests
Information about MCproduction requests entries in CDB
You can check MCSerialNumber entries in CDB:
The scripts used for MC production on the grid are available on launchpad
MICE Project Board, 7th of March 2017.

7. MCproduction on the grid information pages (2/2)
Http access:
33 productions done till March 2016.
New MCproductions.
With
MCSerialNumber
entries in CDB
Old MCproductions.
WithNO
MCSerialNumber
entries in CDB,
only in preprodcdb
MICE Project Board, 7th of March 2017.

8. MICE Project Board, 7th of March 2017.
Latest production:
Number of outputs
stored / time of day
interval
48 17:40-49
:50-59
56 18:00-09
:10-19
:20-29
89 18:30-39
:40-49
:50-59
67 19:00-09
2 19:10-19
8 19:20-29
2 19:30-39
3 19:40-49
4 19:50-59
2 20:00-09
1 20:20-29
1 20:30-39
1 20:40-49
Information about finished MCproductions
Number of jobs started since March is 52811
Parallel running jobs : mean 45.5, maximum 1382.
Storage space used for MCproduction is 642 GB
Processing time for MCproduction not an issue.
No Jobs | The Logging and Bookeeping Subsystem (LB) used by WMS
5632
5643
3185
6926
No jobs started: 52811, parallel running jobs: mean 45.5 maximum 1382.
Broj jobova - statistika:
322 45, ,
MICE Project Board, 7th of March 2017.

9. Resources available for MCproduction
Running Waiting Total Free Queue
arc-ce01.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce02.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce03.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce04.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
ce-01.roma3.infn.it:8443/cream-pbs-fastgrid
ce-01.roma3.infn.it:8443/cream-pbs-grid
…………
ceprod08.grid.hep.ph.ic.ac.uk:8443/cream-sge-grid.q
cream2.ppgrid1.rhul.ac.uk:8443/cream-pbs-mice
dc2-grid-21.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-22.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-25.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-26.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-28.brunel.ac.uk:2811/nordugrid-Condor-default
hepgrid2.ph.liv.ac.uk:2811/nordugrid-Condor-grid
heplnv146.pp.rl.ac.uk:2811/nordugrid-Condor-grid
svr009.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr010.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr011.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr019.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
Status of available slots for grid jobs for MICE VO
MICE VO View
:00
Total (Running/Waiting) Free
Free Used Reserved Free Used Reserved Tag SE
Online Online Online Nearline Nearline Nearline
dc2-grid-64.brunel.ac.uk
…….
gfe02.grid.hep.ph.ic.ac.uk
heplnx204.pp.rl.ac.uk ……
se01.dur.scotgrid.ac.uk
se2.ppgrid1.rhul.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_MISC_TAPE srm-mice.gridpp.rl.ac.uk
MICE_RECO srm-mice.gridpp.rl.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_RAW_TAPE srm-mice.gridpp.rl.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_RAW_TAPE srm-mice.gridpp.rl.ac.uk …
Free GB
Used GB
Available storage space for MICE VO
Imperial SE
Free GB
Used GB
At RAL (srm-mice only):
Disk Used: % 4,593 GB
Disk total: 11,183 GB
Tape used: 20,054 GB
Tape to go
to 300 TB
MICE Project Board, 7th of March 2017.

10. MICE Project Board, 7th of March 2017.
Conclusions
Processed 33 MCproductions on the grid. Used 642 GB of storage space. Processing time for MCproduction not an issue.
Available processing power and storage capacity are not a bottleneck for running more (much more than the order of magnitude) productions per year.
My availability is not an issue.
Permanent employment.
Management of my institute and colleagues from my Laboratory are very happy that we are part of international collaboration on UK based experiment.
Expecting/ready for increase of requests for MC production after finished February/March ISIS user run cycle. Using new G4BeamLine libraries and MAUS-v2.8.4 version on CVMFS.
MICE Project Board, 7th of March 2017.

11. THANK YOU!