1. Bulk production of Monte Carlo
MICE Collaboration
Bulk production of Monte Carlo
Dimitrije Maletic
Institute of Physics, University of Belgrade
MICE Project Board, 7th of March 2017.

2. MICE Project Board, 7th of March 2017.
Outline
Software used for MICE MC simulations
MCproduction on the grid
Information about finished MCproductions
Resources available for MCproduction
Conclusions
MICE Project Board, 7th of March 2017.

3. Divide and conquer Monte Carlo
The full MICE experiment simulation is pieced out in manageable components for speed and versatility.
Target hadroproduction simulation
MICE beam line simulation
MICE experiment simulation
MICE runs with many beam line and experimental configurations and
needs a wide range of simulations.
Tunable currents in the dipoles, quads, solenoids
Choice of proton absorber thickness
Variable absorber material, diffuser thickness
Adjustable geometry during commissioning (detectors move)
Requires a robust infrastructure to handle many MC production jobs
MICE Project Board, 7th of March 2017.

4. Target hadroproduction and G4BeamLine
All secondary particle species are sampled from a
parent distribution originally produced by tracking
1.97 x 109 POT using G4Beamline:
- Geant4 physics based simulation.
- Would be very inefficient to simulate from target each time
G4BeamLine simulation steps:
1. Sample p+, p+, p -, e+, e-, g
from secondary parent distribution
2. Match though the quads Q1-3 fields
3. Most pions decay in flight in the decay solenoid
between the two bending magnets. Propagation
through the fields and scattering handled by G4Beamline, particles that hit the magnets bores are killed
4. Store a file with a list of particles and their parameters 1m downstream of D2.
MICE Project Board, 7th of March 2017.

5. MICE Analysis User Software
Downstream of D2, the MICE Analysis User Software (MAUS) takes over
- Uses the G4BL "chunks" as an input beam
- Geant4 based simulation
- Handles complex detector geometries and field interpolation
- Robust connection with the reconstruction software to ensure
consistency between truth and digitized MC
MICE Project Board, 7th of March 2017.

6. MICE on the grid organisation
MICE MC production on the grid block schema
Image by Henry Nebrensky
MICE on the grid has CERN like organisation into Tears.
MC production running on sites supporting MICE VO.
MAUS or G4BL output (or replica) copied to Imperial SE for http access.
Also, copy of aggregated output files should be on RAL T1 tape.
MICE Project Board, 7th of March 2017.

7. About MICE MC production
MCproduction is regularly discussed at Grid and Data Mover meetings, as a part of Software and Computing Infrastructure project. Talks are also given on Analysis meetings and MICE CMs.
The MC production on the grid starts with the request on the request page.
The production manager (me) should be informed about the request. I discuss the request with Durga.
Then I insert the entry about the MC production into the CDB, and submit the grid jobs. (needs a valid grid certificate and to be included into MICE VOMS)
The MCproduction is using the MAUS software installed on CVMFS on the grid.
Necessary information for MC simulation are http/srm list of G4Beamline chunks, MAUS SW Version, and a simulation datacard details.
Information about MC production and output links are placed at the MCproduction page, linked from MICE Software home page.
MICE Project Board, 7th of March 2017.

8. MCproduction on the grid information pages (1/2)
Information about finished and running MCproductions on the grid:
Information ( and also as examples ) of MCproduction Requests
Information about MCproduction requests entries in CDB
You can check MCSerialNumber entries in CDB:
The scripts used for MC production on the grid are available on launchpad
MICE Project Board, 7th of March 2017.

9. MCproduction request page
G4BeamLine libraries:
3-140+M3 (8161) : 3_140_M3.json M3 (8775): 6_140_M3.json
M3 (8773): 10_140_M3.json M3 (8046): 3_170_M3.json
3-200+M3 (8163): 3_200_M3.json M3 (8794): 6_200_M3.json
M3 (8818): 10_200_M3.json M3 (8180): 3_240_M3.json
6-240+M3 (8873): 6_240_M3.json M3 (8907): 10_240_M3.json
MAUS MC grid production does not simulate all runs from the experiment, but one of the runs with same Beam line and Cooling channel configurations.
7469 Emittance production (Done)
Cards:
-geometry_download_by “run_number"
-geometry_download_run_number 7469
-TOF_calib_by = "date"
-TOF_cabling_by = "date"
-TOF_calib_date_from = ' :00:00'
-TOF_cabling_date_from = ' :00:00'
-TOFscintLightSpeed = # mm/ns
G4BL input:
Software: MAUS-v2.5.0
MeV ECE FC44 (Done)
-geometry_download_run_number 8154
G4BL input:
Software: MAUS-v2.6.0
MICE Project Board, 7th of March 2017.

10. MCproduction on the grid information pages (2/2)
Http access:
33 productions done till March 2016.
New MCproductions.
With
MCSerialNumber
entries in CDB
Old MCproductions.
WithNO
MCSerialNumber
entries in CDB,
only in preprodcdb
MICE Project Board, 7th of March 2017.

11. MICE Project Board, 7th of March 2017.
Information about finished MCproductions
Number of jobs started since March is 52811
Parallel running jobs : mean 45.5, maximum 1382.
Storage space used for MCproduction is 642 GB
There were 33 productions / 365 days.
Simulation processed in few up to 10h a day (depending on a queues on grid sites).
Processing time for MCproduction not an issue.
MICE Project Board, 7th of March 2017.

12. Resources available for MCproduction
Running Waiting Total Free Queue
arc-ce01.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce02.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce03.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
arc-ce04.gridpp.rl.ac.uk:2811/nordugrid-Condor-grid3000M
ce-01.roma3.infn.it:8443/cream-pbs-fastgrid
ce-01.roma3.infn.it:8443/cream-pbs-grid
…………
ceprod08.grid.hep.ph.ic.ac.uk:8443/cream-sge-grid.q
cream2.ppgrid1.rhul.ac.uk:8443/cream-pbs-mice
dc2-grid-21.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-22.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-25.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-26.brunel.ac.uk:2811/nordugrid-Condor-default
dc2-grid-28.brunel.ac.uk:2811/nordugrid-Condor-default
hepgrid2.ph.liv.ac.uk:2811/nordugrid-Condor-grid
heplnv146.pp.rl.ac.uk:2811/nordugrid-Condor-grid
svr009.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr010.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr011.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
svr019.gla.scotgrid.ac.uk:2811/nordugrid-Condor-condor_q2d
Status of available slots for grid jobs for MICE VO
MICE VO View
:00
Total (Running/Waiting) Free
Mcproduction mean
of parallel jobs 45
Available storage space for MICE VO
Free Used Reserved Free Used Reserved Tag SE
Online Online Online Nearline Nearline Nearline
dc2-grid-64.brunel.ac.uk
…….
gfe02.grid.hep.ph.ic.ac.uk
heplnx204.pp.rl.ac.uk ……
se01.dur.scotgrid.ac.uk
se2.ppgrid1.rhul.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_MISC_TAPE srm-mice.gridpp.rl.ac.uk
MICE_RECO srm-mice.gridpp.rl.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_RAW_TAPE srm-mice.gridpp.rl.ac.uk
srm-mice.gridpp.rl.ac.uk
MICE_RAW_TAPE srm-mice.gridpp.rl.ac.uk …
Free GB
Used GB
Imperial SE
Free GB
Used GB
MCproduction
usage 640GB
At RAL (srm-mice only):
Disk Used: % 4,593 GB
Disk total: 11,183 GB
Tape used: 20,054 GB
Tape to go
to 300 TB
MICE Project Board, 7th of March 2017.

13. MICE Project Board, 7th of March 2017.
Conclusions
MAUS MC production, using already made G4BL libraries, was done upon request with no delays, since March last year. As of last month, the G4BeamLine production was restarted.
Processed 33 MCproductions on the grid. Used 642 GB of storage space. Processing time for MCproduction not an issue.
Available processing power and storage capacity are not a bottleneck for running more (much more than the order of magnitude) productions per year.
My availability is not an issue.
Permanent employment. Management of my institute and colleagues from my Laboratory are very happy that we are part of international collaboration on UK based experiment.
Expecting/ready for increase of requests for MC production after finished February/March ISIS user run cycle. Using new G4BeamLine libraries and MAUS-v2.8.4 version on CVMFS.
MICE Project Board, 7th of March 2017.

14. THANK YOU!