1. Submit BOSS Jobs on Distributed Computing System
Xianghu Zhao
Oct 17, 2017

2. Outline Introduction Request a certificate, join BES VO
Setup software environment
Submit jobs
Job monitoring

3. Introduction More than 10 sites joined BESIII distributed computing
About 1600~2000 CPU cores available
Vary because of resource sharing
Detailed user manual on BESIII wiki
ESDIRAC_User_Tutorial

4. Request a Certificate https://cagrid.ihep.ac.cn/
Follow the guide of “Request a User Certificate”
Submit the User Certificate Application Form
Both and printed form to the RA Operator
Online request for certificate
Get certificate in about 2 days
Receive and follow the link

5. Export Certificate The certificate is installed on your web browser
Export to p12 file format
Firefox for example

6. Install Certificate on lxslc
Copy the p12 certificate to lxslc
Convert to PEM format and install to $HOME/.globus
dirac-cert-convert.sh YOUR_CERT.p12
If “command not found”, first source /cvmfs/dcomputing.ihep.ac.cn/dirac/DIRAC_Client/v0r17/ bashrc
Two files will be installed in $HOME/.globus
userkey.pem, usercert.pem

7. Tips on Request Certificate
Recommended browser: Firefox, Chrome
Use the same browser when doing request and getting the certificate

8. Join BES VO
In order to get the access to use the BES distributed computing resources, join the BES V
Follow the guidelines and you will receive for confirmation
You will be notice by after the administrator approves your apply

9. Setup Software Environment
Copy and edit the configuration file for the first time
cp /cvmfs/dcomputing.ihep.ac.cn/frontend/gangadist/.gangarc ~/.gangarc
Modify the value of “gangadir” to your own directory (with enough disk space)
Setup environment
source /cvmfs/dcomputing.ihep.ac.cn/frontend/gangadist/env_grid.sh
You will be prompt to enter password here
Do this step after each login
You can put this line in ~/.bashrc for convenience. Be sure to put this line after BOSS setup

10. Submit Jobs
ganga is provided for submitting batch jobs on distributed computing system
Find examples in “/workfs/bes/zhaoxh/ganga_example”
Use the following command to submit
ganga simrec_byrun.py

11. Ganga Script Written in python The job group is useful for manage jobs
You can select sites which jobs run on
Default for all available sites
Output files will be located under /scratchfs/bes/<user>/<outputDir>

12. More about Ganga Script
A unique random seed is automatically selected for each job
Output data type could be configured by “output_step”: sim, rec, ana
More details about options in script
ESDIRAC_BOSS_Job_Guide

13. Job Types Supported Simulation
Simulation and then reconstruction in a single job
Simulation, reconstruction and analysis in a single job
For jobs with reconstruction, the splitter could only be “UserSplitterByRun”
If you would like to use your own package (not from the official), add option “user_custom_package=True”

14. Job Output
Output files and logs will be found under /scratchfs/bes/<user>/<outputDir>
Be aware that this directory is temporary, old files will be automatically deleted

15. Job Monitoring Visit https://dirac.ihep.ac.cn
Open page “Job Monitor” from left bottom menu
Failed jobs could be rescheduled
Select jobs with job group

16. Task Monitoring Batch jobs could be managed by task
Manage_Jobs_with_Task_Manager

17. Thanks!