Submit BOSS Jobs on Distributed Computing System Xianghu Zhao Oct 17, 2017

Outline Introduction Request a certificate, join BES VO Setup software environment Submit jobs Job monitoring

Introduction More than 10 sites joined BESIII distributed computing About 1600~2000 CPU cores available Vary because of resource sharing Detailed user manual on BESIII wiki http://docbes3.ihep.ac.cn/~offlinesoftware/index.php/B ESDIRAC_User_Tutorial

Request a Certificate https://cagrid.ihep.ac.cn/ Follow the guide of “Request a User Certificate” Submit the User Certificate Application Form Both email and printed form to the RA Operator Online request for certificate Get certificate in about 2 days Receive email and follow the link

Export Certificate The certificate is installed on your web browser Export to p12 file format Firefox for example

Install Certificate on lxslc Copy the p12 certificate to lxslc Convert to PEM format and install to $HOME/.globus dirac-cert-convert.sh YOUR_CERT.p12 If “command not found”, first source /cvmfs/dcomputing.ihep.ac.cn/dirac/DIRAC_Client/v0r17/ bashrc Two files will be installed in $HOME/.globus userkey.pem, usercert.pem

Tips on Request Certificate Recommended browser: Firefox, Chrome Use the same browser when doing request and getting the certificate

Join BES VO In order to get the access to use the BES distributed computing resources, join the BES V https://voms.ihep.ac.cn:8443/voms/bes/ Follow the guidelines and you will receive email for confirmation You will be notice by email after the administrator approves your apply

Setup Software Environment Copy and edit the configuration file for the first time cp /cvmfs/dcomputing.ihep.ac.cn/frontend/gangadist/.gangarc ~/.gangarc Modify the value of “gangadir” to your own directory (with enough disk space) Setup environment source /cvmfs/dcomputing.ihep.ac.cn/frontend/gangadist/env_grid.sh You will be prompt to enter password here Do this step after each login You can put this line in ~/.bashrc for convenience. Be sure to put this line after BOSS setup

Submit Jobs ganga is provided for submitting batch jobs on distributed computing system Find examples in “/workfs/bes/zhaoxh/ganga_example” Use the following command to submit ganga simrec_byrun.py

Ganga Script Written in python The job group is useful for manage jobs You can select sites which jobs run on Default for all available sites Output files will be located under /scratchfs/bes/<user>/<outputDir>

More about Ganga Script A unique random seed is automatically selected for each job Output data type could be configured by “output_step”: sim, rec, ana More details about options in script http://docbes3.ihep.ac.cn/~offlinesoftware/index.php/B ESDIRAC_BOSS_Job_Guide

Job Types Supported Simulation Simulation and then reconstruction in a single job Simulation, reconstruction and analysis in a single job For jobs with reconstruction, the splitter could only be “UserSplitterByRun” If you would like to use your own package (not from the official), add option “user_custom_package=True”

Job Output Output files and logs will be found under /scratchfs/bes/<user>/<outputDir> Be aware that this directory is temporary, old files will be automatically deleted

Job Monitoring Visit https://dirac.ihep.ac.cn Open page “Job Monitor” from left bottom menu Failed jobs could be rescheduled Select jobs with job group

Task Monitoring Batch jobs could be managed by task http://docbes3.ihep.ac.cn/~offlinesoftware/index.php/ Manage_Jobs_with_Task_Manager

Thanks!