1. Glexec/SCAS Pilot: IN2P3-CC status
07/09/2018
2009/04/08
Glexec/SCAS Pilot: IN2P3-CC status
Pierre Girard
CCIN2P3 T1-T

2. Grid deployment at CCIN2P3 Initial plan for pilot of Glexec/Scas
07/09/2018
Content
Grid deployment at CCIN2P3
Initial plan for pilot of Glexec/Scas
Setting-up issues
Conclusion
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

3. Grid Job Management at CCIN2P3
07/09/2018
Grid Job Management at CCIN2P3
Several Grid WN versions at time
AFS
Computing
Element
Computing
Element
Computing
Element
Computing
Element
Glite-WN glexec
Glite-WN prod
BQS
Glite-WN prod
Anastasie
Glite-WN pps
No MW locally on worker WN WN WN WN WN WN WN WN
Globus4-WN
Shared FS (afs.in2p3.fr)
Computing
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

4. Overview of grid job submission
07/09/2018
Overview of grid job submission
Grid Job
Credentials 1
RSL WN
U-job
Submit
Glite-WN
Computing
Element
lcg sh
U-job
SL4.5 4
Job Manager
spawn 2
Local Job Wrapping
lcg sh 3
BQS
#!/bin/sh
#PBS -q T
#PBS -l M=2200MB
#PBS -l T=
#PBS -l scratch=16250MB
#PBS -l platform=LINUX
#PBS --share T1prod …
qsub
U-job
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

5. WN profile selection by BQS JobManager
07/09/2018
WN profile selection by BQS JobManager
Grid Job
Credentials 1
RSL WN
U-job
Submit
BQS-JM config
lcg sh
Glite-WN
Computing
Element 6
U-job
Dynamically link to WN profile
SL4.5
BQS JM 5
rules
spawn 2
Globus4-WN
Glite-WN pps
Glite-WN prod
Glite-WN prod
Glite-WN glexec
Local Job Wrapping
lcg sh 3 4
BQS
Set WN profile
qsub
Glite-WN glexec
U-job
AFS
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

6. Last BQS JM enhancements
07/09/2018
Last BQS JM enhancements
BQS JM control
Submission policy (deny, accept)
Forbearance management if BQS becomes unresponsive
BQS JM Outputs
BQS submission parameters
Class: A (=short), G (=Medium), T (=Long), J (=verylong)
Amount of {Mem, CPU, Scratch}
Farm name
Platform (SL3, SL4, SL5)
Logical resources (list of)
u_dcache_atlas, u_dcache_alice, u_OracleStress_atlas, …
VO Share
Wrapped data
WN profile to be used
profilesDirectory = /afs/in2p3.fr/grid/profiles/glite/ /SL4_64/WN32
Site Name
AFS token (or not)
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

7. Last BQS JM enhancements
07/09/2018
Last BQS JM enhancements
BQS JM configuration capabilities
(Most of) BQS JM outputs are determined according to configuration rules
A rule is basically an assignment
Ex.: SubmissionPolicy = ACCEPT
But can be conditionned depending on some job input data (in the precedence order)
Mapped account
Mapped group
CE queue
Ex.:
UserSubmissionPolicy_atlas050 = DENY
# Specific requirements for ATLAS with queue verylong
GroupVirtualQueueMaxMem_atlas_verylong = default
GroupVirtualQueueMaxCPU_atlas_verylong = max
GroupVirtualQueueMaxScratch_atlas_verylong = default
Configuration syntax
Is quite ugly
Makes the condition combination not possible
But, seems enough for now
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

8. Glexec deployment at IN2P3-CC
07/09/2018
Glexec deployment at IN2P3-CC
Glexec is a tool to be deployed on the WN
to be used by the VOs to manage the « real user jobs » within a job pilot
With a setuid capability (job pilot forks the « real user job » by using another account)
Site authorization by « real user job » based on real user proxy
How the deployment was planned
Deploy the Glite-WN/Glexec relocated on AFS
Use the configuration capabilities to redirect the pilot jobs to this deployment
profilesDirectory = /afs/in2p3.fr/grid/profiles/glite/ /SL4_64/WN32
UserProfilesDirectory_dteam049 = /afs/in2p3.fr/grid/profiles/glite/ /SL4_32/WN32_GLEXEC
Sounded easy…
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

9. Glexec deployment Issues at IN2P3-CC
07/09/2018
Glexec deployment Issues at IN2P3-CC
Glexec requires to be locally installed on Worker
Configuration file absolute path hardcoded
/opt/glite/etc/glite.conf
Only one MW configuration possible
Dynamic library configuration (due to « setuid »)
/etc/ld.so.conf
Only one MW installation possible
Log configuration (syslog)
Not so problematic for now
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

10. Glexec deployment in use at IN2P3-CC
07/09/2018
Glexec deployment in use at IN2P3-CC
We are part of the « SCAS Pilot Service »
Asked to provide SCAS/glexec in production
Load test for SCAS services by Atlas and Lhcb
Deployment done
Useable by both LHCb and Dteam
Through the T1 CEs
According to specific VOMS roles/groups
But
Deployment issues
Break down our WN setup strategy
Relocatable distribution was not ready (home-made)
First tests with LHCb
Were not satisfactory
Raised some questions
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08

11. Glite-WN only Will be activated Glite-3.2.0 (SL5) at IN2P3-CC
07/09/2018
Glite (SL5) at IN2P3-CC
Glite-WN only
Deployed on AFS
Tested with a test CE on BQS Farm « lcg »
Will be activated
as soon as SL5 workers enter the production (done)
A queue will be added to the T2 (T1?) CEs
Pierre Girard - Glexec/SCAS: IN2P3-CC status
2009/04/08