1. The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre 11-15th, 2010 Oleg Lodygensky - LAL - lodygens@lal.in2p3.fr

2. 2 Authors : O. Lodygensky HUST : XtremWeb-HEP 2 EDGeS BOINC-based DGs XtremWeb/XWHEP-based DGs local XtremWeb DG IN2P3 200 PCs 300 (150%) new public XtremWeb DG EDGeS@home 1.000 PCs EDGeS@home EGEE VO public XtremWeb DG IN2P3 300 PCs 6 (2%) public XtremWeb DG AlmereGrid 3.000 PCs ? new public BOINC DG EDGeS@home 10.000 PCs EDGeS@home public BOINC DG SZDG 30.000 PCs 77.000 (256%) public BOINC DG Extremadura 70.000 PCs 22.500 (32%) local BOINC DG Westminster 1.500 PCs 1881 (125%) local BOINC DG Correlation Systems public BOINC DG AlmereGrid 0 PCs 1.700 public BOINC DG Ibervicis 0 PCs 24.000

3. 3 HUST : XtremWeb-HEP Authors : O. Lodygensky 3 EDGeS desktopgrid.vo voms lbwmsbdii an EGEE VO BOINC-EGEE bridge EGEE>DG bridge CE UoW queue SDZGr queue Almere queue Fundecyt queue BOINC-based DGs UoW, Correlation Systems local DGs SZDGr, Almere, Fundecyt public DGs AR UI Core Service Resource lfc bridge myproxy cewms ce

4. 4 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Bridging EGEE and Desktop Grids Grid User  X509 proxy  VOMS Proxy  Submits Job with VOMS proxy Publishes available Resources  Pushes Job   Log  Log VOMS Admin Manages VO Site Admin Manages Site   Accesses Data with VOMS proxy  Accesses Data with VOMS proxy   Gives Job Status Gives Accounting and Auditing VOMS Server Accounting Logging & Bookkeeping Meta-scheduler (WMS) Site Computing Resource Site Storage Resource Grid Admin  Sends back Output Sandbox  Sends back Output Sandbox VOMS Proxy = X509 proxy with VOMS extensions EGEE (Service Grid) Well defined infrastructure: - authentication - authorization - logging and bookkeeping - quality of service (QoS) A strong effort of standardization Volunteer PCs Desktop Grids : Boinc, XWHEP Undefined infrastructure: - anonymity - volatility How to bridge ? - security - monitoring - logging DG User ? Server (scheduler)

5. 5 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP->EGEE resource sharing

6. 6 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP XtremWeb by High Energy Physics (XWHEP) is developped by IN2P3. It is based on XtremWeb 1.8.0. by INRIA. http://www.xtremweb-hep.org/spip.php?rubrique16

7. 7 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP is a generic multi purposes desktop grid platform (DG) enabling eSciences computations over volatile nodes. Main features are : multi applications multi users user rights management access rights management three tiers architecture multi platforms (win32, linux, mac os x) virtual stable cluster over volatile volunteers individual PCs firewall bypassing automatic load balancing fault tolerance XWHEP

8. 8 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XtremWeb by High Energy Physics (XWHEP) is developped by IN2P3. It is based on XtremWeb 1.8.0. by INRIA. XWHEP main goals: full production platform inter grids connections (firstly focusing on EGEE). To achieve this goal, XWHEP proposes a secured DG: certified server; X509 user proxy usage; access rights; usage levels. http://dghep.lal.in2p3.fr/spip.php?rubrique16 XtremWeb by High Energy Physics (XWHEP) Main characteristics : three tiers architecture firewalls bypassing certified servers encrypted communications multi applications multi-OS ✓ Mac OS X ✓ Windows ✓ Linux cycle stealing fault tolerance volunteers PC authorization authentication confidentiality monitoring Multi users User client User task distribution

9. 9 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP : XtremWeb by High Energy Physics LAL : LCG Tiers-2 EGEE seed resource XtremWeb by High Energy Physics (XWHEP) developer & maintainer XWHEP is based on XtremWeb 1.8.0. by INRIA. http://www.xtremweb-hep.org/spip.php?rubrique16 XWHEP ensures security at different levels: authentication : ✓ SSL and X509 certificates authorization and confidentiality : ✓ user rights management ✓ user groups ✓ access rights volunteer nodes integrity : ✓ application repository ✓ sandboxing monitoring and bookeeping : ✓ all messages are logged XWHEP environment ➡ multi users ➡ multi applications

10. 10 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XW User Job Execution Sandbox User data and applications Dynamically downloaded XW user data and application External data servers Server certificate Server public key Job Mgt Deployed XW-Client UI Deployed XW-Computing Service PC Volunteer PC Data Sandbox All communications are encrypted Data Mgt XW Services Local I/O XWHEP : Architecture, authentication and resource integrity Authentication: - login/password - X509 cert. Node integrity: - sandboxing Authentication: - login/password - X509 cert. XW Coordinator (scheduler) XW Data Repository

11. 11 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky User rights define user interaction level none no action worker can update jobs even if not owner (e.g. set job status to COMPLETED) can insert results (as datas) for jobs they have computed advanced user insert/delete public and group applications, workers super user insert/delete users/usergroups stacked rights (2) (1) : standard user can only insert private applications (2) : worker level aims to override access rights accordingly to some security issues standard user insert/retrieve/update/delete datas, applications, jobs, sessions, groups retrieve objects accordingly to object access rights (1) XWHEP : Authorization Access rights help to confine accesses Access Types Default Access Rights Private700 Group750 Public755

12. 12 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP security : access rights Any object in XWHEP is associated with an access rights. Access rights are linux fs like : they are defined for the user (owner), the group and others : 0400 Allow read by owner. 0200 Allow write by owner. 0100 For applications, allow execution by owner. 0040 Allow read by group members. 0020 Allow write by group members. 0010 For applications, allow execution by group members. 0004 Allow read by others. 0002 Allow write by others. 0001 For applications, allow execution by others. Default access rights is 0x755 The xwchmod command helps to change access rights.

13. 13 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Access rights help to confine accesses Access Types Default Access Rights Private700 Group750 Public755 Authorization

14. 14 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Grou p job Privat e job Private applicati on Group applicati on Public applicat ion Publi c job Publi c job Publi c job Publi c job Grou p job Grou p job Grou p job Privat e job Privat e job Privat e job User group and access rights introduces application levels : public, group and private. Jobs access level depends of the level of the referenced application. There is no way to extend job access level, except with administrator user level. XWHEP security : access confinement

15. 15 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP : Confidentiality Three levels to confine deployment and executions: public, group and private Public worker can execute public jobs of all users, and public jobs only. Public job Public worker Group worker can execute public and group jobs of its group, and its group only. A public job submitted by a user of the group, and the group only. A group job submitted by a user of the group, and the group only. Group worker Private worker can execute public, group and private jobs of its owner, and its owner only. A public job submitted by the worker owner, and the owner only. A group job submitted by the worker owner and the owner only. A private job submitted by the worker owner and the owner only. Private worker

16. 16 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP Monitoring

17. 17 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP security : execution confinement XWHEP defines three usage levels to confine executions: I - public level Public worker can execute public jobs of all users, and public jobs only. Public workers Any public job

18. 18 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Group worker can execute public and group jobs of its group, and its group only. A public job submitted by a user of the group, and the group only. A group job submitted by a user of the group, and the group only. XWHEP security : execution confinement XWHEP defines three usage levels to confine executions: II - group level

19. 19 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Private worker can execute public, group and private jobs of its owner, and its owner only. A public job submitted by the worker owner, and the worker owner only. A group job submitted by the worker owner and the worker owner only. A private job submitted by the worker owner and the worker owner only. XWHEP security : execution confinement XWHEP defines three usage levels to confine executions: III - private level

20. 20 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP defines three worker (volunteer PC) types So that we can confine executions Public workers Group workers Private workers Public job Group job Private job Private worker can execute all jobs of its owner (and its owner only) Group worker can execute public and group jobs of all users of its group (and its group only) Public worker can execute public jobs of all users (and public jobs only) XWHEP security : execution confinement

21. 21 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP->EGEE resource sharings Pilot Jobs is a way to use a Grid infrastructure to deploy end user jobs with an external scheduler (i.e. a scheduler which is not part of the infrastructure itself). XtremWeb and Condor teams have introduced this as “Glide-in” in “XtremWeb & Condor : sharing resources between Internet connected Condor pools.” O. Lodygensky, G. Fedak, F. Cappello, V. Neri, M. Livny, D. Thain CCGRID 2003, Tokyo, JAPAN; May 12-15, 2003. EGEE experiments use Pilot Jobs LHCb Dirac CMS Glide-in ATLAS Panda ALICE Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)http://edms.cern.ch/document/855383

22. 22 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP->EGEE resource sharings Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)http://edms.cern.ch/document/855383 XWHEP solves these issues thanks to its innovative features: user rights management user rights delegation user groups user group applications access rights management Security is ensured at three levels: computing node. XWHEP includes a sandbox to isolate end user job computation only validated applications from repository are candidate to run on SG nodes Application and data integrity. application repository and data servers (including XWHEP) ensure integrity User authentication only X.509 certified users can use SG nodes users provide proxy certificate to submit a job to XWHEP scheduler this proxy is used to submit Pilot Jobs to SG

23. 23 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky XWHEP->EGEE Getting X509 proxy jLite by Oleg Sukhoroslov http://code.google.com/p/jlite/ jLite provides a high-level Java API with basic functionality similar to gLite shell commands. This API hides the complexity of underlying middleware and its configuration XWHEP use jLite API to easily manage X509 certificates with VOMS extensions.

24. 24 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky VOMS Server Meta-scheduler (WMS) DG User jLite : proxy init XW Coordinator (scheduler) jLite by Oleg Sukhoroslov http://code.google.com/p/jlite/ Public worker Group worker Private worker User X.509 Cert User X.509 proxy Server certificate Server public key Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)http://edms.cern.ch/document/855383 XWHEP->EGEEbridging XW Bridge Submit Pilot job w/ user X509 proxy Retreived signed DG user job DG user job submission with X509 proxy Download DG user X509 proxy Pilot Job Private worker WN Site Computing Resource Pilot Job submitted as EGEE Job DG user job deployment, status, results

25. 25 Application Respository service WMS (metascheduller) VOMS Server V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky User job Job Request User job submission Get Executable User X.509 Cert Pilot job submission Sched Gate Keeper CE queue SubCluster CE queue CE queue Pilot Job User X.509 proxy jLite : proxy init XWHEP->EGEE resource sharing (Pilot Job) jLite by Oleg Sukhoroslov http://code.google.com/p/jlite/ Server certificate Server public key Private worker Job Request User job Public worker Group worker Private worker Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)http://edms.cern.ch/document/855383 XW Coordinator (scheduller)

26. 26 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky EGEEEGEE XtremWeb User X509 proxy VOMS proxy Submits User Job with VOMS proxy Sends back Job Status and Results VOMS Server XtremWeb Server Submits mono-user Pilot Job with VOMS proxy Gives Pilot Job Status gLite WMS Computing Element Pushes Pilot job Mono-user Pilot Job Requests only 1 User Job Sends 1 User Job with same VOMS proxy User Job Gives Pilot Job Status Sends back results directly XtremWeb Bridge Requests User Jobs Sends User Jobs with VOMS proxy Manages User Job status XWHEP->EGEE resource sharing

27. 27 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky EDGeS Monitoring

28. 28 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky How to port an application from XWHEP to EGEE EGEE is made of Linux computing resourcesEGEE is made of Linux computing resources XWHEP applications must have a Linux versionXWHEP applications must have a Linux version There is no specific need to recompile nor relink binariesThere is no specific need to recompile nor relink binaries

29. 29 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky EGEE->XWHEP resource sharing

30. 30 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky Bridging EGEE and Desktop Grids Grid User  X509 proxy  VOMS Proxy  Submits Job with VOMS proxy Publishes available Resources  Pushes Job   Log  Log VOMS Admin Manages VO Site Admin Manages Site   Accesses Data with VOMS proxy  Accesses Data with VOMS proxy   Gives Job Status Gives Accounting and Auditing VOMS Server Accounting Logging & Bookkeeping Meta-scheduler (WMS) Site Computing Resource Site Storage Resource Grid Admin  Sends back Output Sandbox  Sends back Output Sandbox VOMS Proxy = X509 proxy with VOMS extensions EGEE (Service Grid) Well defined infrastructure: - authentication - authorization - logging and bookkeeping - quality of service (QoS) A strong effort of standardization Volunteer PCs Desktop Grids : Boinc, XWHEP Undefined infrastructure: - anonymity - volatility How to bridge ? - security - monitoring - logging DG User ? Server (scheduler)

31. 31 V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky gLite ensures security: authentication : ✓ X509 cert. and VOMS ext. authorization and confidentiality : ✓ VOMS server monitoring and bookeeping EGEE->XWHEP bridge : authentication, authorization, logging, monitoring XWHEP ensures security at different levels: authentication : ✓ SSL and X509 certificates authorization and confidentiality : ✓ user rights management ✓ user groups ✓ access rights volunteer nodes integrity : ✓ application repository ✓ sandboxing monitoring and bookeeping : ✓ all messages are logged

32. V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky 2 Job Processing :Job Processing : –Translation of EGEE jobs described in JDL to DG Work Unit –Correspondence between SG job Id to WU Id to track status –Gram plugin to connect DG to SG Data Management :Data Management : –The bridge should NOT allow DG volunteers to access directly SE –Instead data are copied to the bridge Security, Logging and AccountingSecurity, Logging and Accounting –EDGeS Application Repository (AR) –Multi-users environment reporting to LB Desktop Grid MonitoringDesktop Grid Monitoring –State of DGs should be reported to BDII EGEE -> XWHEP resource sharing

33. Authors : O. Lodygensky HUST : XtremWeb-HEP 6 What needed to be done when Porting DSP to EDGeS? 1.Develop DG version of application (XtremWeb version) 2.Test DG version Using the EDGeS Development DG 3.Have the application validated Validation is done by EDGeS Validation Team Outcome: validation document Validation includes tests via the EDGeS EGEE->DG test bridge Publish the application in EDGeS Application Repository EGEE users can select application from EDGeS AR DG Admins can download the application from EDGeS AR if they are ready to support it

34. V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky 7 EGEE EDGeS VO WMS service LCG-CE 1 WN LCG-CE N WN … EDGeS CE QM WS client XtremWeb server DB QM XW plugin WS Interface XtremWeb Worker pool P-GRADE Portal EDGeSAR EGEE -> XWHEP resource sharing

35. 35 Volunteer PCs XtremWeb User V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky QM WS client QM XW plugin WS Interface P-GRADE Portal EDGeSAR EGEE -> XtremWeb resource sharing VOMS Server LCG CE Meta-scheduler (WMS) EDGeS CE XW Coordinator (scheduler)

36. V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky 8

37. HUST : XtremWeb-HEP 9

38. V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky 10 Standard gLite tools usage

39. Authors : O. Lodygensky HUST : XtremWeb-HEP 11

40. Authors : O. Lodygensky HUST : XtremWeb-HEP 12

41. Authors : O. Lodygensky HUST : XtremWeb-HEP 13

42. Authors : O. Lodygensky HUST : XtremWeb-HEP 14

43. Authors : O. Lodygensky HUST : XtremWeb-HEP 15

44. Authors : O. Lodygensky HUST : XtremWeb-HEP 16

45. V 5.7.5-1 HUST : XtremWeb-HEP Authors : O. Lodygensky 17