1. Core and Framework DIRAC Workshop 29-31 October Marseille

2. Overview 29-31 OctoberDIRAC Workshop2  What is it?  DISET  Executors  Release tools  Other stuff  RFC proxies + EMI 1 + openssl 1.X+ SHA2

3. What is “Core and Framework”? 29-31 OctoberDIRAC Workshop3  DIRAC is a framework to build distributed computing systems  It provides a set of basic components  Configuration, Accounting, Monitoring, Logging…  No news on this front  Connected via a secure communication protocol  DISET

4. DISET 29-31 OctoberDIRAC Workshop4  DISET is the secure transport used to connect DIRAC components  Provides:  Authentication + Authorization  Remote Procedure Call (RPC)  File transfer  And since v6r5 it also provides:  Asynchronous messaging  Connection keep alive mechanism

5. Asynchronous messaging 29-31 OctoberDIRAC Workshop5 Remote Procedure CallAsynchronous messaging

6. Benefits of asynchronous messaging 29-31 OctoberDIRAC Workshop6  Just one SSL handshake, multiple requests  Server can push data to clients  Both ends can react on disconnect  Added a keep alive mechanism  Also used for RPC and file transfer  But lots of connections to a server can kill it…  So no async messages from jobs for now

7. DIRAC Agents and Services 29-31 OctoberDIRAC Workshop7  DIRAC architecture was based on services and agents  Services are passive components that wait for queries  Agents are components that actively monitor the state of DIRAC and react accordingly  Agents process tasks synchronously  Poll a resource for a state change, react, wait, repeat

8. What’s the problem? 29-31 OctoberDIRAC Workshop8  There’s dead time between a task enters the system and an Agent picks it up  Services receive task changes but they can’t push them to Agents  When there’s a high load, Agents will reduce their idle time, but still have to check if there’s something to do  Add load to the DB on a high load environment  Need to design Agent/DB to allow more than one Agent at the same time

9. Executor framework 9  Two new components:  Mind: Service-like component with access to the DB that pushes tasks.  Executors: Wait for their Mind to push tasks to it and send back the result. 29-31 OctoberDIRAC Workshop

10. Benefits 29-31 OctoberDIRAC Workshop10  Mind can reschedule tasks if an Executor disconnects  Executors are stateless  Any number of Executors can be connected to a Mind.  It will dispatch tasks between them.  Reduced load to the DB since no component polls it  No need to design the DB for concurrent access

11. Optimizers 29-31 OctoberDIRAC Workshop11

12. Code management 29-31 OctoberDIRAC Workshop12  Last user meeting we announced we were migrating to github  30 forks since then  1k issues  Huge success!!

13. Code management 29-31 OctoberDIRAC Workshop13  Tools for generating project releases and installations  Release project is a group of extensions DIRAC that are released together  DIRAC, VMDIRAC, Web…  An installation is a preset of installation arguments for a user community  Many communities have made their extensions  LHCb, CTA, Belle….

14. Other stuff 29-31 OctoberDIRAC Workshop14  The grid will migrate to RFC proxies instead of the current grid proxies  We’re almost ready.  (but we will be…)  EMI-1 had a bug in some components that prevents us from using it  Can’t handle openssl 1.0 generated keys and DIRAC uses openssl 1.0   Solved in EMI-2  Some Certification Authorities (CA) will start issuing SHA-2 certificates  We’re ready