Information System (BDII) M. Jouvin (LAL-Orsay) jouvin@lal.in2p3.fr Grid Administration Training LAL, Orsay, September 2008, 15-19

Information System - M. Jouvin Agenda Why an Information System ? BDII Architecture and Contents User view of BDII Conclusions Information System - M. Jouvin 02/05/2019

Why an Information System ? Grid services need to have a « real-time » (dynamic) view of resources configuration, status and usage WMS needs to know about available CEs, the VOs they accept, the current usage and the free resources… Lcg_utils tools (data management) need a lot of configuration parameters to contact SEs, locate file catalogs for a VO… Without IS, user would have to enter all these site dependent parameters… or all the sites use the same configuration!!! Users want know the resources open to their VO, their shares on these resources… IS required to implement features like resource discovery IS is a critical underlying service in an heterogeneous, multi-communities, grid (not an EGEE specific need) Scalability is a major issue in a large production grid like EGEE Response time to IS queries must be negligible compared to overall user command response time Information System - M. Jouvin 02/05/2019

Information System - M. Jouvin BDII Architecture BDII : EGEE (gLite) implementation of an IS Globus is using another one (MDS) Based on a hierarchy of cooperation LDAP servers BDII is a thin layer on top of LDAP servers to allow permanent updates in background without impact on user queries Updating information about all resources may be long, must not break IS content consistency Global grid view is based on aggregation of lower level of information. 3 types of BDIIs : top-level, site, resource Site BDII (1/site) collects information published by resource BDII on each resource (CE, SE, WMS…) operated by the site Top-level BDII queries all site BDIIs and aggregates the information from all sites Several top-level BDIIs may (must) co-exist and all have the same information, modulo refresh time Information System - M. Jouvin 02/05/2019

BDII Architecture Abbreviations: Each site can run a top BDII. It collects the information coming from the site-BDII % ldapsearch –x –h <hostname -p 2170 –b “o=grid” At each site, a site BDII collects the information given by the resource_BDII % ldapsearch –x –h <hostname> -p 2170 –b “mds-vo-name=<name>,o=grid” Resource BDII run on CEs and SEs at each site and report dynamic and static information –b “mds-vo-name=resource,o=grid” Abbreviations: BDII: Berkeley DataBase Information Index Information System - M. Jouvin 02/05/2019

Information System - M. Jouvin BDII Contents Information organized by site Top level of an object DN is mds-vo-name=local,o=grid Each object DN in BDII has site name in mds-vo-name E.g. …,mds-vo-name=GRIF,mds-vo-name=local,o=grid Information about resources operated by site Configuration parameters for all resources: port, endpoint… Resource status: production, test, unaivalable… Resource current usage: mainly for CE (number of jobs queued, running per queue) and SE (space used, reserved available) Information about VO resources available at site Tags (arbitrary values) published by the VO to represent VO resources available like SW versions… Information published according a schema know as GLUE schema https://forge.gridforum.org/sf/projects/glue-wg Information System - M. Jouvin 02/05/2019

Information System - M. Jouvin User View of BDII Users generally don’t see the BDII directly Used by services like WMS to process user requests (e.g. select a site to execute a job based on user requirements) Used by commands like lcg_utils (data management) to retrieve configuration parameters required to contact a SE, build a file path according to a VO name… One user-oriented command : lcg-infosites Allow a user to discover the resources it has access to Exemple to list all CEs usable by VO dteam: lcg-infosites --vo dteam ce Sophisticated requests possible with ldapsearch command Powerful but complex syntax, not intended for normal end-users ldapsearch -LLL -x -h topbdii.grif.fr:2170 -b mds-vo-name=GRIF-LLR,mds-vo-name=GRIF,mds-vo- name=local,o=grid '(&(objectclass=GlueSA)(GlueSAType=permanent)(GlueSALocalID=cms))' GlueSAStateAvailableSpace GlueSAStateUsedSpace Information System - M. Jouvin 02/05/2019

Information System - M. Jouvin Conclusions BDII is a critical service operated by sites, even if generally not used directly by users Main usage is by gLite services and commands Site BDII does not require a powerful machine but must run on a lightly loaded machine as response time is critical If BDII is not responsive enough, site will temporally disappear from grid To access grid resources, users must have access to a top-level BDII Not necessary to run one per site, generally at least one per country A top-level BDII require a powerful machine : update process consumes resources, requires significant memory (2GB min) Better to use an external, well-known, BDII than run a site- specific one on an inappropriate machine Information System - M. Jouvin 02/05/2019

Information System - M. Jouvin Useful Links Man pages for lcg-infosites and ldapsearch commands GRIF gLite tutorial: https://trac.lal.in2p3.fr/GridSupport/wiki/Tutorial/SystemInfo Information System - M. Jouvin 02/05/2019