gLite Information System Practicals Claudio Cherubino INFN Catania

Outline This presentation covers the following arguments: LCG Command Line Tool lcg-infosites lcg-info R-GMA Command Line Tool R-GMA Browser

LCG Command Line Tool

lcg-infosites - 1 closeSE ce se lfc all help This tool provides information related to Grid services: lcg-infosites --vo <your_vo> option –-is <your_bdii> • It’s mandatory to include the vo and the option. • The -- is option specifies the BDII you want to query. If not supplied, the BDII defined into the environment variable LCG_GFAL_INFOSYS will be used. List of options: closeSE Names of the CEs where the user’s VO is allowed to run together with their corresponding closest SEs ce Number of CPUs, running and waiting jobs and names of the CEs se SEs names together with the available and used space lfc Name of the lfc Catalog for the user’s VO all It groups all the features just described help Prints all the available options and exit

lcg-infosites - 2 To retrieve the status of all available CEs serving the GILDA VO: lcg-infosites –-vo gilda ce –-is grid004.ct.infn.it:2170 **************************************************************** These are the related data for gilda: (in terms of queues and CPUs) #CPU Free Total Jobs Running Waiting ComputingElement ---------------------------------------------------------- 6 6 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long 0 0 0 0 0 cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-long 10 10 0 0 0 dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long 6 6 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short 0 0 0 0 0 cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-short 10 10 0 0 0 dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short 16 16 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-long 10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-long 6 6 0 0 0 grid004.iucc.ac.il:2119/jobmanager-lcgpbs-infinite 16 16 0 0 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-short 10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-short 0 0 0 0 0 cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-infinite 10 10 0 0 0 dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite 2 2 0 0 0 gildace.oact.inaf.it:2119/jobmanager-lcgpbs-long 10 10 0 0 0 gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long 2 2 0 0 0 gildace.oact.inaf.it:2119/jobmanager-lcgpbs-short 10 10 0 0 0 gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short 16 16 2 2 0 grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite 10 10 0 0 0 grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-infinite 2 2 0 0 0 gildace.oact.inaf.it:2119/jobmanager-lcgpbs-infinite 10 10 4 4 0 gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite

lcg-infosites - 3 To retrieve the status of all available SEs serving the GILDA VO: lcg-infosites –-vo gilda se –-is grid004.ct.infn.it:2170 ************************************************************** These are the related data for gilda: (in terms of SE) Avail Space(Kb) Used Space(Kb) Type SEs ---------------------------------------------------------- 912356296 51120 grid005.iucc.ac.il 143511344 2509092 cn02.be.itu.edu.tr 68930000 6200000 egee016.cnaf.infn.it 26700000 8010000 lxcde07.pd.infn.it 140742384 146535224 grid009.ct.infn.it 16160000000 410000000 aliserv6.ct.infn.it 108150932 3033064 gildase.oact.inaf.it 28071068 3088712 testbed005.cnaf.infn.it 348586904 447804 gilda-se-01.pd.infn.it

lcg-infosites - 4 The closeSE feature shows the name of the CEs together with their corresponding closest SEs: lcg-infosites –-vo gilda closeSE Name of the CE: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long Name of the close SE: grid005.iucc.ac.il Name of the CE: cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-long Name of the close SE: cn02.be.itu.edu.tr Name of the CE: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short Name of the CE: cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-short Name of the CE: grid010.ct.infn.it:2119/jobmanager-lcgpbs-long Name of the close SE: grid009.ct.infn.it ......

lcg-info This program allows the user to query, in a deeper way, the BDII. It requires the environmental variable LCG_GFAL_INFOSYS to be set to the value of the BDII to be queried, e.g. grid004.ct.infn.it:2170 Prints the list of the CEs or SEs satisfying a given query along with a list of specified attributes. The query syntax is like this: attr1 op1 valueN, ... attrN opN valueN where attrN is an attribute name. op is =, >= or <=, and the cuts are ANDed. A combination of attribute is supported. The cuts are comma-separated and spaces are not allowed.

lcg-info options --list-attrs Prints a list of attributes that can be queried. --list-ce Lists the CEs which satisfy a query, or all the CEs if no query is given. --list-se Lists the SEs which satisfy a query, or all the SEs if no query is given. --query Restricts the output to the CEs (SEs) which satisfy the given query. --bdii Allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error. --sed Print the output in a "sed-friendly" format. --attrs Specifies the attributes whose values should be printed. --vo Restricts the output to CEs or SEs where the given VO is authorized. Mandatory when VO-dependent attributes are queried upon.

lcg-info examples - 1 To get the list of all the “queryable” attributes: lcg-info --list-attrs Attribute name Glue object class Glue attribute name MaxTime GlueCE GlueCEPolicyMaxWallClockTime CEStatus GlueCE GlueCEStateStatus TotalJobs GlueCE GlueCEStateTotalJobs CEVOs GlueCE GlueCEAccessControlBaseRule TotalCPUs GlueCE GlueCEInfoTotalCPUs FreeCPUs GlueCE GlueCEStateFreeCPUs CE GlueCE GlueCEUniqueID WaitingJobs GlueCE GlueCEStateWaitingJobs RunningJobs GlueCE GlueCEStateRunningJobs CloseCE GlueCESEBindGroup GlueCESEBindGroupCEUniqueID CloseSE GlueCESEBindGroup GlueCESEBindGroupSEUniqueID SEVOs GlueSA GlueSAAccessControlBaseRule UsedSpace GlueSA GlueSAStateUsedSpace AvailableSpace GlueSA GlueSAStateAvailableSpace Type GlueSE GlueSEType SE GlueSE GlueSEUniqueID Protocol GlueSEAccessProtocol GlueSEAccessProtocolType ArchType GlueSL GlueSLArchitectureType Processor GlueSubCluster GlueHostProcessorModel OS GlueSubCluster GlueHostOperatingSystemName Cluster GlueSubCluster GlueSubClusterUniqueID Tag GlueSubCluster GlueHostApplicationSoftwareRunTimeEnvironment Memory GlueSubCluster GlueHostMainMemoryRAMSize OSVersion GlueSubCluster GlueHostOperatingSystemRelease

lcg-info examples - 2 To discover which software is supported by the CEs: lcg-info --list-ce --attrs Tag - CE: gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite PBS INFN PADOVA LCG-2 LCG-2_1_0 LCG-2_1_1 LCG-2_2_0 LCG-2_3_0 LCG-2_3_1 LCG-2_4_0 LCG-2_6_0 R-GMA AFS CMS-1.1.0 VIRGO-1.0 ATLAS-6.0.4 GATE-1.0.0-3 LHCb-1.1.1 IDL-5.4 CMSIM-125 ALICE-4.01.00 ALIEN-1.32.14 POVRAY-3.5 DEMTOOLS-1.0 CMKIN-VALID CMKIN-1.1.0 CMSIM-VALID CSOUND-4.13 MPICH CMS-OSCAR-2.4.5 LHCb_dbase_common-v3r1 GEANT4-6 VLC-0.7.2 EGEODE-1.0 RASTER3D SCILAB-2.6 G95-3.5.0 MAGIC-6.19 CODESA3D-1.0 OpenFOAM-1.2 SI00MeanPerCPU=450 SF00MeanPerCPU=300 VO-gilda-GKS05

lcg-info examples - 3 To list only some attributes of the CEs: lcg-info --list-ce --attrs ‘FreeCPUs,TotalJobs’ - CE: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-long - FreeCPUs 6 - TotalJobs 0 - CE: cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-long - FreeCPUs 0 - CE: dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long - FreeCPUs 10 - CE: grid004.iucc.ac.il:2119/jobmanager-lcgpbs-short - CE: cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-short

lcg-info examples - 4 To get the list of all the CEs which satisfy a given query: lcg-info --list-ce --attrs ‘FreeCPUs,TotalJobs’ --query ‘TotalCPUs=10’ - CE: dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-long - FreeCPUs 10 - TotalJobs 0 - CE: dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short - CE: grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-long - CE: grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-short - CE: dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-infinite

R-GMA Command Line Tool

R-GMA Command Line Tool To Start the R-GMA command line tool run the following command (you need a valid proxy): $RGMA_HOME/bin/rgma On startup you should receive the following message: Welcome to the R-GMA virtual database for Virtual Organisations. ================================================================ Your local R-GMA server is: https://rgmasrv.ct.infn.it:8443/R-GMA You are connected to the following R-GMA Registry services: https://rgmasrv.ct.infn.it:8443/R-GMA/RegistryServlet You are connected to the following R-GMA Schema service: https://rgmasrv.ct.infn.it:8443/R-GMA/SchemaServlet Type "help" for a list of commands.

Entering Command Commands are entered by typing at the rgma> prompt and hitting ‘enter’ to execute the command. A history of the commands executed can be accessed using the Up and Down arrow keys. To search a command from history use CTRL-R and type the first few letters of the command to recall. Command autocompletion is supported (use Tab when you have partly entered a command). The # character can be used to add comments.

General Commands General Commands help Display general help information. help <command> Display help for a specific command. help examples Display list of example commands. exit or quit Exit from R-GMA command line interface. show tables Display the name of all tables existing in the Schema. describe <tablename> Show all information about the structure of a table.

Querying Data RGMA uses the standard SQL SELECT syntax to query data, e.g.: rgma> SELECT * FROM GlueCE In R-GMA there are three basic types of query, and they behave differently: LATEST Queries: return only the most recent tuple for each primary key. HISTORY Queries: return all historical tuples for each primary key. CONTINUOUS Queries: return tuples continuously as they are inserted.

Changing query type The type of query can be changed using the SET QUERY command as follow: rgma> SET QUERY LATEST rgma> SET QUERY HISTORY rgma> SET QUERY CONTINUOUS The current query type can be displayed using: rgma> SHOW QUERY

Maximum AGE of tuples The maximum age of tuples to return can also be controlled. By default there is no limit. To limit the age of latest or historical tuples use the SET MAXAGE command, e.g.: rgma> SET MAXAGE 120 The current maximum tuple age can be displayed: rgma> SHOW MAXAGE To disable the maximum age, set it to none: rgma> SET MAXAGE none The MAXAGE parameter has a different effect with continuous query. If it is specified, the query will initially return a history of matching tuples up to the specified maximum age.

Query Timeout R-GMA query also have a TIMEOUT property. For a latest or history query the timeout exists to prevent a problem (e.g. network failure) from stopping the query from completing. For a continuous query, timeout indicates how long the query will continue to return new tuples. Default timeout is 1 minute and it can be changed using the SET TIMEOUT command, e.g.: rgma> SET TIMEOUT 180 The current timeout can be displayed using: rgma> SHOW TIMEOUT

Producer Data is inserted into R-GMA using a Producer component which handles the INSERT statement. Using the command line tool you may work with one producer at a time. If you change its properties then a new producer will be created. The SQL INSERT statement may be used to add data to the system: rgma> INSERT INTO Table VALUES (’a’, ’b’) Producers may be configured to answer the three different types of query. To set the producer type you can use the SET PRODUCER command, e.g.: rgma> SET PRODUCER LATEST The current producer type can be displayed using: rgma> SHOW PRODUCER

Retention periods For a producer that can answer latest or history queries, tuples must be deleted after some time interval. The producer will return tuples to Latest queries provided that they are younger then the Latest Retention Period (LRP) . Similarly, there is a History Retention Period (HRP) for History queries. These retention periods can be controlled using: SET PRODUCER latestretentionperiod 600 SET PRODUCER historyretentionperiod 2 hours To show the current HRP/LRP use the commands SHOW PRODUCER HRP and SHOW PRODUCER LRP.

Secondary Producer (1) A Secondary producer collects data from individual producers and makes it available with its own Producer component. To instruct the secondary producer to consume from table MyTable: rgma> SECONDARYPRODUCER MyTable The secondary producer may be configured to answer latest and/or history queries: rgma> SET SECONDARYPRODUCER latest By default the secondary producer can only answer latest queries. The current secondary producer type can be displayed using: rgma> SHOW SECONDARYPRODUCER

Secondary Producer (2) If the secondary producer can answer history queries, it has an associated History Retention Period (HRP), as for a primary producer. This period can be controlled using: rgma> SET SECONDARYPRODUCER historyretentionperiod 1 days To show the current secondary producer HRP, use the following command: rgma> SHOW SECONDARYPRODUCER HRP Secondary producers do not have a Latest Retention Period as this is a property of each tuple and is inherited from the individual producers.

Other commands The SHOW PROPERTIES command shows a summary of all properties of the current session: To show a list of all R-GMA producers that produce the table MyTable: rgma> SHOW PRODUCERS OF MyTable

Batch Mode The command line tool can be used in batch mode in three ways: rgma –c <command> Executes <command> and exits. The –c option may be specified more than once. rgma –f <file> Executes commands in <file> sequentially then exits. Each line should contain one command. Embedded in a shell script

Example

R-GMA Browser

Table description

R-GMA Browser as Consumer

Query from R-GMA Browser

Query Results

Further information LCG-2 User Guide https://edms.cern.ch/file/454439//LCG-2-UserGuide.html R-GMA: Relational Grid Monitoring Architecture http://www.r-gma.org R-GMA command line tool http://hepunx.rl.ac.uk/egee/jra1-uk/glite-r1/command-line.pdf R-GMA Browser Home Page https://rgmasrv.ct.infn.it:8443/R-GMA/