1. 3 April 2001 Andrei.Chevel@pnpi.spb.ru Grid testbed Deploying and testing the Globus toolkit

2. Andrei.Chevel@pnpi.spb.ru3 April 2001 Projects for distributed computing in science  Main goal of all projects:  To realize more or less uniform access to distributed data, computing power, other resources  GriPhyN – Grid Physics Network (www.griphyn.org)  DataGrid (grid.web.cern.ch/grid)  PPDG - Particle Physics Data Grid (‘www.cacr.caltech.edu/ppdg’ and ‘ppdg.net’)

3. Andrei.Chevel@pnpi.spb.ru3 April 2001 General Middleware  The term middleware is used to describe separate products that serve as the glue between two applications. Middleware is sometimes called plumbing because it connects two sides of an application and passes data between them. In particular Object Request Broker is example for middleware (from ‘www.zdwebopedia.com’).  Now several slides from Globus report are to be shown (also might be seen ‘www.globus.org’).

4. Andrei.Chevel@pnpi.spb.ru3 April 2001 The Grid: Blueprint for a New Computing Infrastructure I. Foster, C. Kesselman (Eds), Morgan Kaufmann, 1999  Available July 1998; ISBN 1-55860-475-8  22 chapters by expert authors including Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others http://www.mkp.com/grids “A source book for the history of the future” -- Vint Cerf

5. Andrei.Chevel@pnpi.spb.ru3 April 2001 The Grid “Dependable, consistent, pervasive access to [high-end] resources”  Dependable: Can provide performance and functionality guarantees  Consistent: Uniform interfaces to a wide variety of resources  Pervasive: Ability to “plug in” from anywhere

6. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus Project Participants  Globus Project is a large community effort  Globus Toolkit core development  Argonne, USC/ISI, NCSA, SDSC  Globus Toolkit contributors  NASA, DOE ASCI DRM (SNL, LBNL, LLNL), Raytheon, and numerous others  Collaborators  University, lab, industrial, and international partners spanning many scientific and engineering disciplines  Active in Grid Forum  http://www.gridforum.org

7. Andrei.Chevel@pnpi.spb.ru3 April 2001 Grid Services Architecture Grid Fabric Layer Applications TransportMulticast InstrumentationControl interfacesQoS mechanisms Grid Services Layer InformationResource mgmt SecurityData accessFault detection... High-energy physics data analysis Regional climate studies Collaborative engineering Parameter studies On-line instrumentation Application Toolkit Layer High throughput Data- intensive Collab. design Remote viz Remote control

8. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus Approach  A toolkit and collection of services addressing key technical problems  Modular “bag of services” model  Not a vertically integrated solution  General infrastructure tools (aka middleware) that can be applied to many application domains  Inter-domain issues, rather than clustering  Integration of intra-domain solutions  Distinguish between local and global services

9. Andrei.Chevel@pnpi.spb.ru3 April 2001 Technical Focus & Approach  Enable incremental development of grid- enabled tools and applications  Model neutral: Support many programming models, languages, tools, and applications  Evolve in response to user requirements  Deploy toolkit on international-scale production grids and testbeds  Large-scale application development & testing  Information-rich environment  Basis for configuration and adaptation

10. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus Toolkit Grid Services  Security (GSI)  Resource management (GRAM)  Information services (MDS)  Remote file management (GASS)  Communication (I/O, Nexus)  Process monitoring (HBM)

11. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus Toolkit on ram0  It was intalled Globus toolkit 1.1.3;  Job submission, file transfer and part of information service were tested;  globus-job*;  globus-rcp;  grid-proxy*;  grid-info (partly).  Command description might be found on ‘www.globus.org/v1.1/programs

12. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus File Transfer  globus-rcp host-s:file-s host-d:file-d  globus-url-copy s-URL d-URL  ncftp in GSI environment  gsincftp (GSI version for ncftp) - package consists of several programs (client side)  gsiwuftp (GSI version for server wuftp);  comparison in between ftp and gsincftp is shown on the page http://home.fnal.gov/~muzaffar/Timing/#GSI-WUFTP  All testing was done with counter part cluster pcfarm.pnpi.spb.ru

13. Andrei.Chevel@pnpi.spb.ru3 April 2001 Example of session  ram0:/Users/shevel> grid-proxy-initEnter PEM pass phrase:......+++++..+++++  ram0:/Users/shevel> grid-proxy-info -all  subject : /O=Grid/O=Globus/OU=i2net.sunysb.edu/CN=Andrei E. Chevel/CN=proxy  issuer : /O=Grid/O=Globus/OU=i2net.sunysb.edu/CN=Andrei E. Chevel  type : fullstrength : 512 bitstimeleft : 11:59:50  D= /usr/local/globus/ncftp/bin  ram0:/Users/shevel> $D/gsincftpget pcfarm.pnpi.spb.ru. tickit.ps  tickit.ps: 175.10 kB 10.39kB/s  ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /bin/date -u ;echo $?  Tue Mar 27 19:24:26 UTC 2001  0  ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /bin/daterrr -u ;echo $?  GRAM Job submission failed because the executable does not exist (error code 5)  5  ram0:/Users/shevel> globus-job-run pcfarm.pnpi.spb.ru /usr/pbs/bin/qstat  Job id Name User Time Use S Queue  ---------------- ---------------- ---------------- -------- - -----  801.pcfarm xs256.bat kim 30:28:37 R L  802.pcfarm xs248.bat kim 16:17:55 R L

14. Andrei.Chevel@pnpi.spb.ru3 April 2001 Second Example  ram0:/Users/shevel> D=/usr/local/globus/ncftp/bin  ram0:/Users/shevel> $D/gsincftpget gsiftp://pcfarm.pnpi.spb.ru/tickit.ps  ncftpget: local file appears to be the same as the remote file, no transfer necessary.  ram0:/Users/shevel> rm tickit.ps  ram0:/Users/shevel> $D/gsincftpget gsiftp://pcfarm.pnpi.spb.ru/tickit.ps  tickit.ps: 175.10 kB 9.68 kB/s

15. Andrei.Chevel@pnpi.spb.ru3 April 2001 Main advantages (1)  General  in one toolkit (‘bag of tools’) several issues were addressed (security, proxy servers, intrgration with other existing products, using of all availabe resources, etc.)  it is freeware (GPL);  at least part of tools are working well enough;

16. Andrei.Chevel@pnpi.spb.ru3 April 2001 Main advantages(2)  Technical  You have no needs to be logged on remote server to perform separate commands or scripts.  You have no need to confirm every transaction with remote server.  In particular you may use all Globus command inside scripts without compromise of security.

17. Andrei.Chevel@pnpi.spb.ru3 April 2001 Globus middleware in HEP community  LHC collaborations (see ‘ http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/grid/’, ‘ http://cmsdoc.cern.ch/cms/grid/ ’)  GDMP - Grid Data Management Pilot (‘ http://cmsdoc.cern.ch/cms/grid/ ’)  the pilot was built on top of the Globus toolkit.

18. Andrei.Chevel@pnpi.spb.ru3 April 2001 Conclusion  GRID technology is in wide experimenting use now (Globus toolkit as well).  I think it is the time to try to use technical toolkit which is already available from several projects (Globus, GDMP).  To permit to use Globus toolkit it might be considered nearest future scenario  to deploy the Globus toolkit on 1-2 Linux boxes at RCF (rcas????)  to create RHIC/PHENIX CA (and DN).