1. SAN DIEGO SUPERCOMPUTER CENTER By: Roman Olschanowsky roman2u@sdsc.edu Scommands Tutorial

2. SAN DIEGO SUPERCOMPUTER CENTER Scommands Command line access to the SRB Download then compile from http://www.sdsc.edu/srb/index.php/Scommands (make sure you get non-encrypted client only, as well as correct version that matches server version) http://www.sdsc.edu/srb/index.php/Scommands Login to a machine with Scommand binaries via ssh to a *nix machine Win32 binaries from command window

3. SAN DIEGO SUPERCOMPUTER CENTER Scommand Features Command line interface -> SCRIPTING Available for all of the most popular UNIX flavors and DOS S-commands are the most flexible and powerful of the clients They are the fastest, and most reliable They are multithreaded for big gains in data flow They are great for scripts, perl wrappers, batch jobs, etc… Installed man pages via “man [Scommand]” man Sput /utilities/man/man1

4. SAN DIEGO SUPERCOMPUTER CENTER Common Scommands (69 total) Sinit Senv Spwd Sls Scd Sget Sput Ssh Scp Smv (logical) Sphymove (physical) Srm Smkdir Srmdir Serror Schmod Don’t forget to Sexit!

5. SAN DIEGO SUPERCOMPUTER CENTER Scommand Notes Shelp Gives list of commands with brief summary “[Scommand] ” gives usage info (usually) or try –h flag Sinit – establishes a session Senv – displays connection information Spwd – display current working directory Sexit – ends session

6. SAN DIEGO SUPERCOMPUTER CENTER SRB Connection parameters Verify/Create ~/.srb/.MdasEnv Verify/Create ~/.srb/.MdasAuth OR Set environment variables

7. SAN DIEGO SUPERCOMPUTER CENTER ~/.srb/.MdasEnv File mdasCollectionHome '/home/roman.sdsc' srbUser 'roman' mdasDomainName 'sdsc' AUTH_SCHEME 'ENCRYPT1' srbHost ‘srb.sdsc.edu' srbPort ‘7321' defaultResource ‘sfs-tape-tgd'

8. SAN DIEGO SUPERCOMPUTER CENTER Environment Variables #!/bin/sh export mdasCollectionName="/home/du0.npaci" export srbHost=“srb.sdsc.edu" export srbPort=“7321" export srbUser="du0" export mdasDomainName="npaci" export srbAuth="2006SRBclass" export defaultResource=“sfs-tape-tgd" export AUTH_SCHEME="ENCRYPT1"

9. SAN DIEGO SUPERCOMPUTER CENTER ~/.srb/.MdasAuth 2006SRBclass (in SRB version 3.3.1 or higher) $ Sauth Enter your current SRB password: Successfully wrote ~/.srb/.srbAuthFile Remove (no longer needed) ~/.srb/.MdasAuth?:y.$CS2/j6A$"8

10. SAN DIEGO SUPERCOMPUTER CENTER Other authentication methods AUTH_SCHEME: 'ENCRYPT1' – random message encrypted with your password between clients & servers. 'GSI_AUTH' - Use the Globus GSI authentication scheme. 'GSI_DELEGATE' - Use the GSI Delegation (proxy) certificate for authentication. The advantage is that this certificate can be passed from server to server whereby the user's identity continues to be maintained across servers and across zones. This scheme solves the cross zone authentication issues. 'GSI_SECURE_COMM' - Use the GSI authentication scheme and use the GSI I/O library for all socket communication between client and server.

11. SAN DIEGO SUPERCOMPUTER CENTER Extra env vars for GSI auth SRB: (.MdasEnv file or env vars) AUTH_SCHEME ‘GSI_AUTH’ SERVER_DN ‘/C=US/O=NPACI/OU=SDSC/UID=srb/CN=Storage Resource Broker/Email=srb@sdsc.edu’ GLOBUS: X509_USER_PROXY="/home/du0/du0.proxy“ GLOBUS_LOCATION="/usr/local/apps/nmi-2.1" GLOBUS_INSTALL_PATH="/usr/local/apps/nmi-2.1" LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/apps/nmi-2.1/lib" PATH="$PATH:/usr/local/apps/nmi-2.1/bin"

12. SAN DIEGO SUPERCOMPUTER CENTER Step by Step (do inQ tutorial first) 1.Connect 1.First!, we need to be on a UNIX machine 2.ssh to du?@oly.sdsc.edu 3.cd.srb 4.“ls –al”, Notice the.MdasEnv.MdasAuth files 5.edit your.MdasEnv file, making it correct for your assigned du? SRB account mdasCollectionHome '/home/du?.npaci' srbUser 'du?' 6.view your.MdasAuth file 7.Sinit -v 8.“ls –al” notice the new.MdasEnv.?????? file, this file is associated with the current shell and saves your SRB session state (Like your current working dir).

13. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 2.Some basics 1.Sls 2.Smkdir test 3.Sls 4.Scd test 5.Spwd 6.cat.MdasEnv.????? mdasCollectionName '/home/du0.npaci/test' mdasCollectionHome '/home/du0.npaci' mdasDomainName 'npaci' srbUser 'du0' srbHost ‘srb.sdsc.edu' srbPort ‘7321' mcatZone ‘TGzone' defaultResource ‘sfs-tape-tgd‘ 7.Sexit, ls –al, notice it is now gone

14. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 3.Some basics 1.pwd, cd 2.Sinit 3.Spwd (notice you are back in your SRB home dir) 4.Senv 5.Shelp 6.Serror 7.Smkdir /home/Test 8.Serror -3220

15. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 4.Wait, how many are there? 1.Scd /home/Demo/SRB-Tutorial/files-2 2.Sls 3.How many Doc.txt files are there??? 4.Sls –l 5.How many are there now? 6.They are all the same file, the file has been replicated to different resources which is different than copied to different resources. 7.The number after the owner column is the replication number, followed by the physical resource.

16. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 5.Lets try one 1.Scd to go to your SRB home dir 2.Scd tutorial 3.Scp /home/Demo/SRB-Tutorial/comics/bc.gif. 4.Sls –l 5.There should be just one replica of bc.gif 6.Sreplicate –S sfs-tape-sdsc bc.gif 7.Sls –l 8.Is it replicated? 9.Why replicate? Fault tolerance, redundancy and faster access 10.SgetR will list all resources 11.Smv only does logical move (renaming or path change) 12.Sphymove will move the file physically

17. SAN DIEGO SUPERCOMPUTER CENTER Step by Step Best Practices for S-command uploads and downloads Low Latency Just upload, or recursive –r option High Latency If it’s one big file, use Sput or Sget with –M option If it’s a directory with lots of small files, use –b option

18. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 6.Download and Upload a directory 1.Scd /home/Demo/SRB-Tutorial 2.Sget –r comics 3.Scd 4.time Sput –r comics 5.time Sput –b comics comics2 6.time Sput –b comics comics3 7.Sls –l comics; Sls –l comics2 8.time Srm –r comics2 9.time Srm –rf comics3

19. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 7.SRB Trash system 1.Spwd 2.Add /trash to the front 3.Sls /trash/home/du0.npaci 4.That’s why it was so fast, it did not erase it, it moved it (logically). Great news! You can recover your data, just use Smv command to put data you want back. 5./trash/home/du0.npaci/comics2::2005-07-26-10.39.07 6.Adds timestamp to name to avoid conflicts, and to inform you when data was “trashed”

20. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 8.Some user defined meta data 1.Sufmeta 2.Sufmeta –Q Make = Ford 3.Sufmeta –c Contents Funny comics 4.Sufmeta Parts 4 comics/bc.gif 5.Sufmeta Rating “5 star” comics/bc.gif 6.Sufmeta comics/bc.gif

21. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 9.Put your mark on the tutorial 1.vi.txt 2.Contents of file “Picture of my car” 3.Scd /home/Demo/SRB-Tutorial/cars 4.Sput.txt 5.Sls –C.txt 6.Schmod r npaci groups.txt 7.Sls –C.txt 8.Sufmeta Make.txt 9.Sufmeta Model.txt 10.Sufmeta Year.txt

22. SAN DIEGO SUPERCOMPUTER CENTER Step by Step 9.Rodger….Signing off 1.Sexit Thanks! Questions? www.sdsc.edu/srb srb@sdsc.edu