1. Overview of TeraGrid Resources and Usage Selim Kalayci Florida International University 07/14/2009 Note: Slides are compiled from various TeraGrid Documentations

2. What is the TeraGrid?

3. TGUP (TeraGrid User Portal)

4. Accessing TeraGrid User Portal

6. Portal Overview

7. Portal: MyTeraGrid

8. Portal: Resources

9. Portal: Documentation

10. Accessing Resources

11. Web-based SSO via Portal

13. SSO (Non-Portal) from a TeraGrid Resource

14. Example - SSH to tg-login.ncsa.teragrid.org or another resource that you have SSH access. - grid-proxy-info - myproxy-logon -l username -grid-proxy-info -gsissh tg-login.purdue.teragrid.org

15. TeraGrid Resources http://www.teragrid.org/userinfo/hardware/ – Sorted by site – Sorted by machine type http://portal.teragrid.org – My TeraGrid -> Accounts – Resources http://www.ncsa.uiuc.edu/UserInfo/Resource s/ http://www.ncsa.uiuc.edu/UserInfo/Resource s/ – URLs here contain detailed user documentation

16. Moving data to/from TeraGrid systems Sftp clients from your office to TeraGrid – Command line sftp on Linux – GUI sftp clients GSI-SSHTerm sftp button http://portal.teragrid.org – Resources -> File Manager [beta] High speed gridftp between TeraGrid systems – globus-url-copy – Uberftp

17. File Transfers: Small ( <100 MB) Files

18. Large ( >100 MB) File Transfers: globus-url-copy The globus-url-copy client program is a GridFTP client for transferring files from the command line. Usage: – globus-url-copy where or is of the format: if local file, file: if remote file, gsiftp:// /

19. Example – Two Party Transfer Logon to NCSA Abe Cluster. Create a large file on NCSA Abe Cluster: – dd bs=100MB count=1 if=/dev/zero of=testfile Copy this file to Purdue Steele Cluster: – globus-url-copy –vb file:///u/ac/username/testfile gsiftp://tg- steele.purdue.teragrid.org:2811/autohome/u108/userna me/

20. Example – Third Party Transfer Logon to NCSA Abe Cluster Copy the testfile at Purdue Steele Cluster to NCAR Frost Cluster – globus-url-copy -vb gsiftp://tg- steele.purdue.teragrid.org:2811/autohome/u108/username/test file gsiftp://gridftp.frost.ncar.teragrid.org:2811//home/username/ GridFTP server addresses for each site are listed at: – http://www.teragrid.org/userinfo/data/transfer_locat ion.php#deployment http://www.teragrid.org/userinfo/data/transfer_locat ion.php#deployment

21. Optimized Data Transfer with globus- url-copy Using large TCP windows – globus-url-copy –vb -tcp-bs 1048576 file:///u/ac/username/testfile gsiftp://tg- steele.purdue.teragrid.org:2811/autohome/u108/username/ Using large memory buffers – globus-url-copy –vb -bs 1048576 file:///u/ac/username/testfile gsiftp://tg- steele.purdue.teragrid.org:2811/autohome/u108/username/ Using multiple parallel streams – globus-url-copy –vb –p 4 file:///u/ac/username/testfile gsiftp://tg- steele.purdue.teragrid.org:2811/autohome/u108/username/

22. Large ( >100 MB) File Transfers: UberFTP UberFTP is an interactive GridFTP file transfer client. Opens a session with a remote host, within which files may be transfered and directories and files may be manipulated. Requires GSI authentication. Hands-On: – Login to NCSA Abe – uberftp – open tg-steele.rcac.purdue.edu – parallel 2 – tcpbuf 8388608 – ls/lls/put/get/…

23. Imaginations unbound Data movement tips To move a collection of small files, make an archive and move it instead of moving the files individually – tar – zip For high bandwidth links and moderate file or archive size, do not compress, it's usually faster to just move the data [compression is a time waster] For low bandwidth links, compression is usually a time saver – tar z or j options for compression – zip – ssh -C, sftp -C

24. Permanent Storage at NCSA The larger TeraGrid sites provide persistent high-capacity storage Details vary by site, consult local site documentation for specifics Refer to: – http://www.teragrid.org/userinfo/data/storage.php http://www.teragrid.org/userinfo/data/storage.php for detailed information about different quotas, policies, and tools (such as SRB, HPSS) at each site.

25. Managing Your Environment: Softenv

26. Softenv

27. Managing Your Environment: Modules * Try at tg-steele.rcac.purdue.edu

28. Softenv and Modules: Which do I use? Rule of thumb: go with the default on a given machine – When you login for the first time issue ‘softenv’ and ‘module list’ commands – In general, only one should be active by default: go with that one – If you have questions or run into any problems contact help@teragrid.org help@teragrid.org

29. 29 Grid Job Management using Globus Common WS interface to schedulers – Unix, Condor, LSF, PBS, SGE, … More generally: interface for process execution management – Lay down execution environment – Stage data – Monitor & manage lifecycle – Kill it, clean up

30. 30 Grid Job Management Goals Provide a service to securely: Create an environment for a job Stage files to/from environment Cause execution of job process(es) – Via various local resource managers Monitor execution Signal important state changes to client Enable client access to output files – Streaming access during execution

31. 31 GRAM GRAM: Globus Resource Allocation and Management GRAM is a Globus Toolkit component – For Grid job management GRAM is a unifying remote interface to Resource Managers – Yet preserves local site security/control Remote credential management File staging via RFT and GridFTP

32. 32 A Simple Example First, login to queenbee.loni-lsu.teragrid.orgqueenbee.loni-lsu.teragrid.org Command example: % globusrun-ws -submit -c /bin/date Submitting job...Done. Job ID: uuid:002a6ab8-6036-11d9-bae6-0002a5ad41e5 Termination time: 01/07/2005 22:55 GMT Current job state: Active Current job state: CleanUp Current job state: Done Destroying job...Done. A successful submission will create a new ManagedJob resource with its own unique EPR for messaging Use – o option to create the EPR file % globusrun-ws -submit –o job.epr -c /bin/date

33. 33 A Simple Example(2) To see the output, use –s (stream) option % globusrun-ws -submit –s -c /bin/date Termination time: 06/14/2007 18:07 GMT Current job state: Active Current job state: CleanUp-Hold Wed Jun 13 14:07:54 EDT 2007 Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done. If you want to send the output to a file, use –so option % globusrun-ws -submit –s –so job.out -c /bin/date … % cat job.out Wed Jun 13 14:07:54 EDT 2007

34. 34 A Simple Example(3) Submitting your job to different schedulers – Fork % globusrun-ws -submit -Ft Fork -s -c /bin/date (Actually, the default is Fork. So, you can skip it in this case.) – SGE % globusrun-ws -submit -Ft PBS-s -c /bin/date Submitting to a remote site % globusrun-ws -submit -F tg- login.frost.ncar.teragrid.org -c /bin/date

35. 35 Batch Job Submissions % globusrun-ws -submit -batch -o job_epr -c /bin/sleep 50 Submitting job...Done. Job ID: uuid:f9544174-60c5-11d9-97e3-0002a5ad41e5 Termination time: 01/08/2005 16:05 GMT % globusrun-ws -status -j job_epr Current job state: Active % globusrun-ws -status -j job_epr Current job state: Done % globusrun-ws -kill -j job_epr Requesting original job description...Done. Destroying job...Done.

36. 36 Resource Specification Language (RSL) RSL is the language used by the clients to submit a job. All job submission parameters are described in RSL, including the executable file and arguments. You can specify the type and capabilities of resources to execute your job. You can also coordinate Stage-in and Stage-out operations through RSL.

37. 37 Submitting a job through RSL Command: % globusrun-ws -submit -f touch.xml Contents of touch.xml file: /bin/touch touched_it

38. Security - Basics

39. How to get Help First, try searching the Knowledge Base or other Documentation If that doesn’t help, submit a ticket – Send an email to help@teragrid.orghelp@teragrid.org – Use the TeraGrid User Portal ‘Consulting’ tab Can also call TeraGrid Help Desk 24/7: 1-866-907-2383

40. Submitting a Ticket

41. More Info TeraGrid Resource User Guides – http:www.teragrid.org/userinfo/hardware/resources.ph p http:www.teragrid.org/userinfo/hardware/resources.ph p File Transfers and Data Management on TeraGrid – http:www.teragrid.org/userinfo/data http:www.teragrid.org/userinfo/data More Training – https://portal.teragrid.org/gridsphere/gridsphere?cid=o nlinetraining https://portal.teragrid.org/gridsphere/gridsphere?cid=o nlinetraining