Getting Started on Emerald Research Computing Group

its.unc.edu 2  What is Emerald?  Logging into Emerald  File manipulation on Emerald  Submitting jobs on Emerald  Interactive programs on Emerald Course Outline

its.unc.edu 3 Help Documentation  Getting Started on Emerald General overview of Emerald for range of users  Short Course – Getting Started on Emerald Detailed notes for beginning Emerald users

its.unc.edu 4 What is Emerald?  352-processor Linux Cluster Maintained by Research Computing Group  Appropriate for all users regardless of expertise level  Other Servers: Cedar/Cypress (128-processor SGI/Altix) Topsail (4160-processor Dell Linux Cluster)  Mass Storage Account access

its.unc.edu 5 Advantages of Using Emerald  High performance  Large capacity  Parallel processing  Many available software packages  Variety of compiling options

its.unc.edu 6 Emerald Linux Cluster

its.unc.edu 7 Distributed vs. Shared Memory Shared memory - single address space. All processors have access to a pool of shared memory. (examples: Yatta, Cedar/Cypress) Methods of memory access : Bus and Crossbar Distributed memory - each processor has it’s own local memory. Must do message passing to exchange data between processors. (examples: Emerald, Topsail) MEMORY BUS CPU MMMM NETWORK

its.unc.edu 8 Logging Into Emerald  UNIX/Linux/OSX ssh  Windows: SSH Secure Shell Setting up a Profile for Emerald Forwarding X11 packets

its.unc.edu 9 Home and Work Directories on Emerald  Home Directory /afs/isis/home/m/y/my_onyen/ 250MB quota ~/private/ Files backed up daily [ ~/OldFiles ] Space quota/usage in Home Directory:  fs lq  Work Directory /netscr/my_onyen/ No space limit but periodically cleaned

its.unc.edu 10 File Manipulation Commands  SSH Secure File Transfer  Copy files cp command cp /afs/isis/depts/atn/rcg/example_code/Gaussian/water.com /netscr/my_onyen/. cp /afs/isis/depts/atn/rcg/example_code/sas/test.sas /netscr/my_onyen/. cp -r  cp -r ~/private/TestDirectory.  Move files mv commands  mv ~/private/testfile.txt.

its.unc.edu 11 File Manipulation Commands  Tar archive To create a tar file  tar –cvzf TestDirectory.tgz./ To see a tar file’s table of content  tar –tvzf TestDirectory.tgz To untar a tar file  tar –xvzf TestDirectory.tgz

its.unc.edu 12 Submitting Jobs: LSF and Packages  LSF (Load Sharing Facility) Fairly distribute compute nodes among users 60 processor per user limit  Packages ipm commands  ipm add (ipm a)  ipm remove (ipm r)  ipm query (ipm q) Available packages  ications.php ications.php

its.unc.edu 13 Details of LSF Submission host LIM Batch API Master host MLIM MBD Execution host SBD Child SBD LIM RES User job LIM – Load Information Manager MLIM – Master LIM MBD – Master Batch Daemon SBD – Slave Batch Daemon RES – Remote Execution Server queue Load information other hosts other hosts bsub app

its.unc.edu 14 Submitting Jobs: bsub Command  bsub command  All files must be in /netscr/my_onyen/  bsub [- bsub_opts] executable [-exec_opts]  Queues – bqueues command week idle  bsub –o bsub –o out.%J

its.unc.edu 15 Submitting Jobs: Following Job Progress  bjobs bjobs –l JobID Shows current status of job  bhist bhist –l JobID  bkill bkill JobID Ends job prematurely  bfree

its.unc.edu 16 Submitting Jobs: Specialty Scripts  bsas bsub -q week -R blade sas program.sas bsas test.sas  bmatlab bsub -q week -R blade matlab -nodisplay - nojvm -nosplash program.m -logfile program.log bmatlab test.m

its.unc.edu 17 Compiling on Emerald Compilers – FORTRAN 77/90/95 – C/C++ Parallel Computing – OpenMP – MPI (MPICH, LAM/MPI, MPICH-GM)

its.unc.edu 18 Compiling Details on Emerald CompilerPackage nameCommand Intelintel_fortran, intel_CCifort, icc Portland Grouppgipgf77, pgf90,pgcc,pgCC Absoftprofortranf77, f90 GNUgccg77,gcc, g++

its.unc.edu 19 Compiling Details on Emerald  Add a compiler into your working environment ipm add package_name  Compile a code command code.f –o executable  Run executable on a compute node using the bsub command bsub –q week –R blade executable

its.unc.edu 20 Submitting Jobs: Job Output  Output sent to  bsub –o Output saved in working directory  bsub –u Output sent to specified address

its.unc.edu 21 Interactive Jobs: Setup  X-Windows Linux/OSX  X11 client Windows  X-Win32  Offered on UNC Software Acquisition site  Port forwarding on SSH Secure Shell  Setting up a session on X-Win32

its.unc.edu 22 Interactive Jobs: Submission  –Ip bsub –q int –R blade –Ip sas bsub –q int –R blade –Ip gv  Specialty Scripts xsas xstata

its.unc.edu 23 Contacting Research Computing  For assistance with Emerald, please contact the Research Computing Group: Phone: HELP Submit help ticket at