Data Transfers in the ALCF Robert Scott Technical Support Analyst Argonne Leadership Computing Facility

Transferring Data To/From ALCF  sftp and scp (for “small” transfers) –If you must use scp, eureka, and in the future tukey are better systems to scp to/from. All paths will be the same as they are on our other systems. –Eureka is also a better host for compressing and uncompressing large file archives  GridFTP (for large transfers) –Other site must accept our CA –ssh / cryptocard access available –For more information on Grid FTP please go here:  Globus Online (for large transfers) Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 2

Getting Started with Globus Online  You’ll need a Globus Online account –Go here to signup: –Globus Connect: to transfer files to and from your local machine (laptop) –SSH key or crypto card to access an ALCF endpoint –Go here to initiate your transfer: Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 3

What is Globus Online?  Reliable File Transfer –Easy: Simple signup, login and use – and minimized intervention –Fast: GO can move large filesets in hours –Secure: No Need to worry about security configs or certs, and one- time passwords work –Research-focused: Globus Online is the only file transfer service with the scientific researcher in mind  No IT required –No client software installation –Works with existing GridFTP servers –Globus Connect solves “last mile problem” for local machines –Consolidated support and troubleshooting Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 4

Transferring Data To/From ALCF using GO  Globus Online (for large transfers) –Globus Online addresses the challenges faced by researchers in moving, sharing, and archiving large volumes of data among distributed sites. –ALCF BG/P endpoints: alcf#dtn_intrepid, alcf#dtn_surveyor, alcf#dtn –Check if your laboratory, university or research center has already an endpoint. –Globus Connect to transfer files to and from your local machine. – Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 5

Globus Online Demo  Let’s start the Demo…. Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 6

Table of BG/ Q File Systems on ALCF Resources Argonne Leadership Computing Facility – NameSystemTypePathProductionBacked-upUsers vesta-homeVestaGPFS/homeYesNoGeneral Use vesta- scratch (Intensive job output, large files) VestaGPFS/projectsYesNoI/O mira-home (general use) MiraGPFS/homeYesNo— mira-fs0 (Intensive job output, large files) MiraGPFS/projectsYesNoI/O NOTE: all “filesystems” are also available via the /gpfs/$filesystem_name path on these systems.

Thank You!  Questions? Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy 8