A Sneak Peak of What’s New in Globus GridFTP John Bresnahan Michael Link Raj Kettimuthu (Presenting) Argonne National Laboratory and The University of Chicago

GridFTP l A secure, robust, fast, efficient, standards based, widely accepted data transfer protocol l We supply a reference implementation: u Server u Client tools (globus-url-copy) u Development Libraries l Independent implementations interoperate u Fermi Lab has a home grown server that work with ours l Lots of people have developed clients independent of the Globus Project

GridFTP l Two channel protocol like FTP l Control Channel u Communication link (TCP) over which commands and responses flow u Low bandwidth; encrypted and integrity protected by default l Data Channel u Communication link(s) over which the actual data of interest flows u High Bandwidth; authenticated by default; encryption and integrity protection optional

GridFTP DPISPI DPISPI CPI

Striping l GridFTP offers a powerful feature called striped transfers (cluster-to-cluster transfers)

Topics for discussion l Performance enhancement u GridFTP over UDT l Ease of Use enhancements u GridFTP over SSH u GridFTP Where there’s FTP l Resource Management in GridFTP l Future directions

GridFTP over UDT l UDT is an application-level data transport protocol that uses UDP to transfer data l Implement its own reliability and congestion control mechanisms l Achieves good performance on high-bandwidth, high- delay networks where TCP has significant limitations l GridFTP uses Globus XIO interface to invoke network I/O operations

GridFTP over UDT l XIO framework presents a standard open/close/read/write interface to many different protocol implementations u including TCP, UDP, HTTP -- and now UDT l The protocol implementations are called drivers. u A driver can be dynamically loaded and stacked by any Globus XIO application. l Created an XIO driver for UDT reference implementation l Enabled GridFTP to use it as an alternate transport protocol

GridFTP over UDT Argonne to NZ Throughput in Mbit/s Argonne to LA Throughput in Mbit/s Iperf – 1 stream Iperf – 8 streams GridFTP mem TCP – 1 stream GridFTP mem TCP – 8 streams GridFTP disk TCP – 1 stream GridFTP disk TCP – 8 streams GridFTP mem UDT GridFTP disk UDT UDT mem UDT disk

Alternate security mechanism l GridFTP traditionally uses GSI for establishing secure connections l In some situations, preferable to use SSH security mechanism l Leverages the fact that an SSH client can remotely execute programs by forming a secure connection with SSHD

GridFTP over SSH l sshd acts similar to inetd l control channel is routed over ssh u globus-url-copy popens ssh u ssh authenicates with sshd u ssh/sshd remotely starts the GridFTP server as user u stdin/out becomes the control channel

SSHFTP Interactions sshd CPI GridFTP Server 2811 Port 22 ROOT USER ssh Stdin/out

GridFTP Where there’s FTP (GWFTP) l GridFTP has been in existence for some time and has proven to be quite robust and useful l Only few GridFTP clients available l FTP has innumerable clients l GUI Clients? l Windows Clients?

GWFTP l GWFTP - created to leverage the FTP clients l A proxy between FTP clients and GridFTP servers l Not secure from client to proxy u Run on a trusted net ( ) u Data channel routed or direct u If 3pt it is direct and secure u If 2 party must route through proxy, or be insecure

GWFTP (3pt) DPISPI DPISPI Your Client FTP 959 (not secure) GSI Credential GSI Delegated Credential gwtftp GSI Credential

GWFTP (2pt routed) DPI SPI Your Client FTP 959 (not secure) GSI Credential gwtftp GSI Credential DPI

GWFTP (2pt direct) DPI SPI Your Client FTP 959 (not secure) No Security gwtftp GSI Credential DPI

Resource management l Fork/Exec is safer service model u sandboxes leaks/segfaults/security/etc u If 1 session dies service exists l Transient state u We need permanent & shared state between sessions

GFork Server Host GFork Server GridFTP Plugin GridFTP Server Instance Fork GridFTP Server Instance GridFTP Server Instance State Sharing Link Client Inherited Links Control Channel Connections Client

Dynamic Backends l Dynamic list of available backends (DPIs) l Frontend (SPI) listens for registration u Backends register (and timeout) u Select backend(s) to use for a transfer l Backend failure is not system failure l Resources can be provisioned to suit load

Dynamic Backends Frontend Host GFork Server GridFTP Plugin Frontend Instance Fork Lookup available backend Registration Control Connection Backend Host GFork Server GridFTP Plugin Backend Instance Fork

Future directions l Resource Properties u GridFTP server expose state via resource properties l Server load l Connection limits u Act as a WS-MDS provider l Firewall traversal u Simultaneous open u Capability to make use of dynamic firewall port opening