Hepix LAL April 2001 An alternative to ftp : bbftp Gilles Farrache In2p3 Computing Center

Hepix LAL April 2001 Gilles Farrache 2 History  In 1999, the members of the French Babar collaboration decided to import from SLAC to Lyon a big part of the SLAC objectivity database  First approach : via tapes  Needs a lot of manpower  Problems with cartridge compatibility  Delay between data production at SLAC and their availability in Lyon quite long

Hepix LAL April 2001 Gilles Farrache 3 History  Second approach : via the network using standard ftp  Security problems :  Username and password flying unencrypted  Username and password present in.netrc file  Performances problems ( Maximum bandwidth available at this time between SLAC and Lyon was 6 Mbps ) :  Throughput performance on one ftp session : 80 Kbytes/s  ~Seven hours for one two-gigabytes file

Hepix LAL April 2001 Gilles Farrache 4 History  First idea to improve ftp performances : Start n concurrent ftp Total throughput reaches 3.5 Mbps  But you still need no network outage for seven hours  Second idea to improve ftp performances : Cut the file in n parts Start n concurrent ftp (one for each part) Rebuild the file  But you double the disk space usage at each endpoint

Hepix LAL April 2001 Gilles Farrache 5 Goals  Improve performances on single file transfer  Optimize disk space usage  Improve security

Hepix LAL April 2001 Gilles Farrache 6 Performance improvements  Use several TCP streams to transfer a file  Use large TCP windows (RFC 1323)  Compress the data on the fly (using gzip algorithm)

Hepix LAL April 2001 Gilles Farrache 7 Performance improvements  The client opens a control connection to the server  Exchange control information  Each side starts as many processes as requested, each process will send a slice of the file.

Hepix LAL April 2001 Gilles Farrache 8 Performance improvements 100 MB file compress uncompress bbftp client process bbftp server process 100 MB file

Hepix LAL April 2001 Gilles Farrache 9 Performance improvements  Compression plus parallel streams lead to throughput between 7 and 20 Mbps on the 6 Mbps link (often 12 Mbps with a standard Babar file)  But with higher speed links (34 Mbps), compression often reduces performance due to lack of CPU (perhaps we have to use another algorithm)

Hepix LAL April 2001 Gilles Farrache 10 Security improvements  Standard bbftp connection method Use RSA encryption methods to transfer username and password  SSH bbftp connection method (Tim Adye’s idea) Use ssh to remotely start the server

Hepix LAL April 2001 Gilles Farrache 11 Security improvements  Standard bbftp connection method client server connection RSA key pair generation send public key Crypt user name and password using the public key send encrypted data Decrypt user name and password using the private key OK or BAD

Hepix LAL April 2001 Gilles Farrache 12 Security improvements  Standard bbftp connection method without password Start a bbftpcd client which will keep the username and password in memory Start a bbftpc client which will talk with bbftpcd and will give it the command to execute Limitation : bbftpcd and bbftpc have to run on the same computer

Hepix LAL April 2001 Gilles Farrache 13 Security improvements  SSH bbftp connection method client SSHD start the bbftpd process Start a ssh client like ssh -x -l username “bbftpd -s” host SSH clientSSH daemon bbftpd process Use the encrypted connection as control connection

Hepix LAL April 2001 Gilles Farrache 14 Security improvements  SSH bbftp connection method without password If the sshd server side allows the RSA authentication, a connection without any password is possible

Hepix LAL April 2001 Gilles Farrache 15 BBFTP state of the art  Current version patch03  Possibility to play with : TCP window size Read (Write) buffer size Compression Automatic directory creation File mode File access time Number of streams Remote and local umask.bbftprc file RFIO

Hepix LAL April 2001 Gilles Farrache 16 BBFTP state of the art  Current commands : cd get put lcd mkdir mget mput

Hepix LAL April 2001 Gilles Farrache 17 BBFTP state of the art  Control file example : 1 setnbstream 20 2 setremoteumask setoption nocreatedir 4 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 5 setoption createdir 6 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 7 setnbstream 5 8 setrecvwinsize setoption gzip 10 put /home/babar/jon/f2 /home/babar/bbrdist/newfiles/f2

Hepix LAL April 2001 Gilles Farrache 18 BBFTP state of the art  BBFTP home page :  BBFTP discussion list : Archives on :  Current experiments or sites using it : PHENIX ALICE (project)