netkit ftpd/ftp migration version 0.12 Part 5 Yufei 02/11/2011

‘rget’ Adjustable Memory Size – Configuration – Memory size negotiation Parallel File Transfer Direct IO preliminary result

‘rget’ is testable now

Configuration The app attribution, such as buffer size and port number, could be configured via two ways: – Configuration File – Runtime command (client side)

Configurable Items Both Client and Server – Circler buffer size (1024/1K/1KB, default 5M) – Circler buffer block number (default 10) – Parallel reader/sender/recver/writer number (default 1) Client – Active listening port area (default unlimited) Server – Server Listening Port (default 21)

Configuration File The app will search for the Configuration File as follows: 1)check system environment variables ‘SYSTEM_RCFTPRC/SYSTEM_RFTPDRC’ 2) if there is user rc file, the app will override the system configuration file I.Check user environment variables ‘RCFTPRC/RFTPDRC’ II.If ‘RCFTPRC/RFTPDRC’ is not exist, check ‘$HOME/.rcftprc(.rftpdrc)

Memory block size negotiation Because of rDma(direct, no buffer) and block (not tcp stream) transfer interface, the client and server would have to negotiate the memory block size before data transfer. Protocol (three-way handshake) – Client side, who launch data transfer request first, commit the buffer size to the peer side – Server side, consider the request, then give a result. YES or NO with a new size – Client side, start data transfer, or reject the new value.

example Client Buffersize = 5MB, write data to the data sink Server Buffersize = 1MB Hello, I want to use buffer size 5MB for each RDMA_WRITE Ok, you could write here. No, you should use buffer size 1MB at most, and write it here. I am writing with updated memory size Ok, done. The writing complete No, I reject this size, bye bye.

T-sender T-reader T-recver T-writer free list sender list free list writer list RDMA_READ RDMA_WRITE Parallel File Transfer

T-sender T-reader T-recver T-writer free list sender list free list writer list RDMA_READ RDMA_WRITE Make each block independent T-reader T-sender T-writer T-recver

Extend Buffer Block Data Strature next work request record rdma single SGE rdma_mr load CTL – File identification – offset

Problem Tasks dispatch – Simple stripe method for reader Reader i: read (i + n * readernum) series block – Create a new dispatcher, responsible for task generation to reader How to distinguish different files. Each block is independent, the file have to be opened several times. MPSCP from Sandia – m

Direct IO preliminary result O_DIRECT – times faster than standard IO (open/read) – Lustre support O_DIRECT Stream (fopen, fread) – 10% faster than standard IO

Q & A