1 Reliable File Transfer Service Ravi K Madduri Argonne National Laboratory, University of Chicago

2 What is RFT ? l WS-RF compliant Fault-tolerant, High- performance data transfer service u Soft state. u Notifications/Query l Reliability on top of high performance provided by GridFTP. u Fire and Forget. u Integrated Automatic Failure Recovery. l Network level failures. l System level failures etc. u Essentially a Data transfer scheduler with FIFO as a Queue Policy.

3 What is RFT ? RFT Service RFT Client SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server

4 What is RFT ? RFT Service SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server

5 What is RFT ? RFT Client SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server

6 What is RFT ? RFT Service RFT Client SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link GridFTP Server

7 What is RFT ? RFT Service RFT Client SOAP Messages Notifications (Optional) Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server

8 What is RFT ? RFT Service RFT Client SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server

9 RFT Implementation Details l Provides following operations u start() u getStatus(source_url) u getStatusSet(from,offset) u Cancel() l Provides following Resource Properties u RequestStatusType with faults. u OverallStatusType with faults. u Other Aggregation RPs for total bytes, total files etc. l Uses Database to store and retrieve the state.

10 New in GT4.1 l A C client to submit transfers to RFT service l Better performance than the command-line java client l Client API in java to make it easy to write custom clients l Bug fixes

11 Who is Using RFT ? l Two types of users u Embedded (As part of other services) u Stand-Alone. l GRAM u Stage-in, Stage-out, Clean-up l Data Replication Service l TeraGrid u Tgcp l LiGO (Proposed)

12 Future Work l Standardization effort at GGF-DMIS l Add new operations u Pause() u Resume() u addTransfers(List of transfers) u getStatus(StatusType) l Merge create() and start() into one operation l Add support for more data scheduler like functionality in RFT

13 Future Work (continued..) l Priority based transfers l Add support for different transfer protocols l Add new resource properties with predictions for estimated time of finish of transfers l Support for partial file transfers l Better scalability and performance.

14 Q & A