Standard FTP and GridFTP protocols for international data transfer in Pamela Satellite Space Experiment R. Esposito 1, P. Mastroserio 1, F. Taurino 1,2,

Slides:



Advertisements
Similar presentations
Globus FTP Evaluation test Catania – 10/04/2001Antonio Forte – INFN Torino.
Advertisements

Istituto Nazionale di Fisica Nucleare Italy LAL - Orsay April Site Report – R.Gomezel Site Report Roberto Gomezel INFN - Trieste.
Distributed Data Processing
Current Testbed : 100 GE 2 sites (NERSC, ANL) with 3 nodes each. Each node with 4 x 10 GE NICs Measure various overheads from protocols and file sizes.
Chapter VI Data Communication: Delivering Information Anywhere and Anytime By: AP CHEN P. JOVER BSIT - III.
Data Grids Darshan R. Kapadia Gregor von Laszewski
GridFTP: File Transfer Protocol in Grid Computing Networks
GridFTP Introduction – Page 1Grid Forum 5 GridFTP Steve Tuecke Argonne National Laboratory.
The File Mover: An Efficient Data Transfer System for Grid Applications C. Anglano, M. Canonico Dipartimento di Informatica Universita' del Piemonte Orientale,
James 1:5 If any of you lacks wisdom, he should ask God, who gives generously to all without finding fault, and it will be given to him.
A Comparison of Layering and Stream Replication Video Multicast Schemes Taehyun Kim and Mostafa H. Ammar.
Protocols and the TCP/IP Suite Chapter 4 (Stallings Book)
Protocols and the TCP/IP Suite
Fair Sharing of MAC under TCP in Wireless Ad Hoc Networks Mario Gerla Computer Science Department University of California, Los Angeles Los Angeles, CA.
High Performance Cooperative Data Distribution [J. Rick Ramstetter, Stephen Jenks] [A scalable, parallel file distribution model conceptually based on.
Performance Comparison of Congested HTTP/2 Links Brian Card, CS /7/
Hands-On Microsoft Windows Server 2008 Chapter 8 Managing Windows Server 2008 Network Services.
Packet and Circuit Switching
Protocols and the TCP/IP Suite Chapter 4. Multilayer communication. A series of layers, each built upon the one below it. The purpose of each layer is.
Test Configuration for Control For the test configuration we used a VME based control system, constituted by a VME crate with a VMPC4a from Cetia (CPU.
Copyright © Texas Education Agency, All rights reserved.1 Web Technologies Web Administration.
Chapter 4. After completion of this chapter, you should be able to: Explain “what is the Internet? And how we connect to the Internet using an ISP. Explain.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
Computing on the Cloud Jason Detchevery March 4 th 2009.
G053 - Lecture 08 Hosting Websites Mr C Johnston ICT Teacher
Profiling Grid Data Transfer Protocols and Servers George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison USA.
Why GridFTP? l Performance u Parallel TCP streams, optimal TCP buffer u Non TCP protocol such as UDT u Order of magnitude greater l Cluster-to-cluster.
1 Next Few Classes Networking basics Protection & Security.
 Protocols used by network systems are not effective to distributed system  Special requirements are needed here.  They are in cases of: Transparency.
Computer Networks. Why Create Networks? Communication Communication technologies such as , sms, video-conference can be used Makes communication.
Moving Large Amounts of Data Rob Schuler University of Southern California.
File and Object Replication in Data Grids Chin-Yi Tsai.
INTERNET INFRASTRUCTURE. LANS WANS INTRANETS AND EXTRANETS WIRELESS NETWORKS.
HighSpeed TCP for High Bandwidth-Delay Product Networks Raj Kettimuthu.
Paper # – 2009 A Comparison of Heterogeneous Video Multicast schemes: Layered encoding or Stream Replication Authors: Taehyun Kim and Mostafa H.
DYNES Storage Infrastructure Artur Barczyk California Institute of Technology LHCOPN Meeting Geneva, October 07, 2010.
Delay Bound Rich Image Delivery over WLANs Shira S. Krishnan Georgia Tech, ECE Multimedia Communications Laboratory.
LEGS: A WSRF Service to Estimate Latency between Arbitrary Hosts on the Internet R.Vijayprasanth 1, R. Kavithaa 2,3 and Raj Kettimuthu 2,3 1 Coimbatore.
Data Management and Transfer in High-Performance Computational Grid Environments B. Allcock, J. Bester, J. Bresnahan, A. L. Chervenak, I. Foster, C. Kesselman,
CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej.
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
BASIC NETWORK PROTOCOLS AND THEIR FUNCTIONS Created by: Ghadeer H. Abosaeed June 23,2012.
A GRID solution for Gravitational Waves Signal Analysis from Coalescing Binaries: preliminary algorithms and tests F. Acernese 1,2, F. Barone 2,3, R. De.
August 23, 2001ITCom2001 Proxy Caching Mechanisms with Video Quality Adjustment Masahiro Sasabe Graduate School of Engineering Science Osaka University.
PPSP BAR BOF meeting 74th IETF – San Francisco, CA, USA March, 2009 P2P Streaming Protocol (PPSP) Requirements Ning Zong,Huawei Technologies Yunfei Zhang,China.
A Grid Approach to Geographically Distributed Data Analysis for Virgo F. Barone, M. de Rosa, R. De Rosa, R. Esposito, P. Mastroserio, L. Milano, F. Taurino,
A Silvio Pardi on behalf of the SuperB Collaboration a INFN-Napoli -Campus di M.S.Angelo Via Cinthia– 80126, Napoli, Italy CHEP12 – New York – USA – May.
ALCF Argonne Leadership Computing Facility GridFTP Roadmap Bill Allcock (on behalf of the GridFTP team) Argonne National Laboratory.
Super Computing 2000 DOE SCIENCE ON THE GRID Storage Resource Management For the Earth Science Grid Scientific Data Management Research Group NERSC, LBNL.
IS Infrastructure Managing Infrastructure and Services Copyright © 2016 Curt Hill.
1.3 ON ENHANCING GridFTP AND GPFS PERFORMANCES A. Cavalli, C. Ciocca, L. dell’Agnello, T. Ferrari, D. Gregori, B. Martelli, A. Prosperini, P. Ricci, E.
Globus Data Storage Interface (DSI) - Enabling Easy Access to Grid Datasets Raj Kettimuthu, ANL and U. Chicago DIALOGUE Workshop August 2, 2005.
Data Centers and Cloud Computing 1. 2 Data Centers 3.
Week-7 (Lecture-1) Internet Protocols: TCP/IP, Client Server: Protocol: Specification in networking and Communications It defines the procedures to be.
FILE TRANSFER SPEEDS OVER HTTP AND FTP Yibiao Li 06/01/2009 Christmas Meeting 2008/09.
SURENDRA INSTITUTE OF ENGINEERING & MANAGEMENT PRESENTED BY : Md. Mubarak Hussain DEPT-CSE ROLL
GGF 17 - May, 11th 2006 FI-RG: Firewall Issues Overview Document update and discussion The “Firewall Issues Overview” document.
DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories DART Workshop on Infrastructure Chief Investigator: Dr. Asad.
Protocols and the TCP/IP Suite
Lab A: Planning an Installation
Pasquale Migliozzi INFN Napoli
Study course: “Computing clusters, grids and clouds” Andrey Y. Shevel
Chapter 16: Distributed System Structures
Protocols and the TCP/IP Suite
ECEN “Internet Protocols and Modeling”
File Transfer Issues with TCP Acceleration with FileCatalyst
Beyond FTP & hard drives: Accelerating LAN file transfers
Protocols and the TCP/IP Suite
Distributed Systems Bina Ramamurthy 4/22/2019 B.Ramamurthy.
- Measurement of traffic volume -
Presentation transcript:

Standard FTP and GridFTP protocols for international data transfer in Pamela Satellite Space Experiment R. Esposito 1, P. Mastroserio 1, F. Taurino 1,2, G. Tortone 1 1 INFN, Sezione di Napoli, Italy 2 INFM, Unità di Napoli, Italy Introduction Physics applications often involve large amounts of data and/or computing and often require secure resource sharing across organizational boundaries, and are thus not easily handled by today's Internet and Web infrastructures. Pamela (a Payload Antimatter Matter Exploration and Light-nuclei Astrophysics), is a permanent magnet core facility that allows the investigation of the cosmic radiation. It will be put in an elliptical orbit at an altitude between 300 and 600 Km, on board of the Resurs DK1 Russian satellite in the fall of the year Up to 20 GByte per day of data will be transmitted from the satellite to the earth; every day these data will be transferred to all sites of the collaboration. The Napoli INFN Computing Center and the Pamela Teams of the Universities of Rome, Trieste and Stockholm have performed several network tests in order to make a comparison between Standard FTP and GridFTP protocols. Network test using FTP FTP is the most commonly used protocol for data transfer on the Internet, and the most likely candidate for meeting the grid’s needs. The Pamela Satellite Space Experiment has promoted same network test in order to decide which protocol to use in transferring files among its different sites in Europe and Russia. A 1 GByte file has been transferred tens of times per night among the Universities of Napoli, Trieste and Stockholm. The University of Stockholm is connected to the Internet with a 2x100 Mbps link, the University of Napoli is connected with an ATM 32 Mbps link. The average time needed to transfer a 1 GByte file from Stockholm to Napoli has been of 1268 sec about. Interesting considerations can be done comparing the theoretical time needed to transfer a one gigabyte file from Stockholm to Napoli and the measured times in our tests. In Napoli, as mentioned above, there is an ATM 32 Mbps link to the Internet; using TCP protocol on ATM the maximum expected transmission speed is not more then 27  28 Mbps due to the overhead of the protocols: notwithstanding a 32 Mbps link to the Internet, the average speed of our transmission tests has been of 6.4 Mbps. Conclusions An interesting consideration is given by the behavior of the graphs: in the transmission times registered using GridFTP, the minimum and maximum transfer times differ no more then ten seconds from the average value as we have registered transmission times in a range that goes from 1100 to 1600 seconds using the Standard FTP. The good results obtained in these tests encourage the Pamela Computing Team to adopt GridFTP as standard tool for data distribution. Network test using GridFTP GridFTP is a selected subsets of the existing FTP protocol with new features. GridFTP supports automatic negotiation of TCP buffer sizes both for large files and large sets of small files. On wide-area links, using multiple TCP streams can improve aggregate bandwidth over using a single TCP stream; GridFTP supports parallel data transfer. In order to manage large data sets it is necessary to provide third-party control of transfers between storage servers. GridFTP provides this capability by adding GSSAPI security to the existing third-party transfer capability defined in the FTP standard. Many applications require transfer of only a portion of a file. Transferring the entire file could be too expensive. GridFTP will support partial file transfer. The Napoli INFN Computing Center and the Pamela Teams of the Universities of Trieste and Stockholm have performed several network tests in order to make a comparison between Standard FTP and GridFTP protocols. GridFTP is able to saturate the bandwidth at disposal, the average speed transferring 1 GByte files from Stockholm to Napoli has been of 25.8 Mbps, much more then the 6.4 Mbps using the Standard FTP. Standard Ftp average transmission time: 1268 sec Grid Ftp average transmission time: 307 sec Grid Ftp average transmission speed: 25.8 Mbps Standard Ftp average transmission speed: 6.4 Mbps Stockholm – Napoli - Trieste: 1 GByte transmission file Stockholm – Napoli: 1 GByte transmission time Stockholm – Napoli: 1 GByte transmission speed CHEP03 - March 24-28, La Jolla, California