Extending Globus to support Multicast Transmission Karl Jeacle

Slides:



Advertisements
Similar presentations
Push Technology Humie Leung Annabelle Huo. Introduction Push technology is a set of technologies used to send information to a client without the client.
Advertisements

Chapter 17 Networking Patricia Roy Manatee Community College, Venice, FL ©2008, Prentice Hall Operating Systems: Internals and Design Principles, 6/E William.
CCNA – Network Fundamentals
© 2007 Cisco Systems, Inc. All rights reserved.Cisco Public ITE PC v4.0 Chapter 1 1 OSI Transport Layer Network Fundamentals – Chapter 4.
UDP & TCP Where would we be without them!. UDP User Datagram Protocol.
CS3505 The Internet and Info Hiway transport layer protocols : TCP/UDP.
TRANSPORT LAYER  Session multiplexing  Segmentation  Flow control (TCP)  Connection-oriented (TCP)  Reliability (TCP)
Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle.
Distributed components
Socket Programming.
Semester Copyright USM EEE442 Computer Networks Introduction: Protocols En. Mohd Nazri Mahmud MPhil (Cambridge, UK) BEng (Essex, UK)
Computer Network Architecture and Programming
William Stallings Data and Computer Communications 7 th Edition (Selected slides used for lectures at Bina Nusantara University) Transport Layer.
William Stallings Data and Computer Communications 7 th Edition Chapter 2 Protocols and Architecture.
CS335 Networking & Network Administration Tuesday, April 20, 2010.
Communication Protocols III Tenth Meeting. Connections in TCP A wants to send to B. What is the packet next move? A travels through hub and bridge to.
COE 342: Data & Computer Communications (T042) Dr. Marwan Abu-Amara Chapter 2: Protocols and Architecture.
What Can IP Do? Deliver datagrams to hosts – The IP address in a datagram header identify a host IP treats a computer as an endpoint of communication Best.
TRANSPORT LAYER T.Najah Al-Subaie Kingdom of Saudi Arabia Prince Norah bint Abdul Rahman University College of Computer Since and Information System NET331.
1 Chapter Internetworking Part 4 (Transport Protocols, UDP and TCP, Protocol Port Numbers)
Chapter 17 Networking Dave Bremer Otago Polytechnic, N.Z. ©2008, Prentice Hall Operating Systems: Internals and Design Principles, 6/E William Stallings.
Lecture 2 TCP/IP Protocol Suite Reference: TCP/IP Protocol Suite, 4 th Edition (chapter 2) 1.
Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago.
Protocols and the TCP/IP Suite
LWIP TCP/IP Stack 김백규.
LWIP TCP/IP Stack 김백규.
Networks – Network Architecture Network architecture is specification of design principles (including data formats and procedures) for creating a network.
William Stallings Data and Computer Communications 7 th Edition Data Communications and Networks Overview Protocols and Architecture.
Final Year Project Presentation by Daire O’Neill 4EE.
TCP : Transmission Control Protocol Computer Network System Sirak Kaewjamnong.
ICOM 6115©Manuel Rodriguez-Martinez ICOM 6115 – Computer Networks and the WWW Manuel Rodriguez-Martinez, Ph.D. Lecture 26.
FALL 2005CSI 4118 – UNIVERSITY OF OTTAWA1 Part 2.5 Internetworking Chapter 25 (Transport Protocols, UDP and TCP, Protocol Port Numbers)
User Datagram Protocol (UDP) Chapter 11. Know TCP/IP transfers datagrams around Forwarded based on destination’s IP address Forwarded based on destination’s.
Chapter 6-2 the TCP/IP Layers. The four layers of the TCP/IP model are listed in Table 6-2. The layers are The four layers of the TCP/IP model are listed.
Transport Layer3-1 Chapter 3: Transport Layer Our goals: r understand principles behind transport layer services: m multiplexing/demultipl exing m reliable.
1. I NTRODUCTION TO N ETWORKS Network programming is surprisingly easy in Java ◦ Most of the classes relevant to network programming are in the java.net.
1 Networking Chapter Distributed Capabilities Communications architectures –Software that supports a group of networked computers Network operating.
UDT as an Alternative Transport Protocol for GridFTP Raj Kettimuthu Argonne National Laboratory The University of Chicago.
Fundamentals of Proxying. Proxy Server Fundamentals  Proxy simply means acting on someone other’s behalf  A Proxy acts on behalf of the client or user.
Chapter 2 Applications and Layered Architectures Sockets.
Reliable high-speed Grid data delivery using IP Multicast Karl Jeacle & Jon Crowcroft
Institute of Technology Sligo - Dept of Computing Chapter 12 The Transport Layer.
The Client-Server Model And the Socket API. Client-Server (1) The datagram service does not require cooperation between the peer applications but such.
A Multicast Transport Driver for Globus XIO Karl Jeacle
Reading TCP/IP Protocol. Training target: Read the following reading materials and use the reading skills mentioned in the passages above. You may also.
The Globus eXtensible Input/Output System (XIO): A protocol independent IO system for the Grid Bill Allcock, John Bresnahan, Raj Kettimuthu and Joe Link.
TCP-XM “Multicast Transport for Grid Computing” Karl Jeacle
Lect 8 Tahani al jehain. Types of attack Remote code execution: occurs when an attacker exploits a software and runs a program that the user does not.
A Sneak Peak of What’s New in Globus GridFTP John Bresnahan Michael Link Raj Kettimuthu (Presenting) Argonne National Laboratory and The University of.
TCP/IP1 Address Resolution Protocol Internet uses IP address to recognize a computer. But IP address needs to be translated to physical address (NIC).
1 Network Communications A Brief Introduction. 2 Network Communications.
11 CS716 Advanced Computer Networks By Dr. Amir Qayyum.
1 Chapter 24 Internetworking Part 4 (Transport Protocols, UDP and TCP, Protocol Port Numbers)
Data and Computer Communications Chapter 2 – Protocol Architecture, TCP/IP, and Internet-Based Applications.
Fast Retransmit For sliding windows flow control we waited for a timer to expire before beginning retransmission of a packet TCP uses an additional mechanism.
Transport Protocols Relates to Lab 5. An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
Process-to-Process Delivery, TCP and UDP protocols
PART 5 Transport Layer Computer Networks.
Transport Protocols Relates to Lab 5. An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
Transport Protocols Relates to Lab 5. An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
Transport Protocols Relates to Lab 5. An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
Process-to-Process Delivery:
CS4470 Computer Networking Protocols
TCP-XM: Unicast-enabled Reliable Multicast
Transport Protocols An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
TRANSMISSION CONTROL PROTOCOL
Process-to-Process Delivery: UDP, TCP
Transport Protocols Relates to Lab 5. An overview of the transport protocols of the TCP/IP protocol suite. Also, a short discussion of UDP.
STATEL an easy way to transfer data
Transport Layer 9/22/2019.
Presentation transcript:

Extending Globus to support Multicast Transmission Karl Jeacle

All Hands Meeting 2004 Rationale  Would like to achieve high-speed bulk data delivery to multiple sites  Multicasting would make sense  Existing multicast research has focused on sending to a large number of receivers  But Grid is an applied example where sending to a moderate number of receivers would be extremely beneficial

All Hands Meeting 2004 Multicast availability  Deployment is a problem!  Protocols have been defined and implemented  Valid concerns about scalability; much FUD  “chicken & egg” means limited coverage  Clouds of native multicast  But can’t reach all destinations via multicast  So applications abandon in favour of unicast  What if we could multicast when possible…  …but fall back to unicast when necessary?

All Hands Meeting 2004 Multicast TCP?  TCP  “single reliable stream between two hosts”  Multicast TCP  “multiple reliable streams from one to n hosts”  May seem a little odd, but there is precedent…  TCP-XMO – Liang & Cheriton  M-TCP – Mysore & Varghese  M/TCP – Visoottiviseth et al  PRMP – Barcellos et al  SCE – Talpade & Ammar

All Hands Meeting 2004 ACK implosion

All Hands Meeting 2004 Building Multicast TCP  Want to test multicast/unicast TCP approach  But new protocol == kernel change  Widespread test deployment difficult  Build new TCP-like engine  Encapsulate packets in UDP  Run in userspace  Performance is sacrificed…  …but widespread testing now possible

All Hands Meeting 2004 TCP/IP/UDP/IP Sending Application TCP IP UDP IP Receiving Application TCP IP UDP IP If natively implemented test deployment

All Hands Meeting 2004 TCP engine  Where does initial TCP come from?  Could use BSD or Linux  Extracting from kernel could be problematic  More compact alternative  lwIP = Lightweight IP  Small but fully RFC-compliant TCP/IP stack  lwIP + multicast extensions = “TCP-XM”

All Hands Meeting 2004 TCP-XM overview  Primarily aimed at push applications  Sender initiated – advance knowledge of receivers  Opens sessions to n destination hosts simultaneously  Unicast is used when multicast not available  Options headers used to exchange multicast info  API changes  Sender incorporates multiple destination and group addresses  Receiver requires no changes  TCP friendly, by definition

All Hands Meeting 2004 TCP SYN SYNACK ACK FIN ACK DATA SenderReceiver

All Hands Meeting 2004 TCP-XM Sender Receiver 1 Receiver 2 Receiver 3

All Hands Meeting 2004 TCP-XM connection  Connection  User connects to multiple unicast destinations  Multiple TCP PCBs created  Independent 3-way handshakes take place  SSM or random ASM group address allocated  (if not specified in advance by user/application)  Group address sent as TCP option  Ability to multicast depends on TCP option

All Hands Meeting 2004 TCP-XM transmission  Data transfer  Data replicated/enqueued on all send queues  PCB variables dictate transmission mode  Data packets are multicast (if possible)  Retransmissions are unicast  Auto fall back/forward to unicast/multicast  Close  Connections closed as per unicast TCP

All Hands Meeting 2004 Fall back / fall forward  TCP-XM principle  “Multicast if possible, unicast when necessary”  Initial transmission mode is group unicast  Ensures successful initial data transfer  Fall forward to multicast on positive feedback  Typically after ~75K unicast data  Fall back to unicast on repeated mcast failure

All Hands Meeting 2004 TCP-XM reception  Receiver  No API-level changes  Normal TCP listen  Auto-IGMP join on TCP-XM connect  Accepts data on both unicast/multicast ports  tcp_input() accepts:  packets addressed to existing unicast destination…  …but now also those addressed to multicast group  Tracks how last n segs received (u/m)

All Hands Meeting 2004 Grid multicast?  How can multicast be used in Grid environment?  TCP-XM is new multicast-capable protocol  Globus is de-facto Grid middleware  Would like TCP-XM support in Globus…

All Hands Meeting 2004 Globus XIO  eXtensible Input Output library  Allows “i/o plugins” to Globus  API  Single POSIX-like API / set of semantics  Simple open/close/read/write API  Driver abstraction  Hides protocol details / Allows for extensibility  Stack of 1 transport & n transform drivers  Drivers can be selected at runtime

All Hands Meeting 2004 XIO architecture

All Hands Meeting 2004 XIO implementation

All Hands Meeting 2004 XIO/XM driver specifics  Two important XIO data structures 1.Handle  Returned to user when XIO framework ready  Used for all open/close/read/write calls  lwIP netconn connection structure used 2.Attribute  Used to set XIO driver-specific parameters…  … and TCP-XM protocol-specific options  List of destination addresses

All Hands Meeting 2004 XIO code example // init stack globus_xio_stack_init(&stack, NULL); // load drivers onto stack globus_xio_driver_load("tcpxm", &txdriver); globus_xio_stack_push_driver(stack, txdriver); // init attributes globus_xio_attr_init(&attr); globus_xio_attr_cntl(attr, txdriver, GLOBUS_XIO_TCPXM_SET_REMOTE_HOSTS, hosts, numhosts); // create handle globus_xio_handle_create(&handle, stack); // send data globus_xio_open(&handle, NULL, target); globus_xio_write(handle, "hello\n", 6, 1, &nbytes, NULL); globus_xio_close(handle, NULL);

All Hands Meeting 2004 One-to-many issues  Stack assumes one-to-one connections  XIO user interface requires modification  Needs support for one-to-many protocols  Minimal user API changes  Framework changes more significant  GSI is one-to-one  Authentication with peer on connection setup  But cannot authenticate with n peers  Need some form of “GSI-M”

All Hands Meeting 2004 LAN/WAN tests  LAN  Computer Laboratory  Mix of CPU speeds  Linux  All multicast  Throughput to each host is typically at least 10Mb/s, depending on CPU and network load of both sender and receiver.  WAN  UK eScience Network  Mix of CPU speeds  FreeBSD/Linux/Solaris  Some multicast  Throughput varies (Mb/s)  Imperial25  Cardiff13  Manchester11  Southampton10  Belfast5

All Hands Meeting 2004 LAN speed

All Hands Meeting 2004 LAN efficiency

All Hands Meeting 2004 WAN efficiency

All Hands Meeting 2004 Driver availability  Multicast transport driver for Globus XIO  Requires Globus 3.2 or later  Source code online  Sample client  Sample server  Driver installation instructions 

All Hands Meeting 2004 mcp & mcpd  Multicast file transfer application using TCP-XM  ‘ mcpd & ’ on servers  ‘ mcp file host1 host2… hostN ’ on client   Full source code online  FreeBSD, Linux, Solaris

All Hands Meeting 2004 Future  Protocol work  Parallel unicast / multicast transmission  Multicast look ahead & multiple groups  Deliverables  Updates to mcp/mcpd  Experimentation  More detailed testing required  Currently limited to LAN and UK eScience  Will extend to global Intel PlanetLab

All Hands Meeting 2004 eScience volunteers…  1. In place  Cambridge  Cardiff  Imperial  Manchester  Newcastle  Oxford  Southampton  UCL  2. Firewall issues  Belfast  Daresbury  Glasgow  Rutherford  3. Not possible  Edinburgh

All Hands Meeting 2004 All done!  Thanks for listening!  Questions?