Introducing EC-GIN: Europe-China Grid InterNetworking Europe-China Grid InterNetworking European Sixth Framework STREP FP IST Michael Welzl, Dan He OGF 27, 15 oct 2009

Enriched with customised network mechanisms Original Internet technology What is EC-GIN about? Traditional Internet applications (web browser, ftp,..)‏ Real-time multimedia applications (VoIP, video conference,..)‏ Today‘s Grid applications Driving a racing car on a public road Applications with special network properties and requirements Bringing the Grid to its full potential ! EC-GIN EC-GIN enabled Grid applications

Project facts STREP in FP6 Call 6, total funding € 2.2 M STREP in FP6 Call 6, total funding € 2.2 M Start date: 1 November 2006; duration: 3 years Start date: 1 November 2006; duration: 3 years Consortium: 11 partners Consortium: 11 partners  7 European partners  Universität Innsbruck, UIBK, Austria (coordinator)‏  University of Zürich, UniZH, Switzerland  Institut National de Recherche en Informatique et Automatique, INRIA, France  Lancaster University, ULANC, U.K.  Justinmind, JIM, Spain  EXIS IT, Greece  University of Surrey, UniS, U.K.  4 Chinese partners  Beijing University of Posts and Telecommunications, BUPT  Institute of Software, Chinese Academy of Sciences, ISCAS  China Telecommunication Technology Labs, CTTL  China Mobile Group Design Institute Co., Ltd, CMDI

Research Objectives and Challenges Objectives Objectives  The EC-GIN project, based on a number of properties that make Grids unique from the network perspective, aims at developing a tailored network technology in dedicated support of Grid applications. These technical solutions will be supplemented with a secure and incentive-based Grid Services network traffic management system, which will balance the conflicting performance demand and the economic use of resources in the network and within the Grid.” Research Challenges Research Challenges  How to model Grid traffic and simulate a Grid-network?  Much is known about web traffic (e.g. self-similarity) - but the Grid is different  Up to now, Grid-Sim / Net-Sim were two separate worlds (different goals, assumptions, tools, people  How to specify and manage QoS?  Grids need file transfer deadlines, not sustained bit rates  How to align network and Grid economics?  Grid service model, charging model for grid services  Network Mgmt mechanisms in support of those areas in an integrated fashion

Example Scenario: Large File Transfer Multipath file transfer (A  B + A  C  B) beneficial Multipath file transfer not beneficial due to shared bottleneck Also consider: B wants to send to A; obtains prediction; does not know that C sends to A at the same time  prediction is wrong in right diagram

Large File Transfer Scenarios: Questions When does use of multiple paths make sense? When does use of multiple paths make sense?  To increase overall throughput How could this functionality be exposed? How could this functionality be exposed?  As a transport service? What constraints can be considered? What constraints can be considered?  To allow the delivery of a certain level of QoS How could large file transfer be authenticated and authorized? How could large file transfer be authenticated and authorized?  By intermediate nodes? How to align incentives? How to align incentives?  Who should be charged how much?

Some answers (EC-GIN results)‏ Incentives: Private Shared History - PSH2 Incentives: Private Shared History - PSH2  tested in file sharing scenario; closely related to LFT  PSH2 simulation with data from ThePirateBay  Scaled down by 1000 for simulation (70 GB scaled to 70 MB); used 29 files with over 1000 nodes Pricing Pricing  Resource allocation model based on combinatorial double auction  Four entities involved: User broker (UB), Grid service provider (GSP), Grid market auctioneer (GMA), Combinatorial Double Auctioneer, Grid information service (GIS)‏

More results: models and simulation Various Grid traffic measurements and analyses Various Grid traffic measurements and analyses  ATLAS (EGEE), GridFTP, IPTV CDN, PPLive ns-2 code for Grid traffic simulation ns-2 code for Grid traffic simulation  Traffic generators for applications above available from Sourceforge  can be tuned for a wide variety of network conditions and streaming parameters

QoS for Grids: Bulk Data Transfer Service (BDTS) Real network Control Plane: jBDTS Data Plane: FLOC Contribution to OGF- NSI: Network Service Interface

Weighted fairness for data transfers Based on model of multiple TCP flows (extension of Padhye eqn)‏ Based on model of multiple TCP flows (extension of Padhye eqn)‏  based on RTT, loss and parameter n Q  Easy to calculate; n  Q >0 MulTFRC: emulate n TCP flows MulTFRC: emulate n TCP flows  Based on TFRC (Equation Based TCP- friendy Rate Control protocol)‏  Equation exchanged with ours + some minor other changes Applications  Assign priorities to flows belonging to one user  Assign priorities to users (e.g. M-IPTV)‏

... and many other things, available as tools... GridMAP  Passive measurement system High-Speed SOAP Engine  Transparently deployable speed improvement Dynamic optimum server selection  Mobile IPTV Peer Awareness (for fast file transfer with limited number of nodes)‏  Topology discovery and shared bottleneck detection Parallel Transfer Grid (for fast file transfer with large number of nodes)‏  Heuristic based scheme to find and use the best Forwarding Peers

... and integrated in GINTONIC Grid InterNetworking TOolbox Nestled in the Core Grid InterNetworking TOolbox Nestled in the Core  Separate daemons which communicate via “Base”  Architecture somewhat similar to Globus Toolkit  You install the “Base” plus what you need, not everything!

Thanks for your attention! More information, references, code: