Real-World Distributed Computing with Ibis Henri E. Bal Jason Maassen

CONTENTS INTRODUCTION REALWORLD DISTRIBUTED COMPUTING IBIS ARCHITECTURE IBIS PROGRAMMING SYSTEM IPL SMART SOCKETS IBIS PROGRAMMING MODELS IBIS DEPLOYMENT SYSTEM JAVAGAT ZORILLA IBIS DEPLOY MUTIMEDIA CONTENT ANALYSIS EXPERIMENTAL EVALUATION FUTURE SCOPE CONCLUSION

Progress in high-performance distributed computer system in science and industry Provides transparent and efficient computing For more complex applications existing scheduling system is limited. Ibis allows easy programming and development of compute-intensive distributed applications,even for dynamic, faulty and heterogeneous enviornment.

REAL-WORLD DISTRIBUTED COMPUTING Ad-hoc collection of compute resources that communicate with one another via some network connection constitutes a real world distributed computing system. Writing applications of such system is difficult. High performance distributed system can be abstracted these complexities by a single software system that applies to any real world distributed system These system have two logically independent subsystems such as programming system and deployment system.

DISTRIBUTED SYSTEM

IBIS ARCHITECTURE Ibis uses java vertual machine technology. Ibis system architecture follows the duel system approach such as the programming system and deployment system. Programming system should allow efficient programming models that support fault tolarence and malleability.Implemented in the Ibis portability layer(IPL) Deployment system contains a GUI and a library for deploying and managing applications implemented on a middleware interoperatability layer(MIL).

. It should also provide support for distributed file mamagement,user authentication,resource manegement and interoperatability between different middleware system. Ibis is modular and flexible.

IBIS PROGRAMMING SYSTEM IPL IPL is a java based communication library. In IPL the resources can be added or removed dynamically. It provides a range of communication primitives including those for point-to- point and multicast communication. It avoids the copy overhead. It incorporates a mechanism ,Join Elect Leave(JEL).

. JEL is based on the concept of signalling. JEL provides the building blocks for fault tolerance. Java IPL implementations are available using Ibis SmartSocket library,TCP UDP and bluetooth. Also performs implementation using specialised non java libraries such as MX,MPI.

SMART SOCKETS Incoming traffic at a node may be restricted by a firewall. SmartSocket library solves the problem like reverse connection setup,overlay routing,and secure Shell (SSH)tunneling. When creating a connection ,smartsocket initially tries to set up a regular TCP connection. Smartsockets also handles machine with multiple network address via multihoming. Extra identity checks in the protocol ensure that it reaches the correct destination.

IBIS PROGRAMMING MODELS MPJ,MPI binding for java; Satin, a divide and conquer model; RMI, object oriented remote procedural call; Group method invocation(GMI)a generalisation of RMI to group communication, including multicast and all to all communication. Maestro, a fault tolerant and self optimizing data flow model. Jorus, a programming model for data parallel multimedia applications.

IBIS DEPLOYMENT SYSTEM JavaGAT The JavaGAT provides a high-level API that facilitates development of complex applications. This API is object oriented. The primitives provide access to remote data, user authentication, resource management, and storing of application-specific data The JavaGAT also uses intelligent dispatching to integrate multiple middleware systems with different and incomplete functionality into a single, consistent system.

1 import org.gridlab.gat.*; 2 import org.gridlab.gat.io.File; 3 4 public class Copy { 5 public static void main(String [] args)throws Exception { 6 GATContext context = newGATContext(); 7 URI source = new URI(args[0]); 8 URI dest = new URI(args[1]); 9 10 File file = GAT.createFile(context,source); // Create a GAT file 11 12 file.copy(dest); // The actual file or directory copy 13 14 GAT.end(); // Shutdown the JavaGAT 15 } 16 }

Zorilla Zorilla is a lightweight P2P middleware that runs on any real-world distributed system. Zorilla has no central components and is easy to set up and maintain. It supports fault tolerance and malleability by implementing all functionality using P2P techniques. Zorilla which performed with the help of JavaGAT.

Ibis deploy Ibis deploy provides a simple and generic API and GUI that can automatically perform commonly used deployment scenarios. When a distributed Ibis application is running, It also automatically uploads the program codes, libraries, and input files and automatically downloads the output files. Run multiple distributed applications concurrently.

APPLICATION EXAMPLE: MULTIMEDIA CONTENT ANALYSIS DIGITAL CAMERA

EXPERIMENTAL EVALUATION Distributed ACCI super computer 3. Five clustered distributed grid systems. Amazon EC2 cloud system.

RELATED WORKS ProActive. Phoenix. The GRID superscalar framework. The GridRPC specification. The Open Grid Forum.

APPLICATIONS Multimedia computing. Spectroscopic data processing. Human brain-scan analysis. Automatic grammar learning.

OPEN PROBLEMS AND FUTURE WORK Non-Java applications could also use the IPL through the JNI which is complicated. For high programming applications it cannot be used. Intelligent-dispatching technique leads to more complex error reporting and debugging if operations fail. JavaGAT introduces runtime overhead. Visual debugging profiling tools should developed for solving these problems

CONCLUSION Ibis reduces the effort needed to create and deploy applications for real-world distributed systems. To achieve this, it integrates solutions to many fundamental distributed computing problems in a single modular programming and deployment system. Two-way connectivity is rare in a real-world distributed system. However, Smart Sockets achieves this in a transparent manner. Ibis also tries to make distributed programming easier by providing high-level programming models on top of these mechanisms

REFERENCES I. Foster, C. Kesselman, and S. Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations,” Int’l J. High-Performance Computing Applications, Aug. 2001, pp. 200-222. D. Butler, “The Petaflop Challenge,” Nature, 5 July 2007, pp. 6-7. K. Verstoep et al., “Experiences with Fine-Grained Distrib- uted Supercomputing on a 10G Testbed,” Proc. 2008 8th IEEE Int’l Symp. Cluster Computing and the Grid (CCGrid 08), IEEE CS Press, 2008, pp. 376-383