1. The Ibis Project: Simplifying Grid Programming & Deployment Henri Bal bal@cs.vu.nl Vrije Universiteit Amsterdam

3. The ‘Promise of the Grid’ Efficient and transparent (i.e. easy-to-use) wall-socket computing over a distributed set of resources [Sunderam ICCS’2004, based on Foster/Kesselman]

4. Parallel computing on grids ● Mostly limited to ● trivially parallel applications ● parameter sweeps, master/worker ● applications that run on one cluster at a time ● use grid to schedule application on a suitable cluster ● Our goal: run real parallel applications on a large-scale grid, using co-allocated resources

5. Efficient wide-area algorithms ● Latency-tolerant algorithms with asynchronous communication ● Search algorithms (Awari-solver [CCGrid’08] ) ● Model checkers (DiVinE [PDMC’08] ) ● Algorithms with hierarchical communication ● Divide-and-conquer ● Broadcast trees ● …..

6. ● Performance & scalability ● Heterogeneous ● Low-level & changing programming interfaces ● writing & deploying grid applications is hard Reality: ‘Problems of the Grid’ ● Connectivity issues ● Fault tolerance ● Malleability Wide-Area Grid SystemsUser !

7. The Ibis Project ● Goal: ● drastically simplify grid programming/deployment ● write and go!

8. Approach (1) ● Write & go: minimal assumptions about execution environment ● Virtual Machines (Java) deal with heterogeneity ● Use middleware-independent APIs ● Mapped automatically onto middleware ● Different programming abstractions ● Low-level message passing ● High-level divide-and-conquer

9. Approach (2) ● Designed to run in dynamic/hostile grid environment ● Handle fault-tolerance and malleability ● Solve connectivity problems automatically (SmartSockets) ● Modular and flexible: can replace Ibis components by external ones ● Scheduling: Zorilla P2P system or external broker

10. Global picture

11. Rest of talk Satin: divide & conquer Communication layer (IPL) SmartSockets Applications JavaGAT Zorilla P2P

12. Outline ● Grid programming ● IPL ● Satin ● SmartSockets ● Grid deployment ● JavaGAT ● Zorilla ● Applications and experiments

13. Ibis Design

14. Ibis Portability Layer (IPL) ● Java-centric “run-anywhere” library ● Sent along with the application (jar-files) ● Point-to-point, multicast, streaming, …. ● Efficient communication ● Configured at startup, based on capabilities (multicast, ordering, reliability, callbacks) ● Bytecode rewriter avoids serialization overhead

15. Serialization ● Based on bytecode-rewriting ● Adds (de)serialization code to serializable types ● Prevents reflection overhead during runtime Java compiler bytecode rewriter JVM source bytecode

16. Membership Model ● JEL (Join-Elect-Leave) model ● Simple model for tracking resources, supports malleability & fault-tolerance ● Notifications of nodes joining or leaving ● Elections ● Supports all common programming models ● Centralized and distributed implementations ● Broadcast trees, gossiping

17. Programming models ● Remote Method Invocation (RMI) ● Group Method Invocation (GMI) ● MPJ (MPI Java 'standard') ● Satin (Divide & Conquer)

18. Satin: divide-and-conquer ● Divide-and-conquer is inherently hierarchical ● More general than master/worker ● Cilk-like primitives (spawn/sync) in Java ● Supports malleability and fault-tolerance ● Supports data-sharing between different branches through Shared Objects

19. Satin implementation ● Load-balancing is done automatically ● Cluster-aware Random Stealing (CRS) ● Combines Cilk’s Random Stealing with asynchronous wide-area steals ● Self-adaptive malleability and fault-tolerance ● Add/remove machines on the fly ● Survive crashes by efficient recomputations/checkpointing

20. Self-adaptation with Satin ● Adapt #CPUs to level of parallelism ● Migrate work from overloaded to idle CPUs ● Remove CPUs with poor network connectivity ● Add CPUs dynamically when ● Level of parallelism increases ● CPUs were removed or crashed ● Can also remove/add entire clusters ● E.g., for network problems [Wrzesinska et al., PPoPP’07 ]

21. Approach ● Weighted Average Efficiency (WAE): 1/#CPUs * Σ speed i * (1 – overhead i ) overhead is fraction idle+communication time speed i = relative speed of CPU i (measured periodically) ● General idea: Keep WAE between E min (30%) and E max (50%)

22. Overloaded network link ● Uplink of 1 cluster reduced to 100 KB/s ● Remove badly connected cluster, get new one Iteration duration Iteration

23. Connectivity Problems ● Firewalls & Network Address Translation (NAT) restrict incoming traffic ● Addressing problems ● Machines with >1 network interface (IP address) ● Machine on a private network (e.g., NAT) ● No direct communication allowed ● E.g., between compute nodes and external world

24. SmartSockets library ● Detects connectivity problems ● Tries to solve them automatically ● With as little help from the user as possible ● Integrates existing and several new solutions ● Reverse connection setup, STUN, TCP splicing, SSH tunneling, smart addressing, etc. ● Uses network of hubs as a side channel

25. Example

26. [Maassen et al., HPDC’07 ]

27. Overview JavaGAT Zorilla P2P

28. JavaGAT ● GAT: Grid Application Toolkit ● Makes grid applications independent of the underlying grid infrastructure ● Used by applications to access grid services ● File copying, resource discovery, job submission & monitoring, user authentication ● API is currently standardized (SAGA) ● SAGA implemented on JavaGAT

29. Grid Applications with GAT GAT Engine Remote Files Monitoring Info service Resource Management GridLabGlobusUnicoreSSHP2PLocal GAT Grid Application File.copy(...)‏ submitJob(...)‏ gridftp globus Intelligent dispatching [van Nieuwpoort et al., SC’07 ]

30. Zorilla: Java P2P supercomputing middleware

31. Zorilla components ● Job management ● Handling malleability and crashes ● Robust Random Gossiping ● Periodic information exchange between nodes ● Robust against Firewalls, NATs, failing nodes ● Clustering: nearest neighbor ● Flood scheduling ● Incrementally search for resources at more and more distant nodes [Drost et al., HPDC’07 ]

32. Overview

33. Ibis applications ● e-Science (VL-e) ● Brain MEG-imaging ● Mass spectroscopy ● Multimedia content analysis ● Various parallel applications ● SAT-solver, N-body, grammar learning, … ● Other programming systems ● Workflow engine for astronomy (D-grid), grid file system, ProActive, Jylab, …

34. Overview experiments ● DAS-3: Dutch Computer Science grid ● Satin applications on DAS-3 ● Zorilla desktop grid experiment ● Multimedia content analysis ● High resolution video processing

35. DAS-3DAS-3 272 nodes (AMD Opterons) 792 cores 1TB memory LAN: Myrinet 10G Gigabit Ethernet WAN (StarPlane): 20-40 Gb/s OPN Heterogeneous: 2.2-2.6 GHz Single/dual-core Delft no Myrinet

36. Gene sequence comparison in Satin (on DAS-3) Speedup on 1 cluster Run times on 5 clusters Divide&conquer scales much better than master-worker 78% efficiency on 5 clusters (with 1462 WAN-msgs/sec)

37. Barnes-Hut (Satin) on DAS-3 Shared object extension to D&C model improves scalability 57% efficiency on 5 clusters (with 1371 WAN-msgs/sec) Speedup on 1 cluster Run times on 5 clusters

38. Zorilla Desktop Grid Experiment ● Small experimental desktop grid setup ● Student PCs running Zorilla overnight ● PCs with 1 CPU, 1GB memory, 1Gb/s Ethernet ● Experiment: gene sequence application ● 16 cores of DAS-3 with Globus ● 16 core desktop grid with Zorilla ● Combination, using Ibis-Deploy

39. Ibis-Deploy deployment tool 3574 sec1099 sec 877 sec Easy deployment with Zorilla, JavaGAT & Ibis-Deploy

40. Multimedia content analysis ● Analyzes video streams to recognize objects ● Extract feature vectors from images ● Describe properties (color, shape) ● Data-parallel task implemented with C++/MPI ● Compute on consecutive images ● Task-parallelism on a grid

41. MMCA application Client (Java) Broker (Java) Parallel Horus Server Parallel Horus Servers (C++) (any machine world-wide) (local desk-top machine) Ibis (Java) (grid)

42. MMCA with Ibis ● Initial implementation with TCP was unstable ● Ibis simplifies communication, fault tolerance ● SmartSockets solves connectivity problems ● Clickable deployment interface ● Demonstrated at many conferences (SC’07) ● 20 clusters on 3 continents, 500-800 cores ● Frame rate increased from 1/30 to 15 frames/sec [Seinstra et al., IEEE Multimedia’07 ]

43. MMCA ‘ Most Visionary Research’ award at AAAI 2007, (Frank Seinstra et al.)

44. High Resolution Video Processing ● Realtime processing of CineGrid movie data ● 3840x2160 (4xHD) @ 30 fps = 1424 MB/sec ● Multi-cluster processing pipeline ● Using DAS-3, StarPlane and Ibis

45. CineGrid with Ibis ● Use of StarPlane requires no configuration ● StarPlane is connected to local Myrinet network ● Detected & used automatically by SmartSockets ● Easy setup of application pipeline ● Connection administration of application is simplified by the IPL election mechanism ● Simple multi-cluster deployment ● IbisDeploy provides easy to use grid submission

46. Summary ● Goal: Simplify grid programming/deployment ● Key ideas in Ibis ● Virtual machines (JVM) deal with heterogeneity ● High-level programming abstractions (Satin) ● Handle fault-tolerance, malleability, connectivity problems automatically (Satin, SmartSockets) ● Middleware-independent APIs (JavaGAT) ● Modular

47. Acknowledgements Current members Rob van Nieuwpoort Jason Maassen Thilo Kielmann Frank Seinstra Niels Drost Ceriel Jacobs Kees Verstoep Roelof Kemp Kees van Reeuwijk Past members John Romein Gosia Wrzesinska Rutger Hofman Maik Nijhuis Olivier Aumage Fabrice Huet Alexandre Denis

48. More information ● Ibis can be downloaded from ● http://www.cs.vu.nl/ibis http://www.cs.vu.nl/ibis ● Papers: ● Satin [PPoPP’07], SmartSockets [HPDC’07], Gossiping [HPDC’07], JavaGAT [SC’07], MMCA [IEEE Multimedia’07] Ibis tutorials Next one at CCGrid 2008 (19 May, Lyon)

49. EXTRA

50. Ibis in context OrcaManta Object-based language (1986-1996) High-performance Java (1996-2002) DAS Albatross Distributed ASCI supercomputer (1997 - now) Wide-area parallel computing (1997- 2002) GridLabGAT EU FP5 project (2002 – 2005) Grid Application Toolkit Ibis library JavaGAT Multimedia applications e-Science applications

51. Ibis Design TCPMPI Application Ibis Portability Layer (IPL)‏ Smart Sockets MX RMISatinRepMIGMIMPJ Non-Java

52. Zorilla: Java P2P supercomputing middleware Application BootstrapGossipingClusteringScheduling Job management Clusters, grids, desktop machines

53. Disclaimers ● Java-based ● Can interface with C/C++, looses run-anywhere portability ● Limited security/authentication support ● No graphical programming environment yet

54. Ibis in context

55. Highlights (1) ● Many large-scale grid experiments … ● Non-trivially parallel applications running on many heterogeneous resources (co-allocation) ● Up to 1200 CPUs (Grid’5000 + DAS-3) ● … despite connectivity problems ● Firewalls, NAT, non-routed networks ● True self-management experiments ● Deleting/crashing/adding entire clusters

56. Highlights (2) ● Many real-life applications ● Multimedia content analysis, brain imaging, SAT- solvers, N-body simulations, protein analysis ● Many users: ● Applications: medical centers, labs ● System designers: workflow engine for astronomy (D-grid), grid file system, ProActive, Jylab, …

57. JavaGAT framework ● GAT Adaptors (“plugins”) ● Connect GAT to grid services ● Allow for multiple providers (Globus, Unicore, ProActive, Condor, PBS, …) ● GAT Engine ● Runtime delegation of API calls to adaptors ● Determines which adaptor(s) provide the “grid operation” ● Upon failure another adaptor may be called

58. Sequential Fibonacci public long fib(int n) { if (n < 2) return n; long x = fib(n - 1); long y = fib(n – 2); return x + y; }

59. Parallel Fibonacci interface FibInterface extends ibis.satin.Spawnable { public long fib(int n); } public long fib(int n) { if (n < 2) return n; long x = fib(n - 1); long y = fib(n – 2); sync(); return x + y; }

60. Parallel Fibonacci interface FibInterface extends ibis.satin.Spawnable { public long fib(int n); } public long fib(int n) { if (n < 2) return n; long x = fib(n - 1); long y = fib(n – 2); sync(); return x + y; } Mark methods as Spawnable. They can run in parallel.