Download presentation
Presentation is loading. Please wait.
1
The Ibis Project: Simplifying Grid Programming & Deployment Henri Bal, Jason Maassen, Rob van Nieuwpoort, Thilo Kielmann, Niels Drost, Ceriel Jacobs, Frank Seinstra, Roelof Kemp, Kees Verstoep Vrije Universiteit Amsterdam
2
The ‘Promise of the Grid’ Efficient and transparent (i.e. easy-to-use) wall-socket computing over a distributed set of resources [Sunderam ICCS’2004, based on Foster/Kesselman]
3
● Performance & scalability ● Heterogeneous ● Low-level & changing programming interfaces ● writing & deploying grid applications is hard Reality: ‘Problems of the Grid’ ● Connectivity issues ● Fault tolerance ● Malleability Wide-Area Grid SystemsUser !
4
The Ibis Project ● Goal: ● drastically simplify grid programming/deployment ● ``Grids as promised’’
5
Approach ● Minimal assumptions about execution environment ● Virtual Machines (Java) deal with heterogeneity ● Use middleware-independent APIs ● Different programming abstractions ● Designed for dynamic/hostile grid environments ● Modular and flexible: can replace Ibis components by external ones
6
Global picture
7
Grid programming ● Programming models: ● Message passing (RMI, MPJ) ● Divide-and-conquer (Satin) ● IPL (Ibis Portability Layer) ● Java-centric “run-anywhere” library, sent with application ● Can handle fault-tolerance, malleability ● SmartSockets ● Solve connectivity problems automatically (firewalls, NAT, addressing problems)
8
Grid deployment ● JavaGAT: Java Grid Application Toolkit ● Make applications independent of underlying grid ● Used for file copying, resource discovery, job submission & monitoring, user authentication ● API is currently standardized (SAGA) ● Zorilla P2P system ● Job management, gossiping, clustering, flood scheduling
9
Ibis applications ● e-Science (VL-e) ● Brain MEG-imaging ● Mass spectroscopy ● Multimedia content analysis ● Video processing ● Many HPC applications ● SAT-solver, N-body, grammar learning, ….
10
European users ● D-Grid: Workflow engine for astronomy ● U. Erlangen: grid file system ● INRIA: ProActive on Ibis RMI ● U. Patras: Jylab scientific computing ● UPC Barcelona: Grid Superscalar ● HITACHI: Peta-scale data management Grid’5000
11
DAS-3DAS-3 272 nodes (AMD Opterons) 792 cores 1TB memory LAN: Myrinet 10G Gigabit Ethernet WAN (StarPlane): 20-40 Gb/s OPN See CCGrid’08 session 11 (Verstoep)
12
Gene sequence comparison in Satin (on DAS-3) Speedup on 1 cluster Run times on 5 clusters 78% efficiency on 5 clusters (with 1462 WAN-msgs/sec)
13
Multimedia content analysis ● Analyzes video streams to recognize objects ● Extract feature vectors from images ● Describe properties (color, shape) ● Data-parallel task ● Compute on consecutive images ● Task-parallelism on a grid See CCGrid’08 SCALE challenge & sessions 12+16
14
High Resolution Video Processing ● Realtime processing of CineGrid movie data ● 3840x2160 (4xHD) @ 30 fps = 1424 MB/sec ● Multi-cluster processing pipeline ● Using DAS-3, StarPlane and Ibis
15
Summary ● Goal: Simplify grid programming/deployment ● Key ideas in Ibis ● Virtual machines (JVM) deal with heterogeneity ● High-level programming abstractions (Satin) ● Handle fault-tolerance, malleability, connectivity problems automatically (Satin, SmartSockets) ● Middleware-independent APIs (JavaGAT) ● Modular
16
Future work? ● Clouds as promised (CCGrid’09 Shanghai) ● Exaflop as promised (CCGrid’10 Melbourne) ● Cold fusion as promised ● Stock market prediction as promised ● Indiana Jones 5 as promised (2032?)
17
More information ● Ibis can be downloaded from ● http://www.cs.vu.nl/ibis http://www.cs.vu.nl/ibis ● Papers: ● Satin [PPoPP’07], SmartSockets [HPDC’07], Gossiping [HPDC’07], JavaGAT [SC’07], MMCA [IEEE Multimedia’07, CCGrid’08], DAS-3/StarPlane [CCGrid’08] Ibis tutorials CCGrid’08
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.