1. I-Cluster ACM-SIGOPS JTE Cluster Computing Bruno Richard Research Program Manager HP Labs Grenoble

2. Page 22October 2001 HP Labs Focus on infrastructure, information appliances and e- services World’s second largest computer research lab 750 researchers in 6 labs globally Leading-edge collaborations Helps create HP’s IP portfolio

3. Page 32October 2001 HP Labs Grenoble Role Deliver technology related to emerging Internet access devices Two main research areas: PPP –personalization, profiling, privacy –User-adapted computing environment LEA –Local Environment of Access Devices –Wireless communities I-Cluster Compute-intensive services Partnership with ID-IMAG, INRIA

4. Page 42October 2001 Project overview What is I-Cluster? A distributed framework of tools that transparently takes advantage of unused network resources and transforms them into compute- intensive services Project rationale Support distributed virtual functions utilising unmodified, standard hardware Learn how cluster devices interact with each other -- potential and limitations Create environment for development and execution of applications that will use enterprise infrastructure or internet rather than dedicated cluster Apply knowledge gained from I-Cluster to future products

5. Page 52October 2001 I-Cluster Value Supercomputing-enabled Not limited to metacomputing –Cluster computing environment Fine-grained problems –Network communications can be stressed User compliance Self-organized system –Self sustaining From federative model to community model –Server-less –Service oriented –No administration –Peer-to-peer system Real-world conditions –Massively scales (10000 devices) –No specific hardware required –Heterogeneous environments –Roaming, disconnections Applications do not need to be rewritten –But no shared memory

6. Page 62October 2001 I-Cluster Cloud Device I-Cluster Cloud Device Step 4 My service is executed on the cluster within seconds Device Step 1 Request a computing service from the cloud. “Render Star Wars movie for my PDA” I-Cluster Cloud Device Step 2 Identify cluster aggregate that fits the required service Device I-Cluster Cloud Device Step 3 Efficiently distribute the job on identified devices Device Starwars.avi Usage example

7. Page 72October 2001 Research areas I-Cluster cloud P2P community Gathers resources Discovers network topology Mode switch Any PC becomes a cluster machine Use idle periods No (lowest) user impact Match finder Instantiates a cluster on the cloud Allocates devices to given jobs Tetris Inter-intra task scheduling

8. Page 82October 2001 I-Cluster cloud Devices operate in peer to peer: A server answers other’s requests A client actively polls others Local database: a view of the cloud Incomplete –Some elements are unknown –Elements are forgotten Lazy consistency –Best effort consistency Network analysis Topology Bandwidth, latency Congestion analysis Very fast convergence

9. Page 92October 2001 Cloud operation: An example Full network view Blue: I-Cluster devices Yellow: Routers+other A B

10. Page 102October 2001 Cloud operation (cont’d) View from device A A B

11. Page 112October 2001 Cloud operation (cont’d) View from device B A B

12. Page 122October 2001 Cloud operation (cont’d) Combined view After synchronization Each device will forget some items Based on relevance of peers A B

13. Page 132October 2001 I-Cluster mode switch Each I-Cluster PC has 2 modes -User mode -Standard Windows operation -Cluster mode -Cluster Linux distribution -Sandbox for jobs execution -Secure mode (no user data access) Idle periods used for cluster computing -Idleness detection -Automatic switch between modes Lowest user impact -Easy installation -No change in user partitions -Low psychological impact -Automatic transitions Ease of development -Easier than other sandboxing technologies

14. Page 142October 2001 Hidden cluster distribution User hard disk partitions are kept without modification

15. Page 152October 2001 Hidden cluster distribution Then anchors are added Master Boot Record is changed to a new code A hidden zone is used for storing our mode control tools

16. Page 162October 2001 Hidden cluster distribution Then a fake partition is created Big file in user’s file system Contiguous, unfragmented System-protected, unwriteable This partition will be usable as a boot option by the switcher

17. Page 172October 2001 Match finder Called upon user request A service is invoked User data attached to invocation Service requirements available Number of processing nodes Minimum RAM Maximum Network latency … Allocates a cluster within the cloud Service request: - 4 nodes required - At least 256 MB 4 machines allocated Job started there

18. Page 182October 2001 Tetris Optimal use of computing resources Inter/Intra job scheduling Use of job knowledge for intelligent task/resource assignment Use of past experiences to improve future scheduling Agreements between HP-Labs and INRIA

19. Page 192 Tetris duration number of processors

20. Page 202 Tetris duration number of processors

21. Page 212October 2001 Backup slides “It is all about power, space and time” Chessmaster Savielly Grigorievitch Tartakower

22. Page 222October 2001 The experimental platform

23. Page 232 TOP500 results I-Cluster Supercomputing with a mainstream cluster A TOP500 cluster based on 225 standard hp e- Vectra machines Partnership with IMAG-ID, INRIA, Intel 225 HP e-Vectra: - Pentium® III - 733 MHz - 256 MB - Standard Ethernet (100 MBps) 76 Gflop/s * as of April 15 th 2001 (*) Standard LINPACK benchmark http://www.top500.org/