1. Research Achievements Kenji Kaneda

2. Agenda Research background and goal Research background and goal Overview of my research achievements Overview of my research achievements Phoenix Phoenix Virtual Private Grid Virtual Private Grid Summary and recent activities Summary and recent activities

3. Research Background and Goal

4. Background Grid computing Grid computing Parallel computing with harnessing many widely-distributed resources Parallel computing with harnessing many widely-distributed resources E.g.) aggregate of PC clusters spread over multiple LANs

5. Traditional Parallel Computing vs. Grid Computing Traditional parallel computing Reliable processors Reliable processors Single-LAN resources Single-LAN resources Grid computing Unreliable processors Unreliable processors Multi-LANs resources Multi-LANs resources

6. Difficulty in Grid computing Frequent machine/network failures Frequent machine/network failures E.g.) 1 machine failure per a day Restricted Connectivity Restricted Connectivity Administrative policies restrict communications between machines Administrative policies restrict communications between machines E.g.) firewall, NAT, DHCP Gateway TCP Gateway Firewall

7. Research Goal Allow a user to harness a computational grid like traditional parallel computing Allow a user to harness a computational grid like traditional parallel computing Fault tolerance Fault tolerance Transparent communication on WANs Transparent communication on WANs

8. My Research Achievements Design/implementation of middlewares Phoenix Phoenix Parallel programming library for accommodating dynamically joining/leaving resources Parallel programming library for accommodating dynamically joining/leaving resources Virtual Private Grid Virtual Private Grid Command shell for utilizing hundreds of computers spread over multiple LANs Command shell for utilizing hundreds of computers spread over multiple LANs

9. Phoenix

10. Phoenix Parallel programming library for accommodating dynamically joining/leaving resources Parallel programming library for accommodating dynamically joining/leaving resources Programming model for supporting migration of application states Programming model for supporting migration of application states Transparent communication mechanism for WANs Transparent communication mechanism for WANs

11. Programming Model for Supporting Migration of Application States Subsumes a regular message passing model Subsumes a regular message passing model Provides a namespace that does not depend on physical machines Provides a namespace that does not depend on physical machines Programmer uses this name to specify a message destination Programmer uses this name to specify a message destination Programmer can write a program without being aware of physical machines Programmer can write a program without being aware of physical machines

12. Transparent Communication mechanism for WANs Overlay network construction Overlay network construction Application-level message routing Application-level message routing Processes can communicate with one another Processes can communicate with one another even if networks are not fully connected even if networks are not fully connected even if connection topologies change dynamically even if connection topologies change dynamically

13. Demonstration Boot processes on 3 subnets Boot processes on 3 subnets Add processes dynamically Add processes dynamically

14. Demonstration

15. Experiments (1/3) Speedup with fixed resources Speedup with fixed resources POV-Ray: 78 speedup using 104 processors on 3 LANs POV-Ray: 78 speedup using 104 processors on 3 LANs LU: comparable to MPICH (on a single LAN) LU: comparable to MPICH (on a single LAN)

16. Experiments (2/3) Speedup with dynamic resources Speedup with dynamic resources POV-Ray takes advantage of dynamically added resources quickly POV-Ray takes advantage of dynamically added resources quickly

17. Experiments (3/3) Parallel shogi (Japanese chess) program on 720 laptop PCs Parallel shogi (Japanese chess) program on 720 laptop PCs 7~8 speedup 7~8 speedup

18. Related Work Grid enabled MPIs Grid enabled MPIs E.g.) MPICH-G [G. Bosilca et al. SC ’ 02] Based on a traditional message passing model Based on a traditional message passing model Difficult to support dynamic changes of resources Difficult to support dynamic changes of resources Communications libraries for Grids Communications libraries for Grids E.g.) Ibis [A. Denis et al. HPDC ’ 04] Static message routing Static message routing

19. Summary ~ Phoenix ~ Parallel programming library for dynamically changing resources Parallel programming library for dynamically changing resources Good speedup with a large number of machines on multiple LANs Good speedup with a large number of machines on multiple LANs

20. Virtual Private Grid

21. Virtual Private Grid (VPG) Command shell for utilizing hundreds of computers spread over multiple LANs Command shell for utilizing hundreds of computers spread over multiple LANs

22. Features (1/2) User can submit jobs without caring administrative restrictions User can submit jobs without caring administrative restrictions E.g.) cmd1@host1 | cmd2@host2 > file3@host3 Firewall host1host2 Firewall NAT host3 Execute cmd1 Write to file3 Execute cmd2

23. Features (2/2) Fault tolerance Fault tolerance VPG can continue to run even if some machines are added/deleted dynamically VPG can continue to run even if some machines are added/deleted dynamically No central server is required No central server is required

24. Demonstration Environment Environment 3 LANs 3 LANs CPU: Sparc, x86, MIPS, PowerPC CPU: Sparc, x86, MIPS, PowerPC OS: Solaris, Linux, IRIX OS: Solaris, Linux, IRIX

25. Demonstration

26. Related Work Grid job submission tools Grid job submission tools E.g.) Globus, Condor-G Difficult to submit jobs to machines under administrative restrictions Difficult to submit jobs to machines under administrative restrictions

27. Summary ~ Virtual Private Grid ~ Command shell for utilizing hundreds of computers spread over multiple LANs Command shell for utilizing hundreds of computers spread over multiple LANs Fast job submission to more than 100 machines Fast job submission to more than 100 machines

28. Summary and Recent activities

29. Summary ~ My Research Achievements ~ Middlewares for Grid computing Middlewares for Grid computing Phoenix Phoenix Virtual Private Grid Virtual Private Grid

30. Recent Activities (1/2) Virtual SMP Virtual SMP Emulates a multi-processor machines on a loosely-coupled computes Emulates a multi-processor machines on a loosely-coupled computers Virtual dual processor machine on two single processor machines

31. Recent Activities (2/2) Virtual SMP Virtual SMP Easy utilization of distributed resources with a common OS (e.g., Windows, Linux) Easy utilization of distributed resources with a common OS (e.g., Windows, Linux)