1October 9, 2001 Sun in Scientific & Engineering Computing Grid Computing with Sun Wolfgang Gentzsch Director Grid Computing Cracow Grid Workshop, November 5 - 6, 2001

2October 9, 2001 Summary: Sun's Grid Computing Offerings  Sun's existing scalable Grid Computing software stack  Open source building blocks (SGE, Broker, ClusterTools, TCP Portal, Jxta,...)  Encourage Your research contribution to open source (community)  Integration with Globus etc. (SGE/Broker-Globus-SGEs)  Sun Center of Excellence Program (cooperation ! )  Collaboration, joint Grid projects, Sun GridSIG,...

3October 9, 2001 Different Levels of Grids  Stage 1- 1 Owner / 1 Cluster Cluster Grid Domain of SGE & Technical Computing Stack  Stage 2- Multiple Owners, 1+ Clusters, 1 Enterprise, 1 Site Campus Grid Domain of SGE/EE & Multicluster Solutions  Stage 3- Multiple Sites, Multiple Enterprises Global Grid Domain of SGE/EE plus Grid Frameworks

4October 9, 2001 Grid Levels Global Grid (mult.owners, mult.sites) Grid Resource Mgmt, Security, Authentication, Distributed Data User interface GLOBAL CAMPUS LOCAL Campus 1 mult.owners, 1site Campus 2 mult.owners, 1site Resource Sharing & Brokerage Cluster 1 1owner, 1site Cluster Mgmt Resource Mgt Cluster 2 1owner, 1site Cluster Mgmt Resource Mgt Cluster 1 1owner, 1site Cluster Mgmt Resource Mgt Cluster 2 1owner, 1site Cluster Mgmt Resource Mgt

5October 9, 2001 Grid Sun  "The Network is the Computer"  Java, Jini, Jxta,...  July'00: Acquisition of Gridware  "Grid" projects since 1995, Julius, Medusa, Eroppa, Unicore, Autobench,...  Grid Engine, free, open source, ubiquitous, open API  Department for Grid Computing (inSun VSP): Cluster SW/Stack, Grid SW/Stack, Grid Computing Lab, customer pilots  Sun Grid Computing Council

6October 9, 2001 Sun Grid Software Stack Global Grid Sun TCP, SGE Broker, Globus, Avaki, Cactus, Punch,... Sun TCP Technical Computing Portal & iPlanet Security GLOBAL CAMPUS LOCAL Campus 1 Campus 2 Sun TCP, SGE Broker Cluster 1 TCP SGE ClusterTools SRM SunMC Jxta Jiro QFS Cluster 2 TCP SGE ClusterTools SRM SunMC Jxta Jiro QFS Cluster 1 TCP SGE ClusterTools SRM SunMC Jxta Jiro QFS Cluster 2 TCP SGE ClusterTools SRM SunMC Jxta Jiro QFS

7October 9, 2001 Sun Technical Computing Portal "prototype" (soon) commercially available hw/sw solution that...  Enables quick deployment of tech apps over Internet, similar to mail and calendaring  Combines light-weight architecture with  Industry-proven security and system management  Based on iPlanet and Sun Grid Engine

8October 9, 2001

9 Sun Distributed Resource Management  Load balancing maximizes resource utilization  Transparent job submission & machine selection  Monitoring and accounting ==> SGE Sun Grid Engine, open source  Guaranteeing required resources  Full control over resource utilization  Fair and share based resource usage  Implementation of management policies ==> Sun Grid Engine Broker, open source

10October 9, 2001 Managing Compute Resources with Sun Grid Engine Broker Department 1 Department 2 Department 3 Department resource accessCampus wide resource demand Project A Team B-4 Contractor X Project C User 1 Manage the full matrix of demand - Users -Teams -Projects User 2 Department 4 Department 5

11October 9, 2001 Sun Grid Engine Status  Ubiquitous, free, open source, open APIs  Current Release: SGE (July 2001)  Over 13,000 downloads (Nov 2001) (-: 1 Mio downloads in 2084 :-) =>> SGE = The Leading RMS  OpenSource (July 2001) 500,000 lines, Sept 2001: 1000 downloads  Today: Grid Computing everywhere in Sun !! /...

12October 9, 2001 Grid Resource Management LOOSESTRINGENT SGE Broker Shared Policy Model Grid Infrastucture Layer (e.g. Globus, Avaki) SGE Resource Broker SGE site C local policy, user, etc. mgmt. Cluster Grid Campus Grid Global Grid SGE site A local policy, user, etc. mgmt. SGE site B local policy, user, etc. mgmt.

13October 9, 2001  Demo'd at Argonne National Lab ANL, ARL Army Research Lab, Raytheon, and San Diego SDSC  On 2 SGE clusters (eg: SDSC, 30 cpus and 70 cpus)  Globus/SGE interaction through GRAM scripts  Globus jobs from ANL submitted to ARL cluster  Next step: SGE/EE on top of Globus Globus & SGE/Broker on Top of SGE

14October 9, 2001  SGE: Scheduling decisions to select remote site  SGE acting as the resource broker for Globus  Globus: multi-site communication, authentication, security, file transfers,...  SGE/Globus interface to be developed:  SGE/Broker submits and tracks jobs to remote systems using Globus services SGE/Broker as Part of Globus

15October 9, 2001 Sun and Open Grid Standards  Example: DRMAA Distributed Resource Management Application API  "The Glue" between Distributed Resource Management and Applications/Tools  => Makes resource management transparent  Proposed new Working Group at Global Grid Forum in Frascati/Rome, October 2001  Presented by Veridian, Intel, and Sun

16October 9, 2001 Sun's Grid Strategy  Strong Grid core team for developing and productizing core components (like SGE, Grid Broker,TCP)  Sun Grid Computing Council: Integrate Sun technologies and products and port the environment to all Sun platforms  Sun's partners take care of other computing platforms  Collaborate with the Grid community, IT partners and our customers to build all kinds of different Grids  Sun currently is proposing, designing and building some 50 Grids with research labs, universities and industry Sun Grid software stack available TODAY