6/2/20071 Grid Computing Sun Grid Engine (SGE) Manoj Katwal

6/2/20072 Outline Grid and Cluster Terminology What is a Cluster vs. a Grid? Types of Grids Benefits of Grid Computing Areas of concern What is Sun Grid Engine? Example N1 Grid Network Setup Do you know? Grid computing market will reach $ 12 billion by the end of 2007 (IDC)

6/2/20073 Outline, Cont. Managing Jobs and Resources with a Grid Sun N1 Grid Engine Concepts in Sun Grid Engine Job Submission Queues QMON Useful commands References Do you know? Grid computing market will reach $ 12 billion by the end of 2007 (IDC)

6/2/20074 Grid and Cluster Terminology A GRID is a collection of computing resources that perform tasks. A grid appears to users as a large system that provides a point(s) of access to powerful distributed resources. –Resources: Computer hardware, data, applications,networking etc. Cluster: –Tightly coupled homogenous computers, resources –Redundant interconnections that appears as single system with high availability (HB) –Not truly distributed system

6/2/20075 Grid and Cluster Terminology, Cont. Grid: –Loosely coupled heterogeneous computers or resources –Resources can come and go –High scalability – dynamically resources can be added and removed –It is distributed virtually everywhere –High Performance Computing (HPC) Do you know? The current version of Sun Grid Engine is downloadable for free.

6/2/20076 What is a Cluster vs. a Grid? Gentzsch, “Grid Computing: A New Technology for the Advanced Web”

6/2/20077 Types of Grids

6/2/20078 Benefits of Grid Computing Better utilization of resources Increased user productivity Faster time-to-solution of complex scientific, engineering, and enterprise computational tasks Do you know? You can register to and get 200 CPU hours free of charge. No obligationhttp://

6/2/20079 Areas of Concern Users concern about Grid operation and Management include: Confusion about what grid is and how it works Cross-vendor interoperability CPU and application usage billing and accounting Security and user authentication Standardization Global enterprise grids across international boundaries

6/2/ What is Sun Grid Engine? Open source Grid management software – accepts jobs submitted by users Software driving “ job ” distribution to compute nodes in a grid –By job means Unix like jobs (batches) or parallel programs (MPICH- MPI Interface) Enable scheduling and prioritizing of jobs Management of jobs – submit, suspend, abort … Accounting and reporting (Arco, web based) Almost all operating systems are supported

6/2/ Example N1 Grid Network Setup 12 billions dollars …

6/2/ Managing Jobs and Resources with a Grid

6/2/ Concepts in Sun Grid Engine Hosts – Four types –Master host, Secondary Fail over host (optional) –Execution hosts –Admin host –Submit hosts A system can act as more than one type of host.

6/2/ Job Submission

6/2/ Queues Cluster queues Job slots Resource reservation Parallel environment (MPI support) Policy based resource allocation Job checkpointing

6/2/ QMON

6/2/ Useful commands Qsub – to submit job Qhost – getting an overview of available resources Qacct – job accounting information Qmod – managing queue state Qmon – graphical admin tool Qconf – configuring queues

6/2/ References By Jan Stafford, Editor, 30 Aug 2005 | SearchOpenSource.com 42,sid39_gci ,00.html 42,sid39_gci ,00.html California State University, LA cluster (1 head node with 10 clients) Sun Grid Compute Utility Web Portal