Development of High Performance Computing Environment Nizhni Novgorod State University Faculty of Computational mathematics and Cybernetics Information Technologies Laboratory - ITLab Development of High Performance Computing Environment Andrew V. Senin, Software Department

Development of High Performance Computing Environment Contents Introduction Problems of efficient computing resource management Overview of some well-known cluster management systems (CMS) Project requirements CMS Metacluster architecture: .NET web services Cluster integrator Task manager Results Future work Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Introduction High performance computing systems are required for solving many scientific and research works Mostly HPC systems are supercomputers or clusters: Cluster is a set of quite ordinary computers (SMP servers) connected with a high speed interconnect Clusters is a way to achieve high performance having even a small budget Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Problems of efficient computing resource management 24:7 access to the computing resource via the Internet Scheduling – optimal distribution of jobs among computing nodes – is of great significance An administrator can not manually regulate job executing. Thus cluster management system must do it automatically. The usual set of management functions are the following: Providing job submissions to the cluster via the Internet, Automatic scheduling job execution on compute nodes, Statistics logging and generating reports, etc. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Overview of some well-known cluster management systems The most of cluster management systems (CMS) are developed for UNIX family OS Nowadays the number of CMS systems for OS Windows is growing. Among them: Microsoft CCS 2003 (Microsoft Compute Cluster Server 2003, http://www.microsoft.com/windowsserver2003/ccs/default.mspx), LSF (Load Sharing Facility, http://www.platform.com/), Cluster CoNTroller (http://www.mpi-softtech.com) The available CMS systems are not often good for research: High cost of such systems (some systems are freely available for non-commercial use), The source code often is not open and can not be modified to provide experimental studies, High complexity of integration with self-made components. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Project requirements… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Project requirements… The cluster management system “Metacluster 2.0” has to provide: All operations to support all stages of the job execution cycle (adding to the job queue, executing, getting the current state, deleting), An uniform way to manage a set of clusters (possibly under various OS) as an integrated computing environment, Co-operation with third-party CMS, Access by means of Web, command line, graphical or application program interface, An administrator interface to control the workflow,… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Project requirements The cluster management system “Metacluster 2.0” has to provide: A built-in authentication which can differ from OS procedures, A job warehouse to store user jobs and from which jobs can be submitted for executing repeatedly, Job scheduling among clusters and then among nodes of the selected cluster, Accumulating outputs of the executed jobs, Fail-safety operation even in case of some nodes emergency, Logging the resource utilization, generating reports with various statistical data Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… MS .NET Web services Web services provide: Authentication of users (with using the database of Metacluster users), Report generation of the system operation, Remote access to the cluster management system for all registered users. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Cluster integrator Cluster Integrator provides:… Storing a job queue, Providing statistical information on demand of Web services (at least, job states, cluster load, output), Accumulating statistical data (at least logging of operations being done), Adding/deleting jobs on demand of Web services,… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Cluster integrator Cluster Integrator provides: Scheduling and distributing jobs among clusters. User will be able to specify what cluster to use. Scheduling is based on the information of the job requirements, the current cluster loads, user rights, possibly previous statistical data, Migration of jobs between clusters (in future). Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Task manager Task Manager provides: Handling the job queue that is local for a cluster. Adding jobs can be done by the cluster integrator, Scheduling and distributing jobs among compute nodes (various strategies can be applied to schedule jobs), Terminating jobs on request of the Cluster integrator,… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

CMS Metacluster architecture… Task manager Task Manager provides: Monitoring the compute nodes, Accumulating and storing outputs of the jobs (the standard output and result files), Storing statistical data about the cluster load, job workflow, job results in the database. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Results… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Results… CMS Metacluster v.1.0: The first release was shipped in January, 2004, The system was used on the clusters of Nizhni Novgorod State University and Research Institute of Physics and Technologies (Kazan, Russian Academy of Science), The system was intensively used at International schools on parallel programming (Nizhni Novgorod, 2004-06) Итак, я рассмотрел основные возможности системы «Метакластер» и ее структуру. Теперь настало время поговорить о достигнутых на данный момент результатах. Система удовлетворяет всем требованиям, высказанным при постановке задачи. Система представляет из себя сервис операционной системы Windows NT. Сервис – это программа, написанная особым образом, которая начинает работу непосредственно при старте операционной системы, без участия пользователя, не требуя даже входа в систему. Язык реализации – C++, средство Microsoft Visual Studio 6.0. Суммарный размер кода – 6500 строк. Менеджер удаленного доступа реализован на языке PHP 4.0. Используемый сервер базы данных – MysSql. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Results… CMS Metacluster v.2.0 provides: Support of any number of clusters running under OS MS Windows, Remote access via the Web interface, Management by means of the administrator interface, Partial integration with Microsoft CCS 2003, Handling the basic set of job operations: adding, deleting, executing, Monitoring of compute nodes, Authorization system; Job storing, Accumulating job results, Scheduling and distributing jobs among clusters and among compute nodes by using the preliminary scheduling politics. Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Future Work… Short-term results: Adding co-operation with clusters running under Unix-type OS, Co-operating with third-party cluster management systems (Microsoft Compute Cluster Server 2003, PBS, …), Updating the Web-interface, Developing command line and graphical interfaces, Providing API (Web-services, COM etc.),… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Future Work Short-term results: Accumulating statistics of using compute resources, generating reports, Optimizing scheduling politics, Securing communication protocols between the Metacluster components, etc… Nizhni Novgorod, 2006 Development of High Performance Computing Environment

Development of High Performance Computing Environment Contacts Andrew Senin, Nizhniy Novgorod State University, 603950, Nizhniy Novgorod, Gagarina avenue, 23, E-mail: Andrew.Senin@itlab.unn.ru Nizhni Novgorod, 2006 Development of High Performance Computing Environment