1. Development of High Performance Computing Environment
Nizhni Novgorod State University
Faculty of Computational mathematics and Cybernetics
Information Technologies Laboratory - ITLab
Development of High Performance Computing Environment
Andrew V. Senin,
Software Department

2. Development of High Performance Computing Environment
Contents
Introduction
Problems of efficient computing resource management
Overview of some well-known cluster management systems (CMS)
Project requirements
CMS Metacluster architecture:
.NET web services
Cluster integrator
Task manager
Results
Future work
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

3. Development of High Performance Computing Environment
Introduction
High performance computing systems are required for solving many scientific and research works
Mostly HPC systems are supercomputers or clusters:
Cluster is a set of quite ordinary computers (SMP servers) connected with a high speed interconnect
Clusters is a way to achieve high performance having even a small budget
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

4. Problems of efficient computing resource management
24:7 access to the computing resource via the Internet
Scheduling – optimal distribution of jobs among computing nodes – is of great significance
An administrator can not manually regulate job executing. Thus cluster management system must do it automatically. The usual set of management functions are the following:
Providing job submissions to the cluster via the Internet,
Automatic scheduling job execution on compute nodes,
Statistics logging and generating reports, etc.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

5. Overview of some well-known cluster management systems
The most of cluster management systems (CMS) are developed for UNIX family OS
Nowadays the number of CMS systems for OS Windows is growing. Among them:
Microsoft CCS 2003 (Microsoft Compute Cluster Server 2003,
LSF (Load Sharing Facility,
Cluster CoNTroller (
The available CMS systems are not often good for research:
High cost of such systems (some systems are freely available for non-commercial use),
The source code often is not open and can not be modified to provide experimental studies,
High complexity of integration with self-made components.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

6. Development of High Performance Computing Environment
Project requirements…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

7. Project requirements…
The cluster management system “Metacluster 2.0” has to provide:
All operations to support all stages of the job execution cycle (adding to the job queue, executing, getting the current state, deleting),
An uniform way to manage a set of clusters (possibly under various OS) as an integrated computing environment,
Co-operation with third-party CMS,
Access by means of Web, command line, graphical or application program interface,
An administrator interface to control the workflow,…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

8. Development of High Performance Computing Environment
Project requirements
The cluster management system “Metacluster 2.0” has to provide:
A built-in authentication which can differ from OS procedures,
A job warehouse to store user jobs and from which jobs can be submitted for executing repeatedly,
Job scheduling among clusters and then among nodes of the selected cluster,
Accumulating outputs of the executed jobs,
Fail-safety operation even in case of some nodes emergency,
Logging the resource utilization, generating reports with various statistical data
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

9. CMS Metacluster architecture…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

10. CMS Metacluster architecture… MS .NET Web services
Web services provide:
Authentication of users (with using the database of Metacluster users),
Report generation of the system operation,
Remote access to the cluster management system for all registered users.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

11. CMS Metacluster architecture…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

12. CMS Metacluster architecture… Cluster integrator
Cluster Integrator provides:…
Storing a job queue,
Providing statistical information on demand of Web services (at least, job states, cluster load, output),
Accumulating statistical data (at least logging of operations being done),
Adding/deleting jobs on demand of Web services,…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

13. CMS Metacluster architecture… Cluster integrator
Cluster Integrator provides:
Scheduling and distributing jobs among clusters. User will be able to specify what cluster to use. Scheduling is based on the information of the job requirements, the current cluster loads, user rights, possibly previous statistical data,
Migration of jobs between clusters (in future).
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

14. CMS Metacluster architecture…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

15. CMS Metacluster architecture… Task manager
Task Manager provides:
Handling the job queue that is local for a cluster. Adding jobs can be done by the cluster integrator,
Scheduling and distributing jobs among compute nodes (various strategies can be applied to schedule jobs),
Terminating jobs on request of the Cluster integrator,…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

16. CMS Metacluster architecture… Task manager
Task Manager provides:
Monitoring the compute nodes,
Accumulating and storing outputs of the jobs (the standard output and result files),
Storing statistical data about the cluster load, job workflow, job results in the database.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

17. Development of High Performance Computing Environment
Results…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

18. Development of High Performance Computing Environment
Results…
CMS Metacluster v.1.0:
The first release was shipped in January, 2004,
The system was used on the clusters of Nizhni Novgorod State University and Research Institute of Physics and Technologies (Kazan, Russian Academy of Science),
The system was intensively used at International schools on parallel programming (Nizhni Novgorod, )
Итак, я рассмотрел основные возможности системы «Метакластер» и ее структуру. Теперь настало время поговорить о достигнутых на данный момент результатах.
Система удовлетворяет всем требованиям, высказанным при постановке задачи.
Система представляет из себя сервис операционной системы Windows NT. Сервис – это программа, написанная особым образом, которая начинает работу непосредственно при старте операционной системы, без участия пользователя, не требуя даже входа в систему.
Язык реализации – C++, средство Microsoft Visual Studio 6.0. Суммарный размер кода – 6500 строк. Менеджер удаленного доступа реализован на языке PHP 4.0. Используемый сервер базы данных – MysSql.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

19. Development of High Performance Computing Environment
Results…
CMS Metacluster v.2.0 provides:
Support of any number of clusters running under OS MS Windows,
Remote access via the Web interface,
Management by means of the administrator interface,
Partial integration with Microsoft CCS 2003,
Handling the basic set of job operations: adding, deleting, executing,
Monitoring of compute nodes,
Authorization system;
Job storing,
Accumulating job results,
Scheduling and distributing jobs among clusters and among compute nodes by using the preliminary scheduling politics.
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

20. Development of High Performance Computing Environment
Future Work…
Short-term results:
Adding co-operation with clusters running under Unix-type OS,
Co-operating with third-party cluster management systems (Microsoft Compute Cluster Server 2003, PBS, …),
Updating the Web-interface,
Developing command line and graphical interfaces,
Providing API (Web-services, COM etc.),…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

21. Development of High Performance Computing Environment
Future Work
Short-term results:
Accumulating statistics of using compute resources, generating reports,
Optimizing scheduling politics,
Securing communication protocols between the Metacluster components, etc…
Nizhni Novgorod, 2006
Development of High Performance Computing Environment

22. Development of High Performance Computing Environment
Contacts
Andrew Senin,
Nizhniy Novgorod State University,
603950, Nizhniy Novgorod,
Gagarina avenue, 23,
Nizhni Novgorod, 2006
Development of High Performance Computing Environment