The Hungarian ClusterGRID Project Péter Stefán research associate NIIF/HUNGARNET

Slides:



Advertisements
Similar presentations
INSTALLING LINUX.  Identify the proper Hardware  Methods for installing Linux  Determine a purpose for the Linux Machine  Linux File Systems  Linux.
Advertisements

What to expect.  Linux  Windows Server (2008 or 2012)
Network+ Guide to Networks, Fourth Edition
© UC Regents 2010 Extending Rocks Clusters into Amazon EC2 Using Condor Philip Papadopoulos, Ph.D University of California, San Diego San Diego Supercomputer.
Exploiting SCI in the MultiOS management system Ronan Cunniffe Brian Coghlan SCIEurope’ AUG-2000.
Network+ Guide to Networks, Fourth Edition Chapter 1 An Introduction to Networking.
NPACI: National Partnership for Advanced Computational Infrastructure August 17-21, 1998 NPACI Parallel Computing Institute 1 Cluster Archtectures and.
Universidad Politécnica de Baja California. Juan P. Navarro Sanchez 9th level English Teacher: Alejandra Acosta The Beowulf Project.
+ Virtualization in Clusters and Grids Dr. Lizhe Wang.
Peter Stefan, NIIF 29 June, 2007, Amsterdam, The Netherlands NIIF Storage Services Collaboration on Storage Services.
Paper on Best implemented scientific concept for E-Governance Virtual Machine By Nitin V. Choudhari, DIO,NIC,Akola By Nitin V. Choudhari, DIO,NIC,Akola.
Rocks cluster : a cluster oriented linux distribution or how to install a computer cluster in a day.
Distributed Systems Early Examples. Projects NOW – a Network Of Workstations University of California, Berkely Terminated about 1997 after demonstrating.
Grid Toolkits Globus, Condor, BOINC, Xgrid Young Suk Moon.
Virtualization Concept. Virtualization  Real: it exists, you can see it.  Transparent: it exists, you cannot see it  Virtual: it does not exist, you.
Wireless Grid Computing A Prototype Wireless Grid Grant Gifford Mark Hempstead April 30, 2003.
Paper on Best implemented scientific concept for E-Governance projects Virtual Machine By Nitin V. Choudhari, DIO,NIC,Akola.
Hungarian Supercomputing GRID
Chapter 6 Operating System Support. This chapter describes how middleware is supported by the operating system facilities at the nodes of a distributed.
Appendix B Planning a Virtualization Strategy for Exchange Server 2010.
MSc. Miriel Martín Mesa, DIC, UCLV. The idea Installing a High Performance Cluster in the UCLV, using professional servers with open source operating.
Eric Keller, Evan Green Princeton University PRESTO /22/08 Virtualizing the Data Plane Through Source Code Merging.
Computing and the Web Operating Systems. Overview n What is an Operating System n Booting the Computer n User Interfaces n Files and File Management n.
Central Reconstruction System on the RHIC Linux Farm in Brookhaven Laboratory HEPIX - BNL October 19, 2004 Tomasz Wlodek - BNL.
Computer and Automation Research Institute Hungarian Academy of Sciences Automatic checkpoint of CONDOR-PVM applications by P-GRADE Jozsef Kovacs, Peter.
N*Grid – Korean Grid Research Initiative Funded by Government (Ministry of Information and Communication) 5 Years from 2002 to million US$ Including.
The Grid System Design Liu Xiangrui Beijing Institute of Technology.
InstantGrid: A Framework for On- Demand Grid Point Construction R.S.C. Ho, K.K. Yin, D.C.M. Lee, D.H.F. Hung, C.L. Wang, and F.C.M. Lau Dept. of Computer.
1 SIAC 2000 Program. 2 SIAC 2000 at a Glance AMLunchPMDinner SunCondor MonNOWHPCGlobusClusters TuePVMMPIClustersHPVM WedCondorHPVM.
Condor: High-throughput Computing From Clusters to Grid Computing P. Kacsuk – M. Livny MTA SYTAKI – Univ. of Wisconsin-Madison
Support in setting up a non-grid Atlas Tier 3 Doug Benjamin Duke University.
Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”
Quick Introduction to NorduGrid Oxana Smirnova 4 th Nordic LHC Workshop November 23, 2001, Stockholm.
© 2004 IBM Corporation IBM ^ z/VM Design considerations > Security > Performance (SIE)
OPERATING SYSTEM SUPPORT DISTRIBUTED SYSTEMS CHAPTER 6 Lawrence Heyman July 8, 2002.
GAAIN Virtual Appliances: Virtual Machine Technology for Scientific Data Analysis Arihant Patawari USC Stevens Neuroimaging and Informatics Institute July.
Terminal Servers in Schools A second life for your older computers.
Presented By: Samreen Tahir Coda is a network file system and a descendent of the Andrew File System 2. It was designed to be: Highly Highly secure Available.
Production Grid Challenges in Hungary Péter Stefán Ferenc Szalai Gábor Vitéz NIIF/HUNGARNET.
EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,
Brunel University, School of Engineering and Design, Uxbridge, UB8 3PH, UK Henry Nebrensky (not a systems manager) SIRE Group.
Virtual Private Grid (VPG) : A Command Shell for Utilizing Remote Machines Efficiently Kenji Kaneda, Kenjiro Taura, Akinori Yonezawa Department of Computer.
Cluster Software Overview
Grid Computing Framework A Java framework for managed modular distributed parallel computing.
Internet2 AdvCollab Apps 1 Access Grid Vision To create virtual spaces where distributed people can work together. Challenges:
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.
7. Grid Computing Systems and Resource Management
LCG LCG-1 Deployment and usage experience Lev Shamardin SINP MSU, Moscow
Globus and PlanetLab Resource Management Solutions Compared M. Ripeanu, M. Bowman, J. Chase, I. Foster, M. Milenkovic Presented by Dionysis Logothetis.
How can Computer Networks Support National Grid Initiatives? Tamás Máray Péter Stefán NIIF/HUNGARNET.
Construction methods and monitoring in meta-cluster systems LIT, JINR Korenkov V.V, Mitsyn V.V, Chkhaberidze D.V, Belyakov D.V.
Hands-On Virtual Computing
Tool Integration with Data and Computation Grid “Grid Wizard 2”
15-Feb-02Steve Traylen, RAL WP6 Test Bed Report1 RAL/UK WP6 Test Bed Report Steve Traylen, WP6 PPGRID/RAL, UK
Distributed Computing Systems CSCI 6900/4900. Review Definition & characteristics of distributed systems Distributed system organization Design goals.
CEG 2400 FALL 2012 Linux/UNIX Network Operating Systems.
Background Computer System Architectures Computer System Software.
Collaborative Tools for the Grid V.N Alexandrov S. Mehmood Hasan.
SYSTEM MODELS FOR ADVANCED COMPUTING Jhashuva. U 1 Asst. Prof CSE
1 Chapter 2: Operating-System Structures Services Interface provided to users & programmers –System calls (programmer access) –User level access to system.
An Introduction to Local Area Networks An Overview of Peer-to-Peer and Server-Based Models.
Cumulus - dynamic cluster available under Clusterix
Agenda Hardware Virtualization Concepts
Definition of Distributed System
Computing Board Report CHIPP Plenary Meeting
SUBMITTED BY: NAIMISHYA ATRI(7TH SEM) IT BRANCH
Статус ГРИД-кластера ИЯФ СО РАН.
University of Technology
Welcome To : Group 1 VC Presentation
Network+ Guide to Networks, Fourth Edition
Presentation transcript:

The Hungarian ClusterGRID Project Péter Stefán research associate NIIF/HUNGARNET

What is it about? Project background Project goal Technical construction, solutions applied The grid in operation Relationship with the institutes Our users Summary

Background of the Project 2002 spring-summer Ministry of Education procurement project 99 computer cabinets have been installed at Hungarian universities and public libraries 2002 summer-autumn ClusterGRID Technical Board was set up Pilot project is finished in December

The Goal of the Project The installed labs serve two purposes: the institutes’ own purposes (education, service), grid purposes. The goal of the project is to build a countrywide, homogeneous grid system which is in production, and which is capable to solve computing problems of the real users.

Technical Solution Applied Main properties of the solution applied are as follows: Temporal and spatial separation of the two functions. i.e. separate OS, partition, network segment. 3-layer hierarchical architecture, with central entry point. Easy to manage system. In the current implementation the system is centralized, but on demand it can be de-centralized.

Technical Solution Applied The 3 “building blocks” are as follows: Private network structure in grid mode using MPLS VPN, and VLAN techniques. Network root file system on the workers. Network wake-on LAN technique.

Network Structure

Function of Different Elements Workers: carry out computation, memory, NFS root clients. Local masters: DHCP, TFTP, NFS services, routing within the private network, local scheduling.

Function of Different Elements Central entry point: authentication service, DNS service, front-end service. Grid OS: RedHat 7.3, Debian Linux

Function of Different Elements The central machine and the local masters operate continuously. The boot of the workers is an interesting process! The worker boot can be done from any kind of media, such as CD-ROM, floppy disk. The preferred is the network boot opportunity. Switching between different modes must be as automated as possible.

Grid Software Job scheduler: Condor vs. SUN Grid Engine (SGE). Parallel Virtual Machine (PVM) support. Globus system is not used!

Current Implementation There are 8 Hungarian Institutes involved in the system: 5 is outside of Budapest. 400 nodes are enabled for grid operation. The number of nodes has been increasing. About 200 workers in continuous operation.

Current Implementation

Relationship with the Institutes Each institute must provide three responsible persons: IT manager, UNIX admin, network admin. The whole process is controlled by contract. Physical conditions: Capability of continuous operation without supervision. VLAN handling capability. 8 GB disk partition on each worker. Dedicated server machine.

Our Users There are four users so far. The environmental conditions are not as sophisticated as in the case of supercomputers, but the system is usable. The first scientific international journal paper has been created in the field of chemistry. Anybody within the Hungarian Academic Community and foreign partners are welcomed to the system.

Further Information

Summary Project background The goal of the project Technical construction, solutions applied The operating grid Relationship with the institutes Our users Future plans