การติดตั้งและทดสอบการทำคลัสเต อร์เสมือนบน Xen, ROCKS, และไท ยกริด Roll Implementation of Virtualization Clusters based on Xen, ROCKS, and ThaiGrid Roll.

Slides:



Advertisements
Similar presentations
Virtualization Dr. Michael L. Collard
Advertisements

Virtual Machine Technology Dr. Gregor von Laszewski Dr. Lizhe Wang.
Virtualisation From the Bottom Up From storage to application.
Virtualization and Cloud Computing. Definition Virtualization is the ability to run multiple operating systems on a single physical system and share the.
Information Technology Center Introduction to High Performance Computing at KFUPM.
Introduction to Virtualization
Xen and the Art of Virtualization A paper from the University of Cambridge, presented by Charlie Schluting For CS533 at Portland State University.
Virtual Machines Measure Up John Staton Karsten Steinhaeuser University of Notre Dame December 15, 2005 Graduate Operating Systems, Fall 2005 Final Project.
Microsoft Virtual Server 2005 Product Overview Mikael Nyström – TrueSec AB MVP Windows Server – Setup/Deployment Mikael Nyström – TrueSec AB MVP Windows.
Cloud Computing and Virtualization Sorav Bansal CloudCamp 2010 IIT Delhi.
A. Frank - P. Weisberg Operating Systems Structure of Operating Systems.
Virtualization for Cloud Computing
VIRTUALISATION OF HADOOP CLUSTERS Dr G Sudha Sadasivam Assistant Professor Department of CSE PSGCT.
1 MASTERING (VIRTUAL) NETWORKS A Case Study of Virtualizing Internet Lab Avin Chen Borokhovich Michael Goldfeld Arik.
Distributed Systems CS Virtualization- Overview Lecture 22, Dec 4, 2013 Mohammad Hammoud 1.
Virtualization Technology Prof D M Dhamdhere CSE Department IIT Bombay Moving towards Virtualization… Department of Computer Science and Engineering, IIT.
Tanenbaum 8.3 See references
Virtualization Concept. Virtualization  Real: it exists, you can see it.  Transparent: it exists, you cannot see it  Virtual: it does not exist, you.
Operating System Virtualization
Paper on Best implemented scientific concept for E-Governance projects Virtual Machine By Nitin V. Choudhari, DIO,NIC,Akola.
An Introduction to Xen Prof. Chih-Hung Wu
Sumit Kumar Archana Kumar Group # 4 CSE 591 : Virtualization and Cloud Computing.
Microkernels, virtualization, exokernels Tutorial 1 – CSC469.
SAIGONTECH COPPERATIVE EDUCATION NETWORKING Spring 2010 Seminar #1 VIRTUALIZATION EVERYWHERE.
SAIGONTECH COPPERATIVE EDUCATION NETWORKING Spring 2009 Seminar #1 VIRTUALIZATION EVERYWHERE.
A Cloud is a type of parallel and distributed system consisting of a collection of inter- connected and virtualized computers that are dynamically provisioned.
So, Jung-ki Distributed Computing System LAB School of Computer Science and Engineering Seoul National University Implementation of Package Management.
1.  PRAGMA Grid test-bed : Shares clusters which managed by multiple sites Realizes a large-scale computational environment. › Expects as a platform.
Benefits: Increased server utilization Reduced IT TCO Improved IT agility.
Xen Overview for Campus Grids Andrew Warfield University of Cambridge Computer Laboratory.
Secure & flexible monitoring of virtual machine University of Mazandran Science & Tecnology By : Esmaill Khanlarpour January.
Improving Network I/O Virtualization for Cloud Computing.
Virtualization: Not Just For Servers Hollis Blanchard PowerPC kernel hacker.
VirtualBox What you need to know to build a Virtual Machine.
Operating Systems ECE344 Ashvin Goel ECE University of Toronto OS Design.
Politecnico di Torino Dipartimento di Automatica ed Informatica TORSEC Group Performance of Xen’s Secured Virtual Networks Emanuele Cesena Paolo Carlo.
COMS E Cloud Computing and Data Center Networking Sambit Sahu
High Performance Computing on Virtualized Environments Ganesh Thiagarajan Fall 2014 Instructor: Yuzhe(Richard) Tang Syracuse University.
Power-Aware Scheduling of Virtual Machines in DVFS-enabled Clusters
Server Virtualization
 Virtual machine systems: simulators for multiple copies of a machine on itself.  Virtual machine (VM): the simulated machine.  Virtual machine monitor.
EVGM081 Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management Mechanism for Grid Computing Environments Takahiro Hirofuchi,
Introduction to virtualization
A. Frank - P. Weisberg Operating Systems Structure of Operating Systems.
1 Grid Activity Summary » Grid Testbed » CFD Application » Virtualization » Information Grid » Grid CA.
Full and Para Virtualization
Operating-System Structures
Cloud Computing Lecture 5-6 Muhammad Ahmad Jan.
Cloud Computing – UNIT - II. VIRTUALIZATION Virtualization Hiding the reality The mantra of smart computing is to intelligently hide the reality Binary->
Unit 2 VIRTUALISATION. Unit 2 - Syllabus Basics of Virtualization Types of Virtualization Implementation Levels of Virtualization Virtualization Structures.
1 PERFORMANCE DIFFERENTIATION OF NETWORK I/O in XEN by Kuriakose Mathew ( )‏ under the supervision of Prof. Purushottam Kulkarni and Prof. Varsha.
Workspace Management Services Kate Keahey Argonne National Laboratory.
Open Source Virtualization Andrey Meganov RHCA, RHCX Consultant / VDEL
Virtualization Neependra Khare
Virtualization for Cloud Computing
Virtualization.
Virtual Machine Monitors
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING
Is Virtualization ready for End-to-End Application Performance?
Presented by Yoon-Soo Lee
Virtualization Dr. Michael L. Collard
Virtualization overview
Virtual Servers.
OS Virtualization.
Virtualization Layer Virtual Hardware Virtual Networking
Virtualization Techniques
Outline Chapter 2 (cont) OS Design OS structure
CSE 451: Operating Systems Autumn Module 24 Virtual Machine Monitors
Virtualization Dr. S. R. Ahmed.
Presentation transcript:

การติดตั้งและทดสอบการทำคลัสเต อร์เสมือนบน Xen, ROCKS, และไท ยกริด Roll Implementation of Virtualization Clusters based on Xen, ROCKS, and ThaiGrid Roll By: Supakit Prueksaaroon 1, Wittaya Konghaped 2, Vara Varavithya 2, and Sornthep Vannarat 1 1 Large-scale Simulation Research Laboratory, National Electronics and Computer Technology Center 2 Department of Electrical Engineering, Faculty of Engineering, King Mongkut’s Institute of Technology North Bangkok Present by Wittaya Konghaped Department of Electrical Engineering, Faculty of Engineering, King Mongkut’s Institute of Technology North Bangkok

Agenda Introduction to Virtualization technology ROCKS Cluster & ThaiGrid Roll Virtual Cluster Performance Measurement of Virtual Cluster Conclusion

Virtualization Concepts A further virtualization layer in the middle between OS and HW: Virtual Machine Monitor (VMM) Allow for multiple concurrent OS instances Modern PCs are powerful enough for creating the illusion of several OS virtual machines to run simultaneously Apps OS HW OS Apps OS VMM Apps HW Apps

Introduction to VM technology Virtual Machine have 3 technologies Emulation, full system simulation, or "full virtualization with dynamic recompilation" — the virtual machine simulates the complete hardware, allowing an unmodified OS for a completely different CPU to be run.Emulationfull system simulation Native virtualization and "full virtualization" — the virtual machine only partially simulates enough hardware to allow an unmodified OS to be run in isolation, but the guest OS must be designed for the same type of CPU. The term native virtualization is also sometimes used to designate that hardware assistance through Virtualization Technology is used.Native virtualizationVirtualization Technology Paravirtualization — the virtual machine does not simulate hardware but instead offers a special API that requires OS modifications.Paravirtualization From:

Full virtualization There is a complete functional ordering between layers Full abstraction of machine (from BIOS to disks, DMA controlers, video…) Virtualization is fully transparent: guest OS unchanged Much more complex to design and implement VMM OS Applications Hardware the interface of HW is fully abstracted

Paravirtualization Not a really hierachical ordering between layers Virtualization is similar to HW interface but neither complete nor identical Guest OSs must be modified to become VM-aware There is a potential gain in performance, due to specialization of kernel code Easier to design But carefully think about interfaces VMM OS Applications Hardware this interface is much more critical, now

Benefit of Virtualization for HPC Better utilize Hardware resources Independent to custom library Easy to create/destroy the guest OSs The key capabilities for: High Availability computing resources Migration The perfect sandbox

ROCKS The tools for create the Cluster computer. Front-node Compute-nodes ROCKS & ROLL architecture Features Cluster tools for management cluster system Support multiple software for HPC and Grid Bio-roll, Grid-roll, Intel-roll, Viz-roll and so on Easy re-build and install compute nodes

ThaiGrid Roll The complete package of ROCKS and integrated Grid packages for support Grid environment in Thailand. ThaiGrid Monitoring tools ThaiGrid CA ThaiGrid scripts Developed by “Thai National Grid Project”

Problems of Native Cluster Application porting Library incompatibility OS & Software version incompatibility Security Configuration complexity High administrative cost Heterogenious operating system support

In this work Initial Study to implement “Virtual Cluster” We implement VM based on ThaiGrid Roll Objectives: To compare overall performance and get the measurement for large scale Virtual cluster simulation. For ThaiGrid Setup can make use of CPU times for others task such as information Server, DB Server and Co- exist well with Grid Services

Goal Shared Grid resources and running concurrent with production resources (Web, Mail, DB server, etc.)

Definition Definition of this work Domain-0 or (Dom-0) is the based OS that Virtual machines are running on Domain-U or (Dom-U) is the guest OS that are running simultaneously with domain-0 OS Apps OS Domain-0 Apps HW Apps VMM Domain-U

Implement concept Build the ThaiGrid Roll and ROCKS Build the prototype Front-end node & Compute node Install Dom-U image (kernel-xenU xen3_86.1_rhel4.1) Copy Data to image file Setup the Dom-0 machine by using Fedora Core 5 Distributing the images to front-end and all compute nodes Front-end Compute

Images Maker Create the copy of prototype compute node image & configuration files “mount –o loop Edit “/etc/hosts”, “/etc/sysconfig/network”, “/etc/sysconfig/network-script/ifcfg-eth0” and so on “umount Update SGE or PBS “xm create –c config” Create imageMount image Edit all config file Umount image Update Scheduler Boot up

Experiment Details 3 nodes of Satellite Cluster IBM x336 Dual Xeon processors 2.8GHz RAM 4GB Harddisk SCSI 73GB Intel e1000 network interface Inter-connection 10/100 Mbps HPL benchmark

HPL Benchmarks

Parallel Becnhmark

Conclusion The result show the performance nearly the native linux in case of internal machine High gap-performance in case of parallel machines Virtual Cluster should be based on High-Throughput Computing Xen should be improved the I/O performance This result of this work use for initial implement of Virtual Cluster. Virtualization technology show high potential for HPC

Future Work Simulate large-scale virtualization based on this works and projection performance Scheduling move to Sandbox scheduling than job scheduling Security issues Compatibility issues Investigate high throughput application performance on virtualization technology Formulate virtualization efficiency for computation intensive scheduling

Acknowledgement Thank you ThaiGrid for support the cluster machines.

Thank you & Question

HPL CPU context switch overhead HPL parameter N=9000 NB=64

Percentage of CPU context switch overhead HPL: Overhead =~ 0.66(NumberDomain-U) SPECCPU2000: Overhead =~0.77(NumberDomain-U)