IM&T Vacation Program Benjamin Meyer Virtualisation and Hyper-Threading in Scientific Computing.

Slides:

Advertisements

Similar presentations

Virtual Machine Technology Dr. Gregor von Laszewski Dr. Lizhe Wang.

Advertisements

1 Chapter 11: Data Centre Administration Objectives Data Centre Structure Data Centre Structure Data Centre Administration Data Centre Administration Data.

Virtualisation From the Bottom Up From storage to application.

The Development of Mellanox - NVIDIA GPUDirect over InfiniBand A New Model for GPU to GPU Communications Gilad Shainer.

Cloud Computing Mick Watson Director of ARK-Genomics The Roslin Institute.

Profit from the cloud TM Parallels Dynamic Infrastructure AndOpenStack.

Performance Analysis of Virtualization for High Performance Computing A Practical Evaluation of Hypervisor Overheads Matthew Cawood University of Cape.

Introduction to DBA.

© 2003 IBM Corporation IBM Systems and Technology Group Operating System Attributes for High Performance Computing Ken Rozendal Distinguished Engineer.

Office of Technology Operations & Planning Unlocking the Power of Server Virtualization Rebecca Astin Office of Technology Operations and Planning National.

Virtualization Ryan Cahoon Timothy Farkas Christopher Garcia Jeremy Slovak.

DatacenterMicrosoft Azure Consistency Connectivity Code.

High Performance Computing (HPC) at Center for Information Communication and Technology in UTM.

Virtualization for Cloud Computing

Panda: MapReduce Framework on GPU’s and CPU’s

VIRTUALISATION OF HADOOP CLUSTERS Dr G Sudha Sadasivam Assistant Professor Department of CSE PSGCT.

VMware vSphere 4 Introduction. Agenda VMware vSphere Virtualization Technology vMotion Storage vMotion Snapshot High Availability DRS Resource Pools Monitoring.

Design and Implementation of a Single System Image Operating System for High Performance Computing on Clusters Christine MORIN PARIS project-team, IRISA/INRIA.

SALSASALSASALSASALSA Performance Analysis of High Performance Parallel Applications on Virtualized Resources Jaliya Ekanayake and Geoffrey Fox Indiana.

Copyright © 2010 Platform Computing Corporation. All Rights Reserved.1 The CERN Cloud Computing Project William Lu, Ph.D. Platform Computing.

SSI-OSCAR A Single System Image for OSCAR Clusters Geoffroy Vallée INRIA – PARIS project team COSET-1 June 26th, 2004.

ICT Day Term 4,  Virtualisation is growing in usage.  Current CPU’s are designed to support Virtualisation.  Businesses are looking at virtualisation.

SAIGONTECH COPPERATIVE EDUCATION NETWORKING Spring 2010 Seminar #1 VIRTUALIZATION EVERYWHERE.

SAIGONTECH COPPERATIVE EDUCATION NETWORKING Spring 2009 Seminar #1 VIRTUALIZATION EVERYWHERE.

Virtualization Paul Krzyzanowski Distributed Systems Except as otherwise noted, the content of this presentation is licensed.

CERN - IT Department CH-1211 Genève 23 Switzerland t Tier0 database extensions and multi-core/64 bit studies Maria Girone, CERN IT-PSS LCG.

SUMA: A Scientific Metacomputer Cardinale, Yudith Figueira, Carlos Hernández, Emilio Baquero, Eduardo Berbín, Luis Bouza, Roberto Gamess, Eric García,

Cluster Workstations. Recently the distinction between parallel and distributed computers has become blurred with the advent of the network of workstations.

Taking the Complexity out of Cluster Computing Vendor Update HPC User Forum Arend Dittmer Director Product Management HPC April,

COMS E Cloud Computing and Data Center Networking Sambit Sahu

High Performance Computing on Virtualized Environments Ganesh Thiagarajan Fall 2014 Instructor: Yuzhe(Richard) Tang Syracuse University.

Frontiers in Massive Data Analysis Chapter 3.  Difficult to include data from multiple sources  Each organization develops a unique way of representing.

1 SIAC 2000 Program. 2 SIAC 2000 at a Glance AMLunchPMDinner SunCondor MonNOWHPCGlobusClusters TuePVMMPIClustersHPVM WedCondorHPVM.

N. GSU Slide 1 Chapter 05 Clustered Systems for Massive Parallelism N. Xiong Georgia State University.

© 2012 MELLANOX TECHNOLOGIES 1 Disruptive Technologies in HPC Interconnect HPC User Forum April 16, 2012.

Headline in Arial Bold 30pt HPC User Forum, April 2008 John Hesterberg HPC OS Directions and Requirements.

1 CMPE 511 HIGH PERFORMANCE COMPUTING CLUSTERS Dilek Demirel İşçi.

Virtual Infrastructure By: Andy Chau Farzana Mohsini Anya Mojiri Virginia Nguyen Bobby Phimmasane.

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation Ning Liu, Christopher Carothers 1.

Efficient Live Checkpointing Mechanisms for computation and memory-intensive VMs in a data center Kasidit Chanchio Vasabilab Dept of Computer Science,

Partitioned Multistack Evironments for Exascale Systems Jack Lange Assistant Professor University of Pittsburgh.

Full and Para Virtualization

| nectar.org.au NECTAR TRAINING Module 4 From PC To Cloud or HPC.

Virtualization One computer can do the job of multiple computers, by sharing the resources of a single computer across multiple environments. Turning hardware.

Cloud Computing Lecture 5-6 Muhammad Ahmad Jan.

VM vs Container Xen, KVM, VMware, etc. Hardware emulation / paravirtualization Can run different OSs on the same box Dozens of instances OS sprawl problem.

CIP HPC CIP - HPC HPC = High Performance Computer It’s not a regular computer, it’s bigger, faster, more powerful, and more.

Jaime Frey Computer Sciences Department University of Wisconsin-Madison Condor and Virtual Machines.

REMINDER Check in on the COLLABORATE mobile app Best Practices for Oracle on VMware - Deep Dive Darryl Smith Chief Database Architect Distinguished Engineer.

SYSTEM MODELS FOR ADVANCED COMPUTING Jhashuva. U 1 Asst. Prof CSE

Get the Most out of SQL Server Standard Edition Or How to be a SQL Miser.

IMPROVEMENT OF COMPUTATIONAL ABILITIES IN COMPUTING ENVIRONMENTS WITH VIRTUALIZATION TECHNOLOGIES Abstract We illustrates the ways to improve abilities.

BLUE GENE Sunitha M. Jenarius. What is Blue Gene A massively parallel supercomputer using tens of thousands of embedded PowerPC processors supporting.

A Practical Evaluation of Hypervisor Overheads Matthew Cawood Supervised by: Dr. Simon Winberg University of Cape Town Performance Analysis of Virtualization.

© 2010 VMware Inc. All rights reserved Why Virtualize? Beng-Hong Lim, VMware, Inc.

INTRODUCTION TO HIGH PERFORMANCE COMPUTING AND TERMINOLOGY.

Containers vs Others Xen, VMware, etc. ● Emulation/paravirtualization ● Can run different OSs on the same box ● Lower density/scalability ● OS sprawl problem.

XEN – The Art of Virtualisation. So what is Virtualisation? ● Makes use of spare capacity ● Run multiple instances of OSes simultaneously ● Multitasking.

Computer System Structures

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S

Organizations Are Embracing New Opportunities

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING

Presented by Yoon-Soo Lee

Virtualization Dr. Michael L. Collard

Overview Introduction VPS Understanding VPS Architecture

Chapter 15, Exploring the Digital Domain

Virtual machines benefits

Outline Chapter 2 (cont) OS Design OS structure

Presentation transcript:

IM&T Vacation Program Benjamin Meyer Virtualisation and Hyper-Threading in Scientific Computing

HPC Clusters A large set of connected computers Used for computation intensive workloads, rather than I/O orientated operations Each node runs own instance of OS

CSIRO’s Bragg Cluster 128 compute nodes with 16 CPUs each 2048 cores in total 128GB of RAM per node 384 Fermi Tesla M2050 GPUs 172,032 streaming cores

What is Virtualisation?

Hypervisor Software which allows different and multiple operating systems to run on the underlying hardware Ensures all privileged operations are appropriately handled to maintain system integrity Invisible to operating system OS thinks it is running natively VMware ESXi Hypervisor used for this project

Benefits: Heterogeneous Clusters

Benefits: Live Migration Running jobs can be moved to other hardware Allows dynamic scheduling Preemptive failure/down time evasion

Checkpointing Status of OS, application and memory are saved at intervals Allows for easy failure recovery Software debugging Clean compute Security Run time/failure isolation Clean start Benefits

Performance Comparison Floating point operations per second

Performance Comparison Updates to random memory locations per second GUPs 100% 87.1% 54.8%

Performance Comparison MPI (message passing) latency

Hyper-Threading

Thread 1 Thread 2 Thread 1 Thread 2 Physical Cores Logical Cores (seen by OS) Thread 1 Thread 2 Physical Cores non Hyper-Threaded Hyper-Threaded Time Resource 1 Resource 2 Resource 3 Resource 4 Time Resource 1 Resource 2 Resource 3 Resource 4 Hyper-Threading Example

Performance Comparison Floating point operations per second

Performance Comparison Updates to random memory locations per second

References Tim Ho (2012, Nov.). CSIRO Advanced Scientific Computing User Manual [Online]. Available: (2013, Jan.). Top500 HPC Statistics [Online]. Available: (2012, Oct.). IBM Blue Gene #1 in Supercomputing [Online]. Available: (2013). Virtualize for Efficiency, Higher Availability and Lower Costs [Online]. Available: (2012). Tuning a Linux HPC Cluster: HPC Challenge [Online]. Available: