J. J. Rehr & R.C. Albers Rev. Mod. Phys. 72, 621 (2000) A “cluster to cloud” story: Naturally parallel Each CPU calculates a few points in the energy grid.

Slides:

Advertisements

Similar presentations

Parallel Processing with OpenMP

Advertisements

S.Chechelnitskiy / SFU Simon Fraser Running CE and SE in a XEN virtualized environment S.Chechelnitskiy Simon Fraser University CHEP 2007 September 6 th.

Beowulf Supercomputer System Lee, Jung won CS843.

The Who, What, Why and How of High Performance Computing Applications in the Cloud Soheila Abrishami 1.

C-Store: Data Management in the Cloud Jianlin Feng School of Software SUN YAT-SEN UNIVERSITY Jun 5, 2009.

Low Cost, Scalable Proteomics Data Analysis Using Amazon's Cloud Computing Services and Open Source Search Algorithms Brian D. Halligan, Ph.D. Medical.

1. Topics Is Cloud Computing the way to go? ARC ABM Review Configuration Basics Setting up the ARC Cloud-Based ABM Hardware Configuration Software Configuration.

Cloud Computing at Amazon’s EC2 Joe Steele

 Amazon Web Services announced the launch of Cluster Compute Instances for Amazon EC2.  Which aims to provide high-bandwidth, low- latency instances.

Authors: Thilina Gunarathne, Tak-Lon Wu, Judy Qiu, Geoffrey Fox Publish: HPDC'10, June 20–25, 2010, Chicago, Illinois, USA ACM Speaker: Jia Bao Lin.

S oftware T ools for A cademics and R esearchers MASSACHUSETTS INSTITUTE OF TECHNOLOGY Office of Educational Innovation and Technology.

Parallel/Concurrent Programming on the SGI Altix Conley Read January 25, 2007 UC Riverside, Department of Computer Science.

Jharrod LaFon (HPC-3) Jim Williams (HPC-3) 2011 Computer System, Cluster, and Networking Summer Institute Russell Husted (MTU) Derek Walker (NCA&TSU) Povi.

High Performance Computing (HPC) at Center for Information Communication and Technology in UTM.

07/14/08. 2 Points Introduction. Cluster and Supercomputers. Cluster Types and Advantages. Our Cluster. Cluster Performance. Cluster Computer for Basic.

CPP Staff - 30 CPP Staff - 30 FCIPT Staff - 35 IPR Staff IPR Staff ITER-India Staff ITER-India Staff Research Areas: 1.Studies.

High Performance Computing with cloud Xu Tong. About the topic Why HPC(high performance computing) used on cloud What’s the difference between cloud and.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

Amazon EC2 Quick Start adapted from EC2_GetStarted.html.

Cloud Computing using AWS C. Edward Chow. Advanced Internet & Web Systems chow2 Outline of the Talk Introduction to Cloud Computing AWS EC2 EC2 API A.

Cloud Computing Brian Curran, Sabatino DeRico, Mike Delisa, Mudit Goel, Jon Guagenti, Jess Caso, Greg Flynn.

Virtualization Concept. Virtualization  Real: it exists, you can see it.  Transparent: it exists, you cannot see it  Virtual: it does not exist, you.

CS 221 – May 13 Review chapter 1 Lab – Show me your C programs – Black spaghetti – connect remaining machines – Be able to ping, ssh, and transfer files.

Parallel Computing The Bad News –Hardware is not getting faster fast enough –Too many architectures –Existing architectures are too specific –Programs.

Parallel Processing LAB NO 1.

INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 7 2/23/2015.

SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.

Accessing the Amazon Elastic Compute Cloud (EC2) Angadh Singh Jerome Braun.

Tools and Utilities for parallel and serial codes in ENEA-GRID environment CRESCO Project: Salvatore Raia SubProject I.2 C.R. ENEA-Portici. 11/12/2007.

NSF PI Meeting: The Science of the Cloud, Mar 17-18, 2011 Waterview Conference Center, Arlington, VA Cloud Computing Clusters for Scientific Research*

Introduction to Cloud Computing

Using Biological Cyberinfrastructure Scaling Science and People: Applications in Data Storage, HPC, Cloud Analysis, and Bioinformatics Training Scaling.

Presented by: Sanketh Beerabbi University of Central Florida COP Cloud Computing.

Advantages Cloud Computing. customers only pay for the access and interfaces that they need. The customer buys only the services they need Cost Advantages.

Webscale Computing Mike Culver Amazon Web Services.

Penn State, August 2013 Cloud-WIEN2k A Scientific Cloud Computing Platform for Condensed Matter Physics K. Jorissen University of Washington, Seattle,

A Framework for Elastic Execution of Existing MPI Programs Aarthi Raveendran Tekin Bicer Gagan Agrawal 1.

SUMA: A Scientific Metacomputer Cardinale, Yudith Figueira, Carlos Hernández, Emilio Baquero, Eduardo Berbín, Luis Bouza, Roberto Gamess, Eric García,

Infrastructure Clouds MAGIC Meeting Chris Hill, MIT September 7 th 2011 interested in modeling of Earth and planetary systems to both understand basic.

Cloud Computing Instructor: Pankaj Mehra Teaching Assistant: Raghav Gautam Lec. 5 April 22, 2010 ISM 158.

Loosely Coupled Parallelism: Clusters. Context We have studied older archictures for loosely coupled parallelism, such as mesh’s, hypercubes etc, which.

Using Virtual Servers for the CERN Windows infrastructure Emmanuel Ormancey, Alberto Pace CERN, Information Technology Department.

SALSA HPC Group School of Informatics and Computing Indiana University.

PARALLEL COMPUTING overview What is Parallel Computing? Traditionally, software has been written for serial computation: To be run on a single computer.

The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Atmosphere.

GVis: Grid-enabled Interactive Visualization State Key Laboratory. of CAD&CG Zhejiang University, Hangzhou

Scalable Systems Software for Terascale Computer Centers Coordinator: Al Geist Participating Organizations ORNL ANL LBNL.

Company small business cloud solution Client UNIVERSITY OF BEDFORDSHIRE.

7. Grid Computing Systems and Resource Management

| nectar.org.au NECTAR TRAINING Module 4 From PC To Cloud or HPC.

3/12/2013Computer Engg, IIT(BHU)1 CLOUD COMPUTING-1.

CIP HPC CIP - HPC HPC = High Performance Computer It’s not a regular computer, it’s bigger, faster, more powerful, and more.

Transforming Science Through Data-driven Discovery Tools and Services Workshop Atmosphere Joslynn Lee – Data Science Educator Cold Spring Harbor Laboratory,

Multicore Applications in Physics and Biochemical Research Hristo Iliev Faculty of Physics Sofia University “St. Kliment Ohridski” 3 rd Balkan Conference.

Creating Grid Resources for Undergraduate Coursework John N. Huffman Brown University Richard Repasky Indiana University Joseph Rinkovsky Indiana University.

June 28, 2016 Cluster Management for Non-XSEDE Systems Barbara Hallock, Eric Coulter, Sudhakar Pamidighantam.

HPC need and potential of ANSYS CFD and mechanical products at CERN A. Rakai EN-CV-PJ2 5/4/2016.

Auto-scaling Services on Amazon EC2. Auto-scaling of Services (1/2) Write the service, deploy it on the cloud, and it simply scales Simply store the state.

Computing Clusters, Grids and Clouds Globus data service

White Rose Grid Infrastructure Overview

Dag Toppe Larsen UiB/CERN CERN,

Dag Toppe Larsen UiB/CERN CERN,

ECRG High-Performance Computing Seminar

Tools and Services Workshop Overview of Atmosphere

CRESCO Project: Salvatore Raia

CernVM Status Report Predrag Buncic (CERN/PH-SFT).

Bin Ren, Gagan Agrawal, Brad Chamberlain, Steve Deitz

Different types of Linux installation

Using and Building Infrastructure Clouds for Science

Question 1 How are you going to provide language and/or library (or other?) support in Fortran, C/C++, or another language for massively parallel programming.

Presentation transcript:

J. J. Rehr & R.C. Albers Rev. Mod. Phys. 72, 621 (2000) A “cluster to cloud” story: Naturally parallel Each CPU calculates a few points in the energy grid Loosely coupled Very little communication between processes FEFF: Real-space Green’s function code for electronic structure, x-ray spectra, … John Rehr

Is Cloud Computing feasible for on-demand, High- Performance Computing (HPC) for scientific research in the face of declining budgets? Who is interested? Is it for everybody? What kind of code could benefit from it? How do we make it possible? Challenge of NSF Grant Disadvantages of Current HPC Approach Expensive infrastructure: Big clusters =~1000$/node + capital costs + power +cooling + … Expensive HPC staff & maintenance Need expertise in HPC to use efficiently

Advantages of CC for Scientific Computing For “casual” HPC users: –On-demand access without the need to purchase, maintain, or even understand HPCs –Lease vs. buy: lease as many as needed at ~10¢/cpu-hr –Plug & Play HPC scientific codes For developers: –Scientific codes can be optimized and pre-installed For administrators & funding agencies: –HPC access to a wider class of scientists at lower costs

Development Strategy 1.Develop AMI (Amazon Machine Image) customized for HPC scientific applications 2.Test single-instance performance 3.Develop shell-scripts that make the EC2 look and run like a local HPC cluster ( ” virtual supercomputer on a laptop ” ) 4.Test parallel performance

FEFFMPI EC2 AMI Standard Linux AMI: Fedora 8 32-bit distribution with Gnu FORTRAN compilers (gfortran and g77) AWS tools for the EC2: AMI, API and S3 tools LAM for parallel MPI codes Java Runtime Environment 6 Java Development Kit 1.6 EC2 Cluster tools FEFF8.4 serial and parallel versions JFEFF graphical interface for FEFF8.4 Custom Linux distribution replicated on each instance in cluster

Serial Performance of FEFF on EC2 Virtual machine performance similar to “real”

Xeon E5345 (2.33 GHz) ½ Opteron 2218HE (“1.3” GHz) Xeon E5345 (2.33 GHz) Serial Performance of Gasoline on EC2 Virtual Real No penalty from virtualization

1.Start cluster 2.Configure nodes Current MPI Scenario EC2 Compute Instances User interacts with control workstation MPI Master startd MPI Slave startd MPI Slave startd Control Workstation ec2_clust_*

UW EC2 Cluster Tools Tools in the local control machine NameFunctionAnalog ec2_clust_launch N Launches cluster with N instancesboot ec2_clust_connect Connect to a clusterssh ec2_clust_put Transfer data to EC2 clusterscp ec2_clust_get Transfer data from EC2 clusterscp ec2_clust_list List running clusters ec2_clust_terminate Terminate a running clustershutdown The tools hide a lot of the “ugliness”: ec2_clust_connect ssh -i /home/fer/.ec2_clust/.ec2_clust_info.7729.r- de70cdb7/key_pair _fdv.pem 27.compute-1.amazonaws.com

FEFFMPI on EC2 EC2 works well for highly parallelized applications like FEFF EC2 UW