HPC in the Cloud Guy Tel-Zur.

Slides:



Advertisements
Similar presentations
1 Automated SFTP Windows and SUN Linux and SUN. 2 Vocabulary  Client = local=the machine generating the SFTP request  Server = remote = the machine.
Advertisements

© 2010 VMware Inc. All rights reserved Amazon Web Services.
Using EC2 with HTCondor Todd L Miller 1. › Introduction › Submitting an EC2 job (user tutorial) › New features and other improvements › John Hover talking.
Lecture 12: Cloud Computing-C Amazon Web Service Tutorial.
Using ArcGIS for Server in the Amazon Cloud
Infrastructure as a Service (IaaS) Amazon EC2
Creating an AMI at Amazon’s EC2 Joe Steele
Creating a Biolinux AMI at Amazon’s EC2
PC Cluster Setup on Linux Fedora Core 5 High Performance Computing Lab Department of Computer Science and Information Engineering Tunghai University, Taichung,
Deploying an Application on the Cloud Chapter 4. Topics Your experience with Google App Engine and mine with Pop!World Web application Architecture Machine.
Matt Bertrand Building GIS Apps in the Cloud. Infrastructure - Provides computer infrastructure, typically a platform virtualization environment, as a.
Chien-Chung Shen Google Compute Engine Chien-Chung Shen
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
Installing and Setting up mongoDB replica set PREPARED BY SUDHEER KONDLA SOLUTIONS ARCHITECT.
Amazon EC2 Quick Start adapted from EC2_GetStarted.html.
SUSE Linux Enterprise Desktop Administration Chapter 1 Install SUSE Linux Enterprise Desktop 10.
Eucalyptus Virtual Machines Running Maven, Tomcat, and Mysql.
SSH Keys William Stearns
A MAZON W EB S ERVICES Reza Yousefzadeh 12/9/2014.
Installation Overview Charles (Cal) Loomis & Mohammed Airaj LAL, Univ. Paris-Sud, CNRS/IN2P October 2013.
Customized cloud platform for computing on your terms !
Remote Desktop Services Remote Desktop Connection Remote Desktop Protocol Remote Assistance Remote Server Administration T0ols.
Accessing the Amazon Elastic Compute Cloud (EC2) Angadh Singh Jerome Braun.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
VNC Greg Fankhanel Jessica Nunn Jennifer Romero. What is it? Stands for Virtual Network Computing It is remote control software which allows you to view.
K. Liu, Q. Huang, J. Xia, Z. Li, P. Lostritto, Chapter 4 How to use cloud computing?, In Spatial Cloud Computing: a practical approach, edited by.
| nectar.org.au NECTAR TRAINING Module 7 Launching and Connecting.
Customized cloud platform for computing on your terms ! Nirav Merchant
| nectar.org.au NECTAR TRAINING Module 10 Beyond the Dashboard.
How to use Remote Desktop and Remote Support. What is remote desktop? Remotely control your computer from another office, from home, or while traveling.
Linux Exercise. Download and Install the latest CentOS version and latest Ubuntu/Fedora OS. Configure a unique Host Name and a permanent IP Address for.
Unix Machine In Computer Science for Teaching Cliff Zou Spring 2015.
| nectar.org.au NECTAR TRAINING Module 7 Launching and Connecting.
Using the Weizmann Cluster Nov Overview Weizmann Cluster Connection Basics Getting a Desktop View Working on cluster machines GPU For many more.
Cloudbiolinux = Genomics resources for use on cloud platforms. Includes: – An Amazon Machine Image with lots of handy bioinformatics software pre-installed.
| nectar.org.au NECTAR TRAINING Module 10 Beyond the Dashboard.
© 2006 ITT Educational Services Inc. Linux Operating System :: Unit 3 :: Slide 1 Downloading and Installing Software yum pirut Bit Torrent rmp.
Getting Connected CPSC 1010 August 21, Connecting to the SOC Servers Why would we need to connect Work with files Transfer files from your local.
Step 1of 11 Admin Demonstrations Click Here to Start.
Experimental Deployment 1 Amazon Cloud (US-East-1* Zone) Amazon EC2 Instance Glassfish App Server MySQL Database JDBC SSH Developer HTTP Users.
Using MPI on Dept. Clusters Min LI Sep Outline Run MPI programs on single machine Run mpi programs on multiple machines Assignment 1.
Launch Amazon Instance. Amazon EC2 Amazon Elastic Compute Cloud (Amazon EC2) provides resizable computing capacity in the Amazon Web Services (AWS) cloud.
Your EC2 Instance. How to Connect to your EC2 Instance?
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
Configuring Guacamole create a folder for guacamole's configuration: mkdir /etc/guacamole create a file /etc/guacamole/guacamole.properties with the content:
© 2015 MetricStream, Inc. All Rights Reserved. AWS server provisioning © 2015 MetricStream, Inc. All Rights Reserved. By, Srikanth K & Rohit.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Overview of software tools for gLite installation & configuration.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
PuTTY Introduction to Web Programming Kirkwood Continuing Education by Fred McClurg © Copyright 2016, All Rights Reserved ssh client.
INTRODUCTION TO AMAZON WEB SERVICES (EC2). AMAZON WEB SERVICES  Services  Storage (Glacier, S3)  Compute (Elastic Compute Cloud, EC2)  Databases (Redshift,
AMAZON SIMPLE CLOUD DEPLOYMENT Matt Quill Charlie Cano.
Installation Guacamole Is a web application that provides access to desktop environments using remote desktop protocols (such as VNC or RDP); Installation.
Linux Basics Part 2. VIM Editor vi improved Installed on most Linux machines Can be a bit confusing at first... o Cheat sheets FTW Other popular editors:
The Characterisation Virtual Laboratory James Wettenhall Clayton Campus 20 th June 2012.
Introduction and Getting Started guide Alex Zlotnik Technion
Computing Clusters, Grids and Clouds Globus data service
Investigation authentication using AAF for the CVL on NeCTAR
Packaging a Structural Biology Application – imod / 3dmod
Customized cloud platform for computing on your terms !
Chapter 9 Router Configuration (Ospf, Rip) Webmin, usermin Team viewer
AWS Tutorials i2c Lab.
Tools and Services Workshop Overview of Atmosphere
Amazon Web Services as a team project platform
Webinar # April 2017 Isolates in the Cloud
Lab 7 - Topics Establishing SSH Connection Install SSH Configure SSH
OpenStack-alapú privát felhő üzemeltetés
* Introduction to Cloud computing * Introduction to OpenStack * OpenStack Design & Architecture * Demonstration of OpenStack Cloud.
Lecture 16B: Instructions on how to use Hadoop on Amazon Web Services
Windows desktop sharing
Presentation transcript:

HPC in the Cloud Guy Tel-Zur

Sign In

EC2 Dashboard

EC2 AMI for scientific computing in Python and R AMI ID: ami-84bd41ed Source:

Launching Instance

Connect using PuTTY

Install VNC server from the following packages: * xtightvncviewer * xvnc4viewer

Add a user As root: useradd telzur mkdir /home/telzur chown –R telzur /home/telzur su telzur As root: useradd telzur mkdir /home/telzur chown –R telzur /home/telzur su telzur

Passwordless ssh ssh-keygen -t dsa cd.ssh cat./id_dsa.pub >> authorized_keys2 Use a lightweight window manager such as icewm

Open ports 5800, … in the security group for VNC to work

Install a light window manager apt-get install icewm

HPC Cluster

16 cores

EBS /dev/sdh  /vol change made in /etc/fstab

Installations yum install gcc gcc-c++ gcc-gfortran Lapsck Python-devel Need more repositories: rpm -Uvh release-5-4.noarch.rpmhttp://download.fedora.redhat.com/pub/epel/5/x86_64/epel- release-5-4.noarch.rpm yum install atlas.x86_64 blas.x86_64 lapack.x86_64 yum groupinstall XFCE (startxfce4) yum install glibc-devel.i386

Test MPI

1:41 AM PDT We are currently investigating latency and error rates with EBS volumes and connectivity issues reaching EC2 instances in the US-EAST-1 region. 2:18 AM PDT We can confirm connectivity errors impacting EC2 instances and increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region. Increased error rates are affecting EBS CreateVolume API calls. We continue to work towards resolution. 2:49 AM PDT We are continuing to see connectivity errors impacting EC2 instances, increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region, and increased error rates affecting EBS CreateVolume API calls. We are also experiencing delayed launches for EBS backed EC2 instances in affected availability zones in the US-EAST-1 region. We continue to work towards resolution. 3:20 AM PDT Delayed EC2 instance launches and EBS API error rates are recovering. We're continuing to work towards full resolution. 4:09 AM PDT EBS volume latency and API errors have recovered in one of the two impacted Availability Zones in US-EAST-1. We are continuing to work to resolve the issues in the second impacted Availability Zone. The errors, which started at 12:55AM PDT, began recovering at 2:55am PDT 5:02 AM PDT Latency has recovered for a portion of the impacted EBS volumes. We are continuing to work to resolve the remaining issues with EBS volume latency and error rates in a single Availability Zone. 6:09 AM PDT EBS API errors and volume latencies in the affected availability zone remain. We are continuing to work towards resolution. Quality of Service ????

BBC

Who is affected by EC2? Source: (partial list)

Saving the computer as an AMI

2 X 16cores !

Add graphical WM (xfce4) using remote desktop (VNC) – Open port

Price $1.60/hour for this xlarge machine $4,290 for 1 year $6,590 for 3 years This is the TCO ! $1.60/hour for this xlarge machine $4,290 for 1 year $6,590 for 3 years This is the TCO !

vi /etc/ssh/sshd_config Change: password authentication from no to yes Rload sshd by /etc/init.d/sshd reload