NPACI Panel on Clusters David E. Culler Computer Science Division University of California, Berkeley

Slides:



Advertisements
Similar presentations
Evolution of High Performance Cluster Architectures David E. Culler NPACI 2001 All Hands Meeting.
Advertisements

High-Performance Clusters part 1: Performance David E. Culler Computer Science Division U.C. Berkeley PODC/SPAA Tutorial Sunday, June 28, 1998.
♦ Commodity processor with commodity inter- processor connection Clusters Pentium, Itanium, Opteron, Alpha GigE, Infiniband, Myrinet, Quadrics, SCI NEC.
2. Computer Clusters for Scalable Parallel Computing
Beowulf Supercomputer System Lee, Jung won CS843.
Building Beowulfs for High Performance Computing Duncan Grove Department of Computer Science University of Adelaide
NSF Site Visit HYDRA Using Windows Desktop Systems in Distributed Parallel Computing.
Types of Parallel Computers
CSCI-455/522 Introduction to High Performance Computing Lecture 2.
Unique Opportunities in Experimental Computer Systems Research - the Berkeley Testbeds David Culler U.C. Berkeley Grad.
Understanding Application Scaling NAS Parallel Benchmarks 2.2 on NOW and SGI Origin 2000 Frederick Wong, Rich Martin, Remzi Arpaci-Dusseau, David Wu, and.
Novell Server Linux vs. windows server 2008 By: Gabe Miller.
IBM RS6000/SP Overview Advanced IBM Unix computers series Multiple different configurations Available from entry level to high-end machines. POWER (1,2,3,4)
Millennium Overview and Status David Culler and Jim Demmel Computer Science Division
Millennium Overview and Status David Culler and Jim Demmel Computer Science Division
Millennium: Computer Systems, Computational Science and Engineering in the Large David Culler, J. Demmel, E. Brewer, J. Canny, A. Joseph, J. Landay, S.
A Comparative Study of Network Protocols & Interconnect for Cluster Computing Performance Evaluation of Fast Ethernet, Gigabit Ethernet and Myrinet.
Towards I-Space Ninja Mini-Retreat June 11, 1997 David Culler, Steve Gribble, Mark Stemm, Matt Welsh Computer Science Division U.C. Berkeley.
NOW 1 Berkeley NOW Project David E. Culler Sun Visit May 1, 1998.
Developing a Cluster Strategy for NPACI All Hands Meeting Panel Feb 11, 2000 David E. Culler Computer Science Division University of California, Berkeley.
IPPS 981 Berkeley FY98 Resource Working Group David E. Culler Computer Science Division U.C. Berkeley
PARALLEL PROCESSING The NAS Parallel Benchmarks Daniel Gross Chen Haiout.
Linux clustering Morris Law, IT Coordinator, Science Faculty, Hong Kong Baptist University.
Cluster Computing Slides by: Kale Law. Cluster Computing Definition Uses Advantages Design Types of Clusters Connection Types Physical Cluster Interconnects.
NPACI: National Partnership for Advanced Computational Infrastructure August 17-21, 1998 NPACI Parallel Computing Institute 1 Cluster Archtectures and.
1b.1 Types of Parallel Computers Two principal approaches: Shared memory multiprocessor Distributed memory multicomputer ITCS 4/5145 Parallel Programming,
Chapter 2 Computer Clusters Lecture 2.1 Overview.
1 Lecture 7: Part 2: Message Passing Multicomputers (Distributed Memory Machines)
Parallel Computing The Bad News –Hardware is not getting faster fast enough –Too many architectures –Existing architectures are too specific –Programs.
1 Computing platform Andrew A. Chien Mohsen Saneei University of Tehran.
1 In Summary Need more computing power Improve the operating speed of processors & other components constrained by the speed of light, thermodynamic laws,
14th April 1999Hepix Oxford Particle Physics Site Report Pete Gronbech Systems Manager.
CERN - European Laboratory for Particle Physics HEP Computer Farms Frédéric Hemmer CERN Information Technology Division Physics Data processing Group.
University of Illinois at Urbana-Champaign NCSA Supercluster Administration NT Cluster Group Computing and Communications Division NCSA Avneesh Pant
University of Southampton Clusters: Changing the Face of Campus Computing Kenji Takeda School of Engineering Sciences Ian Hardy Oz Parchment Southampton.
CLUSTER COMPUTING STIMI K.O. ROLL NO:53 MCA B-5. INTRODUCTION  A computer cluster is a group of tightly coupled computers that work together closely.
The MPC Parallel Computer Hardware, Low-level Protocols and Performances University P. & M. Curie (PARIS) LIP6 laboratory Olivier Glück.
Cyberinfrastructure for Distributed Rapid Response to National Emergencies Henry Neeman, Director Horst Severini, Associate Director OU Supercomputing.
Ohio Supercomputer Center Cluster Computing Overview Summer Institute for Advanced Computing August 22, 2000 Doug Johnson, OSC.
HYDRA: Using Windows Desktop Systems in Distributed Parallel Computing Arvind Gopu, Douglas Grover, David Hart, Richard Repasky, Joseph Rinkovsky, Steve.
HYDRA: Using Windows Desktop Systems in Distributed Parallel Computing Arvind Gopu, Douglas Grover, David Hart, Richard Repasky, Joseph Rinkovsky, Steve.
6/26/01High Throughput Linux Clustering at Fermilab--S. Timm 1 High Throughput Linux Clustering at Fermilab Steven C. Timm--Fermilab.
Computer Systems Lab The University of Wisconsin - Madison Department of Computer Sciences Linux Clusters David Thompson
10/22/2002Bernd Panzer-Steindel, CERN/IT1 Data Challenges and Fabric Architecture.
The II SAS Testbed Site Jan Astalos - Institute of Informatics Slovak Academy of Sciences.
22nd March 2000HEPSYSMAN Oxford Particle Physics Site Report Pete Gronbech Systems Manager.
2-3 April 2001HEPSYSMAN Oxford Particle Physics Site Report Pete Gronbech Systems Manager.
JLAB Computing Facilities Development Ian Bird Jefferson Lab 2 November 2001.
CLUSTER COMPUTING TECHNOLOGY BY-1.SACHIN YADAV 2.MADHAV SHINDE SECTION-3.
Jefferson Lab Site Report Sandy Philpott Thomas Jefferson National Accelerator Facility (formerly CEBAF - The Continuous Electron Beam Accelerator Facility)
Computing Environment The computing environment rapidly evolving ‑ you need to know not only the methods, but also How and when to apply them, Which computers.
Monte Carlo Data Production and Analysis at Bologna LHCb Bologna.
IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003.
Millennium Executive Committee Meeting David E. Culler Computer Science Division
By Chi-Chang Chen.  Cluster computing is a technique of linking two or more computers into a network (usually through a local area network) in order.
Comprehensive Scientific Support Of Large Scale Parallel Computation David Skinner, NERSC.
COMP381 by M. Hamdi 1 Clusters: Networks of WS/PC.
Randy MelenApril 14, Stanford Linear Accelerator Center Site Report April 1999 Randy Melen SLAC Computing Services/Systems HPC Team Leader.
Improving the Research Bootstrap of Condor High Throughput Computing for Non-Cluster Experts Based on Knoppix Instant Computing Technology RIKEN Genomic.
SYSTEM MODELS FOR ADVANCED COMPUTING Jhashuva. U 1 Asst. Prof CSE
Architecture of a platform for innovation and research Erik Deumens – University of Florida SC15 – Austin – Nov 17, 2015.
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING
Berkeley Cluster Projects
PC Farms & Central Data Recording
U.C. Berkeley Millennium Project
Why PC Based Control ?.
IBM Pervasive Computing Visit June 9, 1997
NCSA Supercluster Administration
The National Grid Service Mike Mineter NeSC-TOE
Cluster Computers.
Presentation transcript:

NPACI Panel on Clusters David E. Culler Computer Science Division University of California, Berkeley

1/29/99NAPCI Clusters2 Clusters have Happened IBM SGI SUN HP Microsoft...

1/29/99NAPCI Clusters3 Performance + Cost-Performance April 1998 T3E vs Intel Pentium II on NAS Benchmarks T3E-900 PII-400 BT LU SP

1/29/99NAPCI Clusters4 Berkeley (NPACI) NOW 100 Sun Ultra2 workstations Inteligent network interface –proc + mem Myrinet Network –160 MB/s per link –300 ns per hop

1/29/99NAPCI Clusters5 Beowolf Consortium LINUX PCs Fast ethernet Basic stand-alone and batch cluster cookbook

1/29/99NAPCI Clusters6 HPVM (NPACE) NT PCs Myricom Network Fast Messages LSF start-up

1/29/99NAPCI Clusters7 Berkeley Millennium PC-based Unix and NT Clusters Departmental and Campus Shared as Computational Economy Gigabit Ethernet SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME NERSC Transport Business Chemistry Astro Physics Biology Economy Math

1/29/99NAPCI Clusters8 What you get off the shelf Go to your favorite web site and order –Dual PII 450 MHz, 1 GB Mem, 36 GB disk –$9,563 at Dell 4400 CPU Hours per Quarter to your self! –$1.20 CPU hours (at 30% over 3 years) Buy 5?

1/29/99NAPCI Clusters9 Three Kinds of Clusters Throughput Clusters Availability Clusters High-Performance Parallel Clusters

1/29/99NAPCI Clusters10 Throughput Clusters Workstation / PC Farms Provide resource pool for large numbers of sequential jobs Used widely in industry –toy story on 2000 sparcstations –ultrasparc on 1000 sparcstations Can also include background on desktops –CONDOR Application specific front-ends attractive –parametric studies, monte carlo Fill cracks in parallel clusters Big farms require infrastructure

1/29/99NAPCI Clusters11 Availability Clusters Use system redundancy to mask faults –all big databases do it VAX Clusters => IBM sysplex => Wolf PackClients Disk array A Disk array B Interconnect Server A Server B

1/29/99NAPCI Clusters12 High-Performance Clusters Utilize modern system area networks and user- level communication layers to construct general purpose parallel machine from commodity parts

1/29/99NAPCI Clusters13 Emerging System Area Networks Gigabit Ethernet –price dropping, widely deployed System Area Networks –Myricom –ServerNet –Synfinity Virtual Interface Architecture –Intel/Microsoft/Compaq std based on univ. research prototypes

1/29/99NAPCI Clusters14 MPI Performance

1/29/99NAPCI Clusters15 Example: NAS Parallel Benchmarks Better node performance than the Cray T3D Better scalability than the IBM SP-2

1/29/99NAPCI Clusters16 Cluster-Wide Parallel I/O Sustain 500 MB/s disk bandwidth and 1,000 MB/s network bandwidth by driving all the disks

1/29/99NAPCI Clusters17 Software Base is Growing Technical software moving to Linux and NT –NAG, matlab, petc,... Cluster prototypes being hardened and packaged Cookbooks emerging Few cluster-integrator companies

1/29/99NAPCI Clusters18 Summary of the State of the Art Cluster designs are emerging in many areas –throughput, availability, parallel computing –technology is advancing Still immature software base –strong ties to free software movement Many small clusters by spit and baling wire Large clusters require engineering –commercial components improving Rapid pace of change presents sys. Admin challenge –not unlike the desktop problem Management tools badly needed

1/29/99NAPCI Clusters19 What Does it Mean for NPACI Where do clusters fit with computational science and engineering needs? Cycles vs Software vs Administration vs Expertise? What role should the center take? What role should partner sites have?