T2K Open Supercomputer Its Concept and Architecture Hiroshi Nakashima (Kyoto U.) with cooperation of Mitsuhisa Sato (U. Tsukuba) Taisuke Boku (U. Tsukuba)

Slides:

Advertisements

Similar presentations

XcalableMP: A performance-aware scalable parallel programming language and "e-science" project Mitsuhisa Sato Center for Computational Sciences, University.

Advertisements

Issues of HPC software From the experience of TH-1A Lu Yutong NUDT.

The Development of Mellanox - NVIDIA GPUDirect over InfiniBand A New Model for GPU to GPU Communications Gilad Shainer.

QCloud Queensland Cloud Data Storage and Services 27Mar2012 QCloud1.

Appro Xtreme-X Supercomputers A P P R O I N T E R N A T I O N A L I N C.

2. Computer Clusters for Scalable Parallel Computing

Today’s topics Single processors and the Memory Hierarchy

Beowulf Supercomputer System Lee, Jung won CS843.

Two Types of Supercomputer developments Yutaka Ishikawa RIKEN AICS University of Tokyo /09/03 Session.

Abstract HyFS: A Highly Available Distributed File System Jianqiang Luo, Mochan Shrestha, Lihao Xu Department of Computer Science, Wayne State University.

SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO SDSC RP Update October 21, 2010.

Information Technology Center Introduction to High Performance Computing at KFUPM.

AASPI Software Computational Environment Tim Kwiatkowski Welcome Consortium Members November 18, 2008.

LinkSCEEM-2: A computational resource for the development of Computational Sciences in the Eastern Mediterranean Mostafa Zoubi SESAME SESAME – LinkSCEEM.

Linux Clustering A way to supercomputing. What is Cluster? A group of individual computers bundled together using hardware and software in order to make.

S. Gadomski, "ATLAS computing in Geneva", journee de reflexion, 14 Sept ATLAS computing in Geneva Szymon Gadomski description of the hardware the.

SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO IEEE Symposium of Massive Storage Systems, May 3-5, 2010 Data-Intensive Solutions.

© UC Regents 2010 Extending Rocks Clusters into Amazon EC2 Using Condor Philip Papadopoulos, Ph.D University of California, San Diego San Diego Supercomputer.

HIGH PERFORMANCE COMPUTING ENVIRONMENT The High Performance Computing environment consists of high-end systems used for executing complex number crunching.

IBM RS6000/SP Overview Advanced IBM Unix computers series Multiple different configurations Available from entry level to high-end machines. POWER (1,2,3,4)

HELICS Petteri Johansson & Ilkka Uuhiniemi. HELICS COW –AMD Athlon MP 1.4Ghz –512 (2 in same computing node) –35 at top500.org –Linpack Benchmark 825.

SUMS Storage Requirement 250 TB fixed disk cache 130 TB annual increment for permanently online data 100 TB work area (not controlled by SUMS) 2 PB near-line.

An Introduction to Princeton’s New Computing Resources: IBM Blue Gene, SGI Altix, and Dell Beowulf Cluster PICASso Mini-Course October 18, 2006 Curt Hillegas.

CSC Site Update HP Nordic TIG April 2008 Janne Ignatius Marko Myllynen Dan Still.

1 HPC and the ROMS BENCHMARK Program Kate Hedstrom August 2003.

Hitachi SR8000 Supercomputer LAPPEENRANTA UNIVERSITY OF TECHNOLOGY Department of Information Technology Introduction to Parallel Computing Group.

Earth Simulator Jari Halla-aho Pekka Keränen. Architecture MIMD type distributed memory 640 Nodes, 8 vector processors each. 16GB shared memory per node.

Sun FIRE Jani Raitavuo Niko Ronkainen. Sun FIRE 15K Most powerful and scalable Up to 106 processors, 576 GB memory and 250 TB online disk storage Fireplane.

IBM RS/6000 SP POWER3 SMP Jari Jokinen Pekka Laurila.

High Performance Computing (HPC) at Center for Information Communication and Technology in UTM.

Virtual Network Servers. What is a Server? 1. A software application that provides a specific one or more services to other computers  Example: Apache.

CPP Staff - 30 CPP Staff - 30 FCIPT Staff - 35 IPR Staff IPR Staff ITER-India Staff ITER-India Staff Research Areas: 1.Studies.

HPC at IISER Pune Neet Deo System Administrator

CCS machine development plan for post- peta scale computing and Japanese the next generation supercomputer project Mitsuhisa Sato CCS, University of Tsukuba.

Sobolev Showcase Computational Mathematics and Imaging Lab.

:: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: GridKA School 2009 MPI on Grids 1 MPI On Grids September 3 rd, GridKA School 2009.

The Red Storm High Performance Computer March 19, 2008 Sue Kelly Sandia National Laboratories Abstract: Sandia National.

Jaguar Super Computer Topics Covered Introduction Architecture Location & Cost Bench Mark Results Location & Manufacturer Machines in top 500 Operating.

Taking the Complexity out of Cluster Computing Vendor Update HPC User Forum Arend Dittmer Director Product Management HPC April,

Loosely Coupled Parallelism: Clusters. Context We have studied older archictures for loosely coupled parallelism, such as mesh’s, hypercubes etc, which.

Massive Supercomputing Coping with Heterogeneity of Modern Accelerators Toshio Endo and Satoshi Matsuoka Tokyo Institute of Technology, Japan.

JLab Scientific Computing: Theory HPC & Experimental Physics Thomas Jefferson National Accelerator Facility Newport News, VA Sandy Philpott.

Parallel Programming on the SGI Origin2000 With thanks to Igor Zacharov / Benoit Marchand, SGI Taub Computer Center Technion Moshe Goldberg,

Contact: Hirofumi Amano at Kyushu Mission 40 Years of HPC Services Though the R. I. I.

SALSASALSASALSASALSA FutureGrid Venus-C June Geoffrey Fox

ITEP computing center and plans for supercomputing Plans for Tier 1 for FAIR (GSI) in ITEP  8000 cores in 3 years, in this year  Distributed.

EGEE is a project funded by the European Union under contract IST HellasGrid Hardware Tender Christos Aposkitis GRNET EGEE 3 rd parties Advanced.

Towards Exascale File I/O Yutaka Ishikawa University of Tokyo, Japan 2009/05/21.

In Large-Scale Cluster Yutaka Ishikawa Computer Science Department/Information Technology Center The University of Tokyo

2009/4/21 Third French-Japanese PAAP Workshop 1 A Volumetric 3-D FFT on Clusters of Multi-Core Processors Daisuke Takahashi University of Tsukuba, Japan.

Rob Allan Daresbury Laboratory NW-GRID Training Event 25 th January 2007 Introduction to NW-GRID R.J. Allan CCLRC Daresbury Laboratory.

Nanco: a large HPC cluster for RBNI (Russell Berrie Nanotechnology Institute) Anne Weill – Zrahia Technion,Computer Center October 2008.

| nectar.org.au NECTAR TRAINING Module 4 From PC To Cloud or HPC.

RAL PPD Tier 2 (and stuff) Site Report Rob Harper HEP SysMan 30 th June

CIP HPC CIP - HPC HPC = High Performance Computer It’s not a regular computer, it’s bigger, faster, more powerful, and more.

The Worldwide LHC Computing Grid Frédéric Hemmer IT Department Head Visit of INTEL ISEF CERN Special Award Winners 2012 Thursday, 21 st June 2012.

Pathway to Petaflops A vendor contribution Philippe Trautmann Business Development Manager HPC & Grid Global Education, Government & Healthcare.

HPC University Requirements Analysis Team Training Analysis Summary Meeting at PSC September Mary Ann Leung, Ph.D.

Multicore Applications in Physics and Biochemical Research Hristo Iliev Faculty of Physics Sofia University “St. Kliment Ohridski” 3 rd Balkan Conference.

CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Operating Systems.

CNAF - 24 September 2004 EGEE SA-1 SPACI Activity Italo Epicoco.

Early Results of Deep Learning on the Stampede2 Supercomputer

White Rose Grid Infrastructure Overview

LinkSCEEM-2: A computational resource for the development of Computational Sciences in the Eastern Mediterranean Mostafa Zoubi SESAME Outreach SESAME,

Appro Xtreme-X Supercomputers

Performance profiling and benchmark for medical physics

Hadoop Clusters Tess Fulkerson.

Early Results of Deep Learning on the Stampede2 Supercomputer

SiCortex Update IDC HPC User Forum

Presentation transcript:

T2K Open Supercomputer Its Concept and Architecture Hiroshi Nakashima (Kyoto U.) with cooperation of Mitsuhisa Sato (U. Tsukuba) Taisuke Boku (U. Tsukuba) Yutaka Ishikawa (U. Tokyo)

Fr/Jp Supercomputing WS © 2007 H.Nakashima Contents T2K Open Supercomputer Alliance Who & Why Allied ? T2K Open Supercomputer Specification How Opened ? What Specified & Not Specified? T2K Open Supercomputer Node Architecture & Site Configuration FAQ Past & Future

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Alliance Who Allied ? Kyoto U. Academic Center for Computing & Media Studies U. Tokyo Information Technology Center U. Tsukuba Center for Computational Sciences

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Alliance Why Allied ? Change procurement style from passive & vender initiative choice from product market to new scheme with active & university initiative creation from technology market aiming at high-performance solution with most advanced technologies for wide-spectrum of university users

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Specification How Opened ? Open Hardware Arch commodity device base e.g. x86, IB/Myri-10G  most efficient/advanced components from current IT market no special/dedicated components for HPC Open Software Stack open source & standard OS/middleware e.g. Linux, MPI, Globus  bases for also open source HPC middleware, libraries and tools Open to User’s Needs in addition to FP users in computational science attract INT users who are using PC clusters for searching/mining, natural lang. processing, genomic informatics,...

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Specification What Specified ? Common Requirements Hardware shared memory node of 16+ x86 cores and 32+GB ECC memory with 40+GB/sec (aggr.) bundle (even #) of inter-node links of 5+GB/sec (aggr.) on-node 250+GB RAID-1 disk (optional) and IPMI2.0 Software Red Hat or SuSE Linux Fortran, C and C++ with OpenMP and auto-parallelizer Java with JIT compiler MPI of 4+GB/sec and 8.5-  sec RT latency BLAS, LAPACK and ScaLAPACK Benchmarks (not required performance numbers) SPEC CPU2006, SPEC OMP2001, HPC Challenge (part) our own for memory, MPI and storage performance

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Specification What Not Specified ? Site Matters Hardware core/socket/node performance # of nodes and total performance inter-node connection and its performance storage system capacity and its performance power consumption Software batch job scheduler, fault tolerance, system management commercial library and applications Benchmarks performance numbers of common benchmarks site-specific application benchmarks

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Node Architecture 64bit x GFlops 8GB DDR2-667 Per-Node Peformance Rpeak = GFlops Memory =32GB 40GB/sec Links =5-8 GB/sec

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Node Architecture 64bit x GFlops 8GB DDR2-667 Per-Node Peformance Rpeak = GFlops Memory =32GB 40GB/sec Links =5-8 GB/sec Disclaimer: Pictures and Diagrams in this and following three slides are not always based on the specifications but may be example configurations. Thus the similarities of them to existing products are just incident. Performance and other numbers are not always minimum requirements or those given in the specifications.

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Configuration of U. Tsukuba # nodes= 512+ Rpeak= 75+ TFlops Memory= 16+ TB Storage System 400 TB, 10GB/sec Full-Bisection Interconnect for 128 nodes: 1TB/sec 1/8-1/1 x Full-Bisection Interconnect 0.5-4TB/sec

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Configuration of U. Tokyo # nodes= 950+ Rpeak= 140+ TFlops Memory= 30+ TB Full-Bisection Interconnect for 128 nodes: 640GB/sec x GB/sec x 3 Full-Bisection Interconnect for 512 nodes: 2.5TB/sec Storage System 1 PB, 15GB/sec

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer Configuration of Kyoto U. # nodes= 400+ Rpeak= 60+ TFlops Memory= 13 TB Storage System 800 TB, 16GB/sec Fat-node Subsystem # nodes= 6 Cores= Rpeak= 7.7 TFlops Memory= 6 TB Full-Bisection Interconnect 3.2TB/sec

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer FAQ Q1:Unified procurement? A1:No, independent three procurements and thus the winners may be different. Q2:Yet another Tsubame or TACC’s Ranger? A2:Absolutely NOT. At least, our node has a large inter-node bandwidth, 6-8 x Tsubame and 3-4 x Ranger. Q3: Prototype of 10PFlops riken.jp? A3:We don’t know, but hope some continuity. Q4:Alliance untied when systems installed? A4:No, we’ll continue R&D collaboration.

Fr/Jp Supercomputing WS © 2007 H.Nakashima T2K Open Supercomputer What Going On? Past May06:first meeting Sep.06:public announcement Oct.06:RFP released & procurement start Mar.07:draft specification released Oct.07:final specification released & bidding start Near Future Nov.07:bidding close Dec.07:contract award notice Jun.08:Open Supercomputers begin operations Far Future Open Supercomputers will retire in 2012(Kyoto), 2013(Tsukuba) and 2014(Tokyo). but our R&D collaboration will continue further.