Jaguar Super Computer

Topics Covered Introduction Architecture Location & Cost Bench Mark Results Location & Manufacturer Machines in top 500 Operating Systems & Supporting Languages Typical Applications Interesting sibling interesting about the assigned machine Questions

Introduction Jaguar is ranked Number 2 in the world by the Top500 List of the world’s fastest computers. Jaguar is Number 1 for open science, with major projects in climate science, astrophysics, fusion research, and many other critical areas Jaguar achieved teraflops on High-Performance Linpack benchmark, more than 85 percent of its theoretical peak of 119 teraflops Jaguar also solved the largest Linpack problem ever (a matrix problem of order 2.2 million containing nearly 5 trillion elements )

Architecture Processor : AMD X86_64 Opteron Quad Core 2300 MHz (9.2 GFlops )  Has Two of dedicated Notes – Compute nodes – Service nodes – Compute nodes : – Designed to run MPI tasks efficiently and reliably to completion. – Each compute node contains a quad-core 2.1 GHz AMD Opteron processor and 8GB of memory. Aggregate system performance is approximately 263 TF. – Approximately 600 TB are available in the scratch file system. – The compute nodes are running the Compute Node Linux (CNL) OS – Each node is connected to a Cray SeaStar router through HyperTransport, and the SeaStars are all interconnected in a 3-D-torus topology. – The resulting interconnect has very high bandwidth, low latency, and extreme scalability.

Architecture Service nodes : – Designed to provide system and I/O connectivity and also serve as login nodes from which jobs are compiled and launched. – Jaguar has a total of 7,832 XT4 compute nodes in addition to input/output (I/O) and login service nodes. – The service nodes consist of a 2.6 GHz dual-core AMD Opteron processor with 8 GB of memory. Service nodes run SuSE LINUX – Service nodes run SuSE LINUX

Architecture Scalable Interconnect: XT4 Internal Interconnect – The Cray XT5 system incorporates a high-bandwidth, low-latency interconnect based on the Cray SeaStar2+™ chip. – The interconnect directly connects all the nodes in a Cray XT5 system in a 3D torus topology, eliminating the cost and complexity of external switches and allowing for easy expandability and upgradability to future high performance Cray interconnect technologies. – Communications processing and high-speed routing on a single device. – Each communications chip is composed of a HyperTransport™ link, a Direct Memory Access (DMA) engine, a communications and management processor, a high speed interconnect router and a service port – The peak bidirectional bandwidth of each link is 9.6 GB/s with sustained bandwidth in excess of 6Gb/s

Architecture

Networking : InfiniBand network - The XT4 and XT5 parts of Jaguar are combined into a single system using an InfiniBand network that links each piece to the Spider file system - The InfiniBand network SION (or scalable I/O network) connects all major NCCS systems. File System: - Spider, a 10-petabyte Luster-based shared file system, connects to every system in the ORNL computing complex - It will serve all NCCS platforms and connect to every internal network. - Because all simulation data will reside on Spider, file transfers among computers and other systems will be un­necessary.

Architecture Scalable Software: – The Cray XT5 utilizes the Cray Linux Environment™ (CLE) – The Linux environment features a compute kernel which can be configured to match different workloads – The Cray XT5 system maintains a single root file system across all nodes,

Location & Cost Location: Oak Ridge National Laboratory (ORNL), United States – ORNL is a multi program science and technology laboratory managed for the U.S. Department of Energy by UT-Battelle, LLC Cost: Difficult to find as the computer has been evolved constantly over time.

Bench Mark -Results Benchmark results: Jaguar, a Cray XT5 system at ORNL, is #2 in the top 500 supercomputers list. It achieves a maximal Lin pack performance of PFlop/s with a theoretical peak performance of PFlop/s CoresRmax (GFlops)Rpeak(GFlops)Power

Location & Manufacturer Machines in top 500 Location in Top 5002 Machines in Top 104 Machines in Top 5010 Manufacturer Including No of Machines in Top RankComputer 2Jaguar –Cray XT5 Quadcore 2.3 Ghz Cray Inc 7NERSC/LBNL Franklin-CrayXT4Quadcore 2.3Ghz Cray Inc 8Jaguar –Cray XT4 Quadcore 2.1 Ghz Cray Inc 9RedStorm- sandia/ Cray RedStorm,3/4,2.4/2.2 Ghz dual/ Quadcore Cray Inc

Operating Systems & Supporting Languages The Compute Node Linux (CNL): is the next generation of lightweight kernels for compute nodes on the CRAY XT3 and Cray XT4 computer systems. The CNL operating system provides a runtime environment based on the SUSE SLES distribution Supported languages: CompilersC,C++,Fortran Math Libraries BLAS, FFTs, LAPACK, ScaLAPACK, SuperLU Scientific LibrariesCray scientific Library

Typical Applications Energy Assurance :Petascale leadership systems will arm sci­entists with better data to aggressively pursue renewable energy sources and more efficiently and safely exploit conven­tional energy options Climate :The potential of petascale simulations to clarify the evolution of the climate system is difficult to overstate. Nearly every aspect of climate simulation stands to benefit from the upcoming petascale Materials : In materials science, innovations made possible by petascale computing promise to bolster American competitiveness in multiple technological sectors Biology : Biologists will use petaflop computers for detailed studies showing how proteins carry out crucial tasks. Simulations of larger structures at longer timescales and finer resolution will allow exploration of protein structure and behavior

Interesting Sibling Kraken at UT. It’s a Cray XT4 but as of Feb 2 has been replaced by an XT5 and will probably be jumping up higher on the list of the top 500. It is also located very near Jaguar as it is at the University of Tennessee and is part of a partnership with ORNL the owner of the Jaguar

Interesting About The Assigned Machine Jaguar is actually the most powerful supercomputer in the world for open scientific use. Its peak performance is more than 119 trillion calculations per second (119 teraflops). To support its extraordinary concentration of computing power, the NCCS has put in place high-speed fiber-optic networks to expedite data movement

Jaquar At a Glance  Cray XT  Top500 rank: 2  1.64-petaflops peak theoretical performance  petaflops actual performance on HPL benchmark program  182,000 processing cores  AMD quad-core Opteron TM 2.3 gigahertz processors  InfiniBand network  Cray SeaStar network interface and router  362 terabytes of memory  578 terabytes per second of memory bandwidth  284 gigabytes per second of input/output bandwidth  10-gigabyte-per-second connections to ESnet and Internet 2 networks  High-Performance Storage System scales to store increasing amounts of data  Spider, a 10-petabyte Luster-based shared file system, connects to every system in the ORNL computing complex  disk subsystem transfers data at greater than 200 gigabytes per second

Questions??