HPCx: Multi-Teraflops in the UK A World-Class Service for World-Class Research Dr Arthur Trew Director

HPCx rationale  UK academic research is increasingly dependent upon high-end compute facilities  recent technological and Grid advances highlighted the need to upgrade UK resources  HPCx objectives are thus: –aim “to deliver optimum service resulting in world- leading science” –address “the problems involved in scaling to the capability levels required”

What is HPCx?  Consortium of leading UK organisations committed to creating and managing the new resource for the next 6 years –led by University of Edinburgh  multi-stage project to deliver a world-class academic computing resource, the largest in Europe  £54M/$100M budget  Grid-enabled, a key component in the UK e-Science programme

HPCx Consortium Members  Daresbury Laboratory, CCLRC  IBM  EPCC, University of Edinburgh  UoE HPCx Ltd – wholly-owned subsidiary of the University of Edinburgh and lead contractor

 the University of Edinburgh is one of the top 5 research universities in the UK  EPCC is the leading computer centre in Europe, bridging the gap between academia and industry  … and provides both HPC and novel computing solutions to a wide range of problems and users  long experience of providing national HPC services including: –Meiko Computing Surfaces –Thinking Machines CM200 –Cray T3D/T3E

EPCC overview Academic: o National HPC facilities o Research o Support Training: o Academia o Industry European leadership: o Visitor programmes o Technology Transfer o Strategic Planning Industry: o Projects o Consultancy o Standards Technology Transfer

Daresbury Laboratory  A multi disciplinary research lab with over 500 people  Provides large-scale research facilities both for UK academic and industrial research communities  Runs the UK’s Collaborative Computational Projects  Daresbury hosts the HPCx hardware

 IBM provides the technology for HPCx  Long standing involvement in HPC including the development of a number of ASCI machines and 4 of the top dozen machines in the 21 st TOP500 list: –ASCI White: R max = 7.3 TFlop/s –SP Power3 (6656 Processors): R max = 7.3 TFlop/s –xSeries (1920 Processors): R max = 6.6 TFlop/s –HPCx (1280 processors): R max = 3.2 TFlop/s  IBM has the long term technology road map essential to a 6 year project such as HPCx

HPCx in Place

HPCx: Phase 1  System will be commissioned in three main stages Phase 1 covering consists of: –40 Regatta-H SMP nodes, 1280 processors –Peak performance 6.6 TFlop/s, with 3.4 Tflop/s sustained Linpack currently 16 th in the Top500 –1.28 TB total memory capacity –Over 50 TB of storage capacity –Double plane Colony switch with total peak bandwidth of 250 MB/s per processor

The e-Science Grid CeSC (Cambridge)

HPCx Phases 2 & 3  Phase 2 ( ) –aiming for 6 TFlop/s sustained on Linpack and 2.5 TFlop/s on sPPM –O(48) Regatta-H+ SMP nodes –interconnect upgraded to Federation switch –doubling of I/O and storage already built a cluster with 8 Regatta-H+ frames and a pre- release Federation switch undertaking a phased upgrade during 1H2004  Phase 3 ( ) –target of 12 TFlop/s sustained on Linpack –may be additional nodes or alternative technology

HPCx Science Support Outreach Life sciences New applications Applications Support Helpdesk Training Liaising with users Users Technology Software Engineering Underpinning technology Grid/e-Science Systems & Networking Flexible and responsive capability computing service Smooth transitions between phases Terascaling Capability applications Scalable algorithms Performance optimisation  18 staff in 5 dual-centre functional support teams

HPCx Status: Usage

 currently 28 active consortia, and over 450 users  Life Sciences outreach activity supported by IBM HPCx Status: Application Areas

Atomic and Molecular Physics  The UK Multiphoton, Electron Collision and Bose Einstein Condensates (MECBEC) HPC Consortium  Two flagship projects model two-electron atoms (helium) and molecules (hydrogen) exposed to intense, ultra-short laser pulses  Modelling involves the grid solution of multi- dimensional, time-dependent partial differential equations)  Visualisation techniques crucial in extracting information Simulation of Double Ionization of laser-driven helium performed at Queen’s University Belfast  Requires large amounts of computing power

Environmental Science: POLCOMS  POLCOMS is a multi-disciplinary model developed at the Proudman Oceanographic Laboratory  … a 3-D hydrodynamic model integrating coasts and oceans using a wide range of associated models A simulation of Chlorophyll density in UK waters using the POLCOMS model  POLCOMS is a step towards real time modelling of coastal zones, enabling better analysis of impacts to, and sustainability of, the marine environment

Material Science: Crystal  CRYSTAL computes electronic structure and related properties of periodic systems  Developed jointly by Daresbury and the University of Turin  A Fortran90 and MPI program that performs Hartree-Fock, density functional and other approximation calculations  On HPCx, CRYSTAL used to calculate the structure of the Crambin molecule, the largest Hartree-Fock calculation ever converged (1284 atoms) –next, the Rusticyanin molecule (6284 atoms)

Engineering: UKTC  UK Turbulence Consortium is developing world leading turbulence simulation codes using HPCx  essential that they can easily facilitate the scientific usage of the resulting data  the calculated data are transferred from HPCx to a remote site for analysis  so, the Grid is becoming increasingly important

Chemistry: RealityGrid  RealityGrid is a UK collaboration –which aims to grid-enable the realistic modelling and simulation of complex condensed matter structures at the meso and nanoscale levels  HPCx terascaling team has worked on parallelisation & optmisation of RealityGrid codes, such as LB3D  RealityGrid also uses the Grid for data transfer, computational steering, and remote visualisation –using Globus on HPCx –TeraGyroid won award at SC2003

Summary  HPCx is the new UK HPC resource –capability computing for world-leading science –largest academic computer in Europe  Series of IBM pSeries clusters –6.6TF  12TF  22TF –remaining very competitive until  Very successful first year HPCx: Supporting the Future of UK Research

Summary: Website  For further information: The front page of the HPCx website