What is MCSR? Who is MCSR? What Does MCSR Do? Who Does MCSR Serve? What Kinds of Accounts? Why Does Mississippi Need Supercomputers? What Kinds of Research? What Kinds of Instruction? What Kinds of Workshops? How Much Does it Cost? What Kinds of Software? What Supercomputers and Clusters?

What is MCSR? Mississippi Center for Supercomputer Research Established in 1987 by the Mississippi Legislature Mission: Enhance Computational Research Climate at Mississippi’s 8 Public Universities also: Support High Performance Computing (HPC) Education in Mississippi

What Does MCSR Do? We make Mississippi scientists: - more competitive for federal grants - more productive in research We provide extraordinary learning opportunities for Mississippi college students - instructional accounts - computing workshops - helpdesk support

Who Does MCSR Serve? MCSR serves faculty, researchers, and students at all of Mississippi’s 8 public universities Alcorn State University Jackson State University Mississippi Sate University Mississippi Valley State University The University of Southern Mississippi Delta State University The University of Mississippi Mississippi University for Women

Who Uses More MCSR Computing Time?

What Types of Computing Access is Available? Research Accounts - provided for faculty and student “researchers” - good for the duration of employment or enrollment Instructional Accounts - provided at an instructor’s request - for all students enrolled in a semester course - valid for the duration of the semester - can be “converted” to research account

Research vs. Class Accounts?

Who Uses Research Accounts?

Who Uses Instructional Accounts?

What Types of Courses Use MCSR? Over 82 University Courses Supported since 2000 C/C++, Fortran, MPI, OpenMP, MySQL, HTML, Javascript, Matlab, PHP, Perl, …. http://www.mcsr.olemiss.edu/education.php

Why Do Mississippi Researchers Need Supercomputers? Economics: researchers in a poor state like Mississippi can still make a big splash. Computational simulations are faster, cheaper, and safer than laboratory experiments alone.

What Kinds of Research @ MCSR? 90% of MCSR calculations are computational chemistry Cleanup of high explosive materials. Design of high energy density rocket fuels The chemical underpinnings of high powered lasers Mutation studies of enzyme activity Designing weather-proofing coatings for machinery Other Areas Hurricane forecasting Blast resistant Coatings Better 3-D imaging for diagnosing brain tumors

What Types of Workshops by MCSR? MCSR consultants taught over 140 free seminars in FY08. Over 60 training topics available, and growing. Fixed schedule or on-demand. Unix/programming, Math Software, Stats Software, Computational Chemistry Software

Do Researchers and Students Pay to Use MCSR? No. MCSR services are provided at no cost to the individual, department, or institution. Funded researchers may ask for priority access. Mississippi researchers may claim the value of MCSR computing services received as an in-kind contribution from their institution when seeking federal grants.

How Much Does MCSR Cost Mississippi?

What Is the Value of MCSR to Mississippi? RETURN ON INVESTMENT = GRANTS SUPPORTED / MCSR BUDGET For FY 2008: $32,832,097 / $845,535 = $38.30 per dollar spent

What Software Environments @ MCSR Programming C/C++, FORTRAN, Java, Perl, PHP, MPI… Science/Engineering PV-Wave, IMSL, GSL, Math Libraries, Abaqus Math/Statistics SAS, SPSS, Matlab, Mathematica Chemistry Gaussian, Amber, NWChem, GAMESS, CPMD, MPQC, MolPro, GROMACS

What Supercomputers @ MCSR?

Supercomputers at MCSR: sweetgum . - SGI Origin 2800 128-CPU Supercomputer - 64 GB of shared memory

Supercomputers at MCSR: redwood NSF Expiration Sep 30, 2004; HPVCI Expiration Date Dec 31, 2002. - 224 CPU SGI Altix 3700 Supercomputer - 224 GB of shared memory

Supercomputers at MCSR: mimosa 253 CPU Intel Linux Cluster – Pentium 4 Distributed memory – 500MB – 1GB per node Gigabit Ethernet

Supercomputers at MCSR: mimosa

Supercomputers at MCSR: sequoia 22 nodes 176 cores 352 GB Memory 20 TB Storage InfiniBand Interconnect

Supercomputers at MCSR: sequoia

Introduction to Parallel Programming at MCSR Message Passing Computing Processes communicate via calls to message passing library routines Programmers “parallelize” algorithm and add message calls At MCSR, this is via MPI programming with C or Fortran Sweetgum – Origin 2800 Supercomputer (128 CPUs) Mimosa – Beowulf Cluster with 253 Nodes Redwood – Altix 3700 Supercomputer (224 CPUs) Sequoia – Altix XE 310 InfiniBand Cluster (176 cores) Shared Memory Computing Threads coordinate/communicate results via shared memory variables Care must be taken not to modify the wrong memory areas At MCSR, this is via OpenMP programming with C or Fortran on sweetgum, redwood, or sequoia

Speed-Up http://www.mcsr.olemiss.edu/Engr692_TimingWorshkeet.xls

What MCSR Systems for USM Class Accounts Sweetgum MPI or OpenMP 1 to 16 CPUs Up to 900mb per CPU PBS scripts preferred #PBS –l ncpus=4 Interactive computations will be killed after 30 minutes Queues: SM-4P, SM-8P, MM-8P, MM-16P Processors: Mix of 195 MHz and 300 MHz O/S: Irix (like Unix) Compilers: SGI’s Fortran, C/C++, GNU C/C++, w/ SGI MPT

What MCSR Systems for USM Class Accounts Mimosa MPI 1 to 18 nodes 400 GB Memory per node PBS Scripts Only (no interactive jobs allowed) #PBS –l nodes=4 Queues: MCSR-CA Processors: single 1.4 GHz P4/node O/S: SUSE Linux 10.3 Compilers: Portland Group (PGI) Fortran, C/C++ w/mpich qstat –f (to find out what nodes your job is running on)

What MCSR Systems for USM Class Accounts Sequoia OpenMP (multiple processors on the same node) MPI (multiple processors on the same or different nodes) Hybrid (OpenMP within node, MPI across nodes) 1 to 4 nodes, 1 to 8 CPUs per node PBS Queues: SM-4P (for up to 4 CPUs on 1 node) MCSR-Test (up to 8 CPUs on each of 4 nodes) PBS Scripts Only (no interactive jobs allowed) #PBS –l nodes=4:ppn=8 (to run on all 8 CPUs of all 4 nodes) #PBS –l ncpus=8:select= 16 GB Memory per node (2 GB per CPU) qstat –f (to find out what nodes your job is running on)

Sequoia for USM Class Accounts To run on 4 nodes, and 8 processors per node (32 processes) #PBS –l nodes=4:ppn=8 To run on 2 nodes, and 4 processors per node (8 processes) #PBS –l nodes=2:ppn=4 To run on 1 node, and up to 8 processors (OpenMP) #PBS –l nodes=1:ppn=8 To run on 8 processors, regardless of number of nodes #PBS –l ncpus=8 To run 8 processors, with preferences about node placement #PBS –l place=scatter (distribute across as many nodes as can) #PBS –l place=pack (pack processes onto as few nodes as can) #PBS –l place=free (place processes on first available processors)

Parallel Efficiency NSF Expiration Sep 30, 2004; HPVCI Expiration Date Dec 31, 2002.