1 Cactus in a nutshell... n Cactus facilitates parallel code design, it enables platform independent computations and encourages collaborative code development between different groups. n Cactus was developed as a 3D parallel high-performance tool for Numerical Relativity which would allow easy development and maintenance especially for collaborations. n Modular structure: user code (a “thorn”) plugs into compact core code (“flesh”) through an extensible interface. Cactus allows a programmer to concentrate on his area of expertise. n User friendly: thorns can be written in C, C++ or Fortran 77/90, communication programming is optionally abstracted away from the user. n Cactus offers access to a range of computational capabilities as e.g. parallel I/O, checkpoiting, remote steering & visualization, performance analysis and provides easy access to many cutting edge software packages as Globus HDF5, PETSc. n Developed, supported and expanded by the Albert-Einstein-Institute in collaboration with many other institutes and universities worldwide, released under GNU Opensource license. Information & Download:

2 “Big mesh sizes” Collaboration technology needed! n A scientist’s view on a large scale computation problem: Initial Data Evolution Algorithms Analysis routines (Bettter be Fortran) “Easy job submission” “Large Data Output” “Parallel would be great” Scientist cannot be required to become experts in IT technology.

3 Collaboration technology needed! n An IT expert’s view on a large scale computation problem: Next Gen. Highspeed Comm. Layers High-performance parallel I/O Code instrumentation + steering Load scheduling Interactive Viz. Metacomputing “Programmers, use this!” IT experts cannot write the applications that make use of their technology

4 Large Scale Computations Require a mix of Technologies and Expertise n Scientific/Engineering Components The “physics” in the game n Numerical Algorithm Components Finite differences? Finite elements? Structured meshes? n Different Computational Components l Architectures and plattforms l Parallelism (MPI, PVM, OpenMP, ???) l parallel I/O, Data Management, Queueing Systems l Visualization of all that comes out! How can experts work together effectively? Need a code environment that encourages collaborative HPC.

5 Cactus “glue” Cactus „glue“ Initial Data Evolution Algorithms Elliptic Equation Solver AMR/FMR Parallelisation MPI/PVM/ OpenMP/Shmem (High Performance-) Input/Output Checkpointing Code Instrumentation / Bencharking Interactive Steering Interactiv Visualisation Cactus’ goal is to provide a modular framework that allows each technology to be of impact to other unrelated fields.

6 Cactus “glue” in more detail n Cactus “Flesh” (core) written in C n User “thorns” (modules) grouped in arrangements, can be written in F77, F90, C, C++, (Java) n Thorn -- Flesh interface fixed in 3 files written in CCL (Cactus Configuration Language): l interface.ccl: Grid Variables, Arrays, Scalars (integer, real, logical, complex) l param.ccl: Parameters and their allowed values l schedule.ccl: Scheduling of routines, dynamic memory allocation n Object orientated features (public, private, protected variables, inheritance) define scope of objects and ensure non-contaminating modularity. n Flesh provides standard interfaces for Parallelization, Interpolation, Reduction, I/O, etc. Underlying technology can be replaced invisibly to the user

7 How to use Cactus n Develop thorns, according to the interface rules l interface.ccl + schedule.ccl + param.ccl l Specify calling sequence of the routines for different problems and algorithms (schedule.ccl) For a given code: strip off parameter parsing / memory allocation / output n At compile time: l Specify which thorns are to be compiled into executable. l Redundancy allowed: Multiple thorns providing the same functionality. E.g. different Evolutions systems, IO methods. n At run time: l In the parameter file, turn on the thorns needed for your simulation.

8 Cactus architectures & environments Cactus applications developed on a standard workstation or laptop can be seamlesly run on a cluster or supercomputer. Supported architectures: n SGI Origin n Cray T3E n Compaq DEC Alpha n Linux (IA32/64,PPC,NOWs) n Windows NT n IBM SP2 n Hitachi n HP n SUN Ultra. Homogenous + heterogenous networks: “What is a supercomputer and what is not ? “ Computing in a resource continuum

9 Cactus, Egrid, KDI n The European Grid Forum (Egrid) aims at the cooperative use of the distributed computing resources that are accessible via wide area networks. n 18 participating organisations of the 1 st Egrid workshop in Poznan chose to install Cactus as a first testbed application.

10 HPC in Numerical Relativity n SC 97: “Distributed Spacetime”, Intercontinental Metacomputing at AEI / Argonne / Garching / NCSA (512 node T3E calculation with live visualization San Jose) n SC98: “Colliding Neutron Stars across the Atlantic”, Connecting T3E’s in Berlin, Garching, San Diego n SC99: Cactus used by several independent demos as underlying framework. n In Fall 99: