The Astrophysics Simulation Collaboratory Portal Case Study of a Grid-Enabled Application Environment HPDC-10 San Francisco Michael Russell, Gabrielle Allen, Greg Daues, Ian Foster, Tom Goodale, Edward Seidel, Jason Novotny, John Shalf, Wai-Mo Suen, Gregor von Laszewski

Collaboratory: Fully- integrated one-stop-shop for your research community CACTUS is a freely available, modular, portable and manageable environment for collaboratively developing parallel, high- performance multi-dimensional simulations

Cactus Community DLR Geophysics (Bosl) Numerical Relativity Community Cornell Crack prop. NASA NS GC Livermore SDSS (Szalay) Intel Microsoft Clemson “Egrid” NCSA, ANL, SDSC AEI Cactus Group (Allen) NSF KDI (Suen) EU Network (Seidel) Astrophysics (Zeus) US Grid Forum DFN Gigabit (Seidel) “GRADS” (Kennedy, Foster, Dongarra, et al) ChemEng (Bishop) San Diego, GMD, Cornell Berkeley (Shalf)

NSF-KDI Project Astrophysics Simulation Collaboratory = Physics: Accretion-induced collapse of Neutron Stars = Numerics: ; Full evolution of Einstein’s equations in 3D (AEI) ; Relativistic Hydrodynamics (WashU) ; Zeus MHD (NCSA) ; Adaptive Mesh Refinement (Rutgers) = Resources: Distributed all over the US and Europe (NCSA, SDSC, Rutgers, ANL, UNM, Golm, Garching) = People: Also distributed worldwide ; Different fields of expertise ; Different levels of expertise

Complex Workflow Acquire Code Modules Configure And Build Bugs? Report/Fix bugs Set Params Initial Data Run Many Test Jobs Steer, Kill, Or restart Correct? Select largest Rsrc and run For a week Remove vis and steer Novel Results? Archive TB’s Of Data Select and Stage data to Storage array Regression Rmt Vis Data Mine Observation Y N Y N N Y Papers Nobel Prizes

Users View Has To Be Easy!

ASC Portal (Major Components) = Web based portal interface ; Simplicity for users ; Available e-commerce infrastructure ; Central location for collaborative interactions = Globus Grid Services ; Widely deployed at HPC sites in US and Europe ; Uniform security model (single-sign-on) ; Uniform access to essential services (batch queues, files, information) = Cactus Computational ToolKit ; Modular/parallel multi-physics framework ; Multi-platform (NT, Unix, Linux ; Alpha, x86, IA64,MIPS,SR8000) ; Multilanguage (C, C++, F90, F77, Java, Python, Perl) ; Robust toolkit (Numerics, Parallel I/O, Remote Steering and Vis, PETSC) ; AMR Integration (GrACE) ; Revision control and active bug tracking support infrastructure

ASC Portal (Web Components) = Apache ; SSL/Secure HTTP = Tomcat ; Java Server Pages Automation = Java CoG ; 100% Pure Java implementation of Globus Services and API’s = GPDK ; Wraps CoG in JSP-amenable Java Beans ; Provides higher level services like connection pooling = SQL/RDBMS ; Portal internal state information (rather than serialization or flatfiles) ; Proven scalability for server replication ; Independent of implementation of portal automation

Inside the Portal Webserver

Manage Compute Resources 1. Select machines in ASC Grid2. Add to user’s profile

Cactus Software 1. Select project and thorns2. Run cvs checkout

Build Applications 1. Edit configuration2. Run “make…”

Parameter Files 1. Edit parameter file

Cactus Simulations 1. Edit simulation2. Run simulation3. Steer simulation

Thorn HTTPD Thorn which allows simulation any to act as its own web server Connect to simulation from any browser anywhere Monitor run: parameters, basic visualization,... Change steerable parameters See running example at Wireless remote viz, monitoring and steering

Remote Visualization IsoSurfaces and Geodesics Contour plots (download) Grid Functions Streaming HDF5 Amira Isoview LCAVision Amira LCA Vision OpenDX

Go! Clone job with steered parameter Queue time over, find new machine Add more resources Found a horizon, try out excision Look for horizon Calculate/Output Grav. Waves Calculate/Output Invariants Find best resources Free CPUs!! NCSA SDSC RZG SDSC LRZArchive data Dynamic Grid Computing Can already do some of this! Track !

GridLab: New Paradigms for Dynamic Grids = Dynamic Distributed apps with Grid-threads (gthreads) = Code should be aware of its environment ; What resources are out there NOW, and what is their current state? ; What is my allocation? ; What is the bandwidth/latency between sites? = Code should be able to make decisions on its own ; A slow part of my simulation can run asynchronously…spawn it off! ; New, more powerful resources just became available…migrate there! ; Machine went down…reconfigure and recover! ; Need more memory…get it by adding more machines! = Code should be able to publish this information to Portal for tracking, monitoring, steering… ; Unexpected event…notify users! ; Collaborators from around the world all connect, examine simulation.

GridLab: Enabling Dynamic Grid Applications Large EU Project Under Negotiation with EC… AEI, Lecce, Poznan, Brno, Amsterdam, ZIB- Berlin, Cardiff, Paderborn, SZTAKI, Compaq, Sun, Chicago, ISI, Wisconsin Grid Application Toolkit for both Apps and Infrastructure 20 positions opening up!! Come join us!