Dynamic Grid Computing: The Cactus Worm The Egrid Collaboration Represented by: Ed Seidel Albert Einstein Institute Egrid@egrid.org cactus@cactuscode.org 1

Grid Computing: a new paradigm Computational Resources Scattered Across the World Compute servers Handhelds File servers Networks Playstations, etc… How to take advantage of this for scientific/engineering simulations? Harness multiple sites and devices Simulations at new level of complexity and scale 2

Components for Grid Computing Resources: Egrid (www.egrid.org) A “Virtual Organization” in Europe for Grid Computing Over a dozen sites across Europe Many different machines Infrastructure: Globus Metacomputing Toolkit Develops fundamental technologies needed to build computational grids.  Security: logins, data transfer Communication Information (GRIS, GIIS) 3

Components for Grid Computing Application: Cactus Computational Toolkit Modular Toolkit for Parallel Computation Numerical/Computational Infrastructure to solve PDE’s Enables Grid applications of many types… www.cactuscode.org 4

Grid Computing Scenarios: The Vision Distributed Computing: Sit here, compute there, monitor and steer… Managing intelligent parameter surveys jobs to new machines, e.g. analysis tasks Dynamic staging … seeking out and moving to faster/larger/cheaper machines as they become available Scripting capabilities (management, launching new jobs, checking out new code, etc) Dynamic load balancing 5

Application Code as Information Server/Gatherer Code should be aware of its environment What resources are out there? What is their current state? What is my allocation? What is the bandwidth and latency between sites? How can I adjust myself to take advantage of the current state? Code should be able to make decisions on its own A slow part of my simulation can run asynchronously…spawn it off! New, more powerful resources just became available…migrate there! Code should be able to publish this information to central server for tracking, monitoring, steering... Cactus has modules to enable this for any application 6

Cactus Worm: Illustration of basic scenario Cactus simulation starts, launched from a portal Queries a Grid Information Server, finds available resources Migrates itself to next site Uses some logic to choose next resource Securely starts up remote simulation Transfers memory contents to remote simulation (using streaming HDF5, scp, GASS, whatever…) Registers new location to server, terminates previous simulation User tracks and monitors with continuous remote viz and control using thorn http, streaming data, etc...… Continues around Europe, and so on… If we can do this, much of what we want can be done! 7

Grand Picture: we are very close! Viz of data from previous simulations in SF café Remote steering and monitoring from airport Remote Viz in St Louis Remote Viz and steering from Berlin DataGrid/DPSS Downsampling IsoSurfaces http Streaming Data T3E: Garching Origin: NCSA Globus Simulations launched from Cactus Portal Grid enabled Cactus runs on distributed machines 8

Next… Tonight: Global Grid Forum BOF Tomorrow: manchester Booth at 10:30 Thanks to Sun for sponsoring us!