Lightweight grid computing workshop, 3rd May 2006 WEDS: Web services Environment for Distributed Simulation James Suter Centre for Computational Science University College London Lightweight grid computing workshop, 3rd May 2006

Contents Problems with existing grid middleware Lightweight middleware? WEDS Enabling grid-based computational science Examples

Grid Computing We define grid computing as distributed computing performed transparently across multiple administrative domains. By computing, we refer to any activity involving digital information. Transparency, implying minimal complexity for the users of the technology, has been missing from all existing distributed computing infrastructures. A general computational grid should provide easy access to many different types of resources to enable users to pick and choose those required to achieve their intended scientific objectives. See: Phil Trans R Soc London A (2005)

UK and US Grid Technologies Storage devices Instruments: XMT devices, LUSI,… Grid Middleware HPC resources Scalable MD, MC, mesoscale modelling User with laptop Steering Performance control/monitoring Visualization engines VR and/or AG nodes

Grid Computing Such demands have so far proved hard to meet and as a result very few scientists have wanted to get near so-called grid technology. Important for the development of the grid as a whole that as many diverse groups of scientists begin to use the grid; it is critical that the uptake of grid technologies is outside of specialized grid-centric projects. Existing grid middleware obstructs rather than facilitates access to grid resources. This has hindered scientific progress with the grid.

Problems for users The existing toolkits have an excessively heavy set of software and administrative requirements, even for relatively simple demands from applications. Existing toolkits are painful and difficult to install and maintain. Existing standards bodies and the task forces within the UK e-Science programme are not engaging sufficiently with the applications community. The effort and time required to overcome these problems are such that many scientists are reticent to deal with grid resources. Application scientists generally use legacy applications, often written without knowledge of grid computing. Porting of these codes to the grid environment with ease of installation and facility is crucial to user uptake. Existing toolkits are painful and difficult to install and maintain, due to excessive reliance on custom-patched libraries, poor package management, and a severe lack of documentation for end-users , and run a substantial risk of producing and implementing Grid architectures which are irrelevant to the requirements of application scientists

Our solution We have therefore developed simple, lightweight Grid middleware that is "good enough" for rapid adoption, rather than waiting for a solution which will, supposedly, suit all needs. Such a toolkit must be substantially more portable, lightweight, and modular in design than current middleware. By being produced in very close collaboration with application scientists and having full documentation, this will allow end- users to port existing codes and use Grid techniques with the minimum of hassle.

Lightweight middleware What do we mean by lightweight? Minimal dependencies on third-party software Small learning-curve for new users – removing the need to learn new programming methods Interoperable with other WSRG implementations Easy to write, and so to adapt to new specs, etc. Uses WSRF::Lite (interoperable with other WSRG implementations)

Lightweight middleware OGSI::Lite/WSRF::Lite by Mark McKeown of Manchester University Lightweight OGSI/WSRF implementation, written in Perl uses existing software where possible; simple installation We have developed WEDS--a Web service hosting Environment for Distributed Simulation

About WEDS Developed to make life easier for application scientists Easy to deploy – sits inside a WSRF::Lite container, has no additional software requirements For use within an administrative domain Provides all the tools and glue required to: expose an unaltered binary application as a service create and interact with service instances Broker service manages creation of services, to load balance across a pool of machines See Coveney et al., 2004, NeSC Tech Rpt

WEDS Architecture Client Broker Machine Service Service Factory Each resource runs a WSRF::Lite container containing a WEDS machine service and factory services for each hosted application. Each machine that a user wishes to use is registered with a broker service The user contacts the broker with the details of the job to run The broker match-makes the job details with the capabilities advertised by each machine service and decides where to invoke the service The broker passes back the contact details of the service instance to the client Client Broker Machine Service Service Factory Wrapper Service The script takes the name of the simulation to execute and the arguments to pass to it, and passes these details to the broker, which selects a machine on which to deploy the process. A file service is also launched on the user's machine, its WS-address passed to the broker along the other information. The wrapper starts on the remote resource, and attempts to resolve each argument passed as if it were an input file by negotiating with the file service; successfully resolved files are copied to the remote simulation working directory. The simulation is then launched. The script waits for the simulation to complete, transferring standard output from the process as it is produced, and then transfers all of the produced output files back to the user's local working directory. Invoked Application Managed resource

More complex uses of WEDS Under this framework, an executable can be deployed remotely using a 'standard' script as if it were being run locally. An object-oriented representation of a simulation wrapper can be written in Perl, which allows scripts (or users) to start, visualize and control simulations with commands as simple as $g=$sim->getValue('gravity'); or $sim->setValue('pressure', 200); The term 'loosely coupled' is often used and well describes the way a library of Web services could be easily connected to run simulations or analysers, which automatically exchange their results between each other

The Polysteer Application Developed by Daniel Mason (Imperial; ReG partner) New Monte Carlo polymer simulation code Create as many chain conformations as possible Task farming of configuration generation Equilibration is difficult from arbitrary start point Need to watch chains relax Attach visualisation client Monte Carlo moves are complex Modify parameters on-the-fly to optimise efficiency Attach steering client The polysteer program has been developed by Daniel Mason, working at Oxford University with Adrian Sutton. As the relaxation timescales for long-chain polymers may be ms or more, the generation of typical conformations for modelling and simulation is practically inaccessible using molecular dynamics. We therefore have designed a Monte Carlo code, which, with the correct set of moves, can explore the configuration space of considerably larger polymer chains. Our code has the flexibility to manipulate (basically all the) runtime parameters using a steering client. Equilibration is further complicated by the fact that since there is hard-core repulsion between atoms, any move which reduces overlap will be accepted in the early stages of relaxation. As a single move is a strongly correlated movement of atoms, this can lead to large steps away from the relaxed configuration. A visualiser is needed to restart simulations which have been pushed in this way far from equilibrium.

Lightweight hosting of Polysteer application

Visualisation Visualisation client attaches to a running simulation Data transfer via files using ReG steering library Fortran main code to Java visualiser Showing each atom is unreadable Potentials treat CHx, Bz as single entities We visualise ellipsoids rather than spheres Our visualiser represents the atoms as they are in the main program. The simplification from the every-atom picture ( above ) to ellipsoidal model ( below ) is extremely useful for determining the important points in the simulation. Data is passed to the visualiser as .xyz files, or directly using sockets and the reg steering interface.

Work flow example: Fourier transform of molecular dynamics output A user script coordinates a workflow between molecular dynamics, Fourier transform and visualization services. The binary data moves directly between elements, not via the user's machine, halving bandwidth demands. Example: MD program LAMMPS interfacing with a Fourier Transform analysis program molecular dynamics simulation service could feed its output to a Fourier transform service or an autocorrelation function service, which feed their results to a visualization service, each component in the 'pipeline' or 'workflow' being essentially unaware of its context.

Steering example: LB2D LB2D is a workstation class lattice-Boltzmann code written in C simulating multiphase fluid flow in two dimensions. Preparing the code for use with WEDS involved linking to a steering library A single broker and machine service allows a single simulation process to be executed. The simulation involved stepping the value of the parameter g_cc from 2, for the first 200 time-steps, to 0, for the second 200 time-steps, and back to 2 for the final 200 time-steps, giving a simulation run 600 time-steps long in total. A steering and visualization API (Application Programming Interface) has been written in C, allowing simulation code to easily import and export data in the well-established XDR (eXternal Data Representation)14 format. A comprehensive RealityGrid steering and visualization API already exists (Pickles et al. 2004a,b), but it was felt that its complexity and its implementation, based around OGSI rather than WSRF, made it unsuitable for incorporation into WEDS. The underlying mechanism for the WEDS steering API is file based. Such an implementation has advantages, both in speed and in its capacity for asynchronous transfer of large datasets, but also has limitations; most obviously, the wrapper service as described in §2 must have access to the simulation working directory. The client side interface has been designed so that a possible future socket-based development of the protocol could expose the same methods as the current API, ensuring compatibility between current and future versions. The data transferred consists of an XDR binary stream with an ASCII header, which describes the content of the file and the dimensionality and type of the data it contains. It is data of this type that is transferred from program to program to operate a workflow. Tools have been written to convert files between this WEDS format and others which are more widely used, such as HDF5, VTK (Visualization ToolKit) and plain XDR. 6. An example implementation with LB2D LB2D is a workstation class lattice-Boltzmann code written in C, running in two dimensions. Preparing the code for use with WEDS involved linking to the steering library, as discussed in §5. Around 130 functional lines of code were added to allow 26 quantities to be visualized--including the complete simulation state, for checkpointing--and a subset of nine of those to be steered. A particular simulation will now be presented, along with its results and the short script that steered it. A single broker and machine service were set up to allow a single simulation process to be executed. The simulation involved stepping the value of the parameter g_cc from 2, for the first 200 time-steps, to 0, for the second 200 time-steps, and back to 2 for the final 200 time-steps, giving a simulation run 600 time-steps long in total (see figure 2). During the middle third of the simulation, the two fluids are completely miscible, allowing a relaxation of the clean separation achieved after the first third, and the establishment of a second length-scale in the system when the interfluid force is switched on again for the final third. To realize this simulation plan, an extra steerable and visualizable parameter was added to the code, pausetime, having the property that the simulation would always be paused if the current time-step tpausetime. To code a particular time-critical steering or visualization event, the value of pausetime was set to force the simulation to halt upon reaching a certain point. It would then receive further instructions, the final instruction being a greater value of pausetime. The methods used to steer and visualize the simulation are setValue and getValue, as presented in §2. The simulation object, $sim, is created by invoking a constructor with details of the simulation name ('LB2D') and broker WS-address; the broker is contacted and a wrapper created on the machine resource we have made available. The wrapper WS-address is stored within the $sim object so that it can be contacted directly in any calls we make to our simulation. The first interaction with the simulation process is to set the maximum time, 600, at which point we want the process to terminate. The tmax variable has this property. 1. $sim->setValue('tmax', 600); We then tell the simulation to pause when it reaches time-step 200 and enter a loop until this time is met, sleeping briefly between each call to check status so that we do not affect the performance of our simulation. 1. $sim->setValue('pausetime', 200); 2. while ($sim->getValue('t')<200) {sleep 3}. The value of g_cc is then set to zero, allowing the fluids to mix, and pausetime set to 400. The simulation process then restarts, and we again enter a loop until this new time is reached. 1. $sim->setValue('g_cc', 0); 2. $sim->setValue('pausetime', 400); 3. while ($sim->getValue('t')<400) {sleep 3}. When this point is reached, the forces between the different fluids are reinstated by setting g_cc back to its default value of 2. The simulation is unpaused by setting pausetime to -1--meaning do not ever pause--and the simulation continues to process until it reaches the termination time we set at the beginning, time-step 600. 1. $sim->setValue('g_cc', 2); 2. $sim->setValue('pausetime', -1).

Summary Lightweight middleware such as WEDS greatly facilitates deployment of applications on grids We’re now working with several “computational user communities” from physics through to biology All our middleware will be in the public domain: Download from www.realitygrid.org/WEDS

Acknowledgements Prof. Peter Coveney, Radhika Saskena, Matt Harvey, Laurent Pedesseau, Mark Mc Keown, Stephen Pickles, Daniel Mason, Jonathan Chin EPSRC OMII