1. Lightweight grid computing workshop, 3rd May 2006
WEDS: Web services Environment for Distributed Simulation James Suter Centre for Computational Science University College London
Lightweight grid computing workshop, 3rd May 2006

2. Contents Problems with existing grid middleware
Lightweight middleware?
WEDS
Enabling grid-based computational science
Examples

3. Grid Computing
We define grid computing as distributed computing performed transparently across multiple administrative domains.
By computing, we refer to any activity involving digital information. Transparency, implying minimal complexity for the users of the technology, has been missing from all existing distributed computing infrastructures.
A general computational grid should provide easy access to many different types of resources to enable users to pick and choose those required to achieve their intended scientific objectives.
See: Phil Trans R Soc London A (2005)

4. UK and US Grid Technologies
Storage devices
Instruments: XMT devices, LUSI,…
Grid Middleware
HPC resources
Scalable MD, MC, mesoscale modelling
User with laptop
Steering
Performance control/monitoring
Visualization engines
VR and/or AG nodes

5. Grid Computing
Such demands have so far proved hard to meet and as a result very few scientists have wanted to get near so-called grid technology.
Important for the development of the grid as a whole that as many diverse groups of scientists begin to use the grid; it is critical that the uptake of grid technologies is outside of specialized grid-centric projects.
Existing grid middleware obstructs rather than facilitates access to grid resources. This has hindered scientific progress with the grid.

6. Problems for users
The existing toolkits have an excessively heavy set of software and administrative requirements, even for relatively simple demands from applications.
Existing toolkits are painful and difficult to install and maintain.
Existing standards bodies and the task forces within the UK e-Science programme are not engaging sufficiently with the applications community.
The effort and time required to overcome these problems are such that many scientists are reticent to deal with grid resources.
Application scientists generally use legacy applications, often written without knowledge of grid computing. Porting of these codes to the grid environment with ease of installation and facility is crucial to user uptake.
Existing toolkits are painful and difficult to install and maintain, due to excessive reliance on custom-patched libraries, poor package management, and a severe lack of documentation for end-users
, and run a substantial risk of producing and implementing Grid architectures which are irrelevant to the requirements of application scientists

7. Our solution
We have therefore developed simple, lightweight Grid middleware that is "good enough" for rapid adoption, rather than waiting for a solution which will, supposedly, suit all needs.
Such a toolkit must be
substantially more portable, lightweight, and modular in design than current middleware.
By being produced in very close collaboration with application scientists and having full documentation, this will allow end- users to port existing codes and use Grid techniques with the minimum of hassle.

8. Lightweight middleware
What do we mean by lightweight?
Minimal dependencies on third-party software
Small learning-curve for new users – removing the need to learn new programming methods
Interoperable with other WSRG implementations
Easy to write, and so to adapt to new specs, etc.
Uses WSRF::Lite (interoperable with other WSRG implementations)

9. Lightweight middleware
OGSI::Lite/WSRF::Lite
by Mark McKeown of Manchester University
Lightweight OGSI/WSRF implementation, written in Perl
uses existing software where possible; simple installation
We have developed WEDS--a Web service hosting Environment for Distributed Simulation

10. About WEDS Developed to make life easier for application scientists
Easy to deploy – sits inside a WSRF::Lite container, has no additional software requirements
For use within an administrative domain
Provides all the tools and glue required to:
expose an unaltered binary application as a service
create and interact with service instances
Broker service manages creation of services, to load balance across a pool of machines
See Coveney et al., 2004, NeSC Tech Rpt

11. WEDS Architecture Client Broker Machine Service Service Factory
Each resource runs a WSRF::Lite container containing a WEDS machine service and factory services for each hosted application.
Each machine that a user wishes to use is registered with a broker service
The user contacts the broker with the details of the job to run
The broker match-makes the job details with the capabilities advertised by each machine service and decides where to invoke the service
The broker passes back the contact details of the service instance to the client
Client
Broker
Machine Service
Service Factory
Wrapper Service
The script takes the name of the simulation to execute and the arguments to pass to it, and passes these details to the broker, which selects a machine on which to deploy the process. A file service is also launched on the user's machine, its WS-address passed to the broker along the other information. The wrapper starts on the remote resource, and attempts to resolve each argument passed as if it were an input file by negotiating with the file service; successfully resolved files are copied to the remote simulation working directory. The simulation is then launched. The script waits for the simulation to complete, transferring standard output from the process as it is produced, and then transfers all of the produced output files back to the user's local working directory.
Invoked
Application
Managed resource

12. More complex uses of WEDS
Under this framework, an executable can be deployed remotely using a 'standard' script as if it were being run locally.
An object-oriented representation of a simulation wrapper can be written in Perl, which allows scripts (or users) to start, visualize and control simulations with commands as simple as
$g=$sim->getValue('gravity'); or
$sim->setValue('pressure', 200);
The term 'loosely coupled' is often used and well describes the way a library of Web services could be easily connected to run simulations or analysers, which automatically exchange their results between each other

13. The Polysteer Application
Developed by Daniel Mason (Imperial; ReG partner)
New Monte Carlo polymer simulation code
Create as many chain conformations as possible
Task farming of configuration generation
Equilibration is difficult from arbitrary start point
Need to watch chains relax
Attach visualisation client
Monte Carlo moves are complex
Modify parameters on-the-fly to optimise efficiency
Attach steering client
The polysteer program has been developed by Daniel Mason, working at Oxford University with Adrian Sutton.
As the relaxation timescales for long-chain polymers may be ms or more, the generation of typical conformations for modelling and simulation is practically inaccessible using molecular dynamics.
We therefore have designed a Monte Carlo code, which, with the correct set of moves, can explore the configuration space of considerably larger polymer chains. Our code has the flexibility to manipulate (basically all the) runtime parameters using a steering client.
Equilibration is further complicated by the fact that since there is hard-core repulsion between atoms, any move which reduces overlap will be accepted in the early stages of relaxation. As a single move is a strongly correlated movement of atoms, this can lead to large steps away from the relaxed configuration. A visualiser is needed to restart simulations which have been pushed in this way far from equilibrium.

14. Lightweight hosting of Polysteer application

15. Visualisation Visualisation client attaches to a running simulation
Data transfer via files using ReG steering library
Fortran main code to Java visualiser
Showing each atom is unreadable
Potentials treat CHx, Bz as single entities
We visualise ellipsoids rather than spheres
Our visualiser represents the atoms as they are in the main program.
The simplification from the every-atom picture ( above ) to ellipsoidal model ( below ) is extremely useful for determining the important points in the simulation.
Data is passed to the visualiser as .xyz files, or directly using sockets and the reg steering interface.

16. Work flow example: Fourier transform of molecular dynamics output
A user script coordinates a workflow between molecular dynamics, Fourier transform and visualization services. The binary data moves directly between elements, not via the user's machine, halving bandwidth demands.
Example: MD program LAMMPS interfacing with a Fourier Transform analysis program
molecular dynamics simulation service could feed its output to a Fourier transform service or an autocorrelation function service, which feed their results to a visualization service, each component in the 'pipeline' or 'workflow' being essentially unaware of its context.

17. Steering example: LB2D
LB2D is a workstation class lattice-Boltzmann code written in C simulating multiphase fluid flow in two dimensions. Preparing the code for use with WEDS involved linking to a steering library
A single broker and machine service allows a single simulation process to be executed. The simulation involved stepping the value of the parameter g_cc from 2, for the first 200 time-steps, to 0, for the second 200 time-steps, and back to 2 for the final 200 time-steps, giving a simulation run 600 time-steps long in total.
A steering and visualization API (Application Programming Interface) has been written in C, allowing simulation code to easily import and export data in the well-established XDR (eXternal Data Representation)14 format. A comprehensive RealityGrid steering and visualization API already exists (Pickles et al. 2004a,b), but it was felt that its complexity and its implementation, based around OGSI rather than WSRF, made it unsuitable for incorporation into WEDS.
The underlying mechanism for the WEDS steering API is file based. Such an implementation has advantages, both in speed and in its capacity for asynchronous transfer of large datasets, but also has limitations; most obviously, the wrapper service as described in §2 must have access to the simulation working directory. The client side interface has been designed so that a possible future socket-based development of the protocol could expose the same methods as the current API, ensuring compatibility between current and future versions.
The data transferred consists of an XDR binary stream with an ASCII header, which describes the content of the file and the dimensionality and type of the data it contains. It is data of this type that is transferred from program to program to operate a workflow. Tools have been written to convert files between this WEDS format and others which are more widely used, such as HDF5, VTK (Visualization ToolKit) and plain XDR.
6. An example implementation with LB2D
LB2D is a workstation class lattice-Boltzmann code written in C, running in two dimensions. Preparing the code for use with WEDS involved linking to the steering library, as discussed in §5. Around 130 functional lines of code were added to allow 26 quantities to be visualized--including the complete simulation state, for checkpointing--and a subset of nine of those to be steered.
A particular simulation will now be presented, along with its results and the short script that steered it. A single broker and machine service were set up to allow a single simulation process to be executed. The simulation involved stepping the value of the parameter g_cc from 2, for the first 200 time-steps, to 0, for the second 200 time-steps, and back to 2 for the final 200 time-steps, giving a simulation run 600 time-steps long in total (see figure 2). During the middle third of the simulation, the two fluids are completely miscible, allowing a relaxation of the clean separation achieved after the first third, and the establishment of a second length-scale in the system when the interfluid force is switched on again for the final third.
To realize this simulation plan, an extra steerable and visualizable parameter was added to the code, pausetime, having the property that the simulation would always be paused if the current time-step tpausetime. To code a particular time-critical steering or visualization event, the value of pausetime was set to force the simulation to halt upon reaching a certain point. It would then receive further instructions, the final instruction being a greater value of pausetime.
The methods used to steer and visualize the simulation are setValue and getValue, as presented in §2. The simulation object, $sim, is created by invoking a constructor with details of the simulation name ('LB2D') and broker WS-address; the broker is contacted and a wrapper created on the machine resource we have made available. The wrapper WS-address is stored within the $sim object so that it can be contacted directly in any calls we make to our simulation.
The first interaction with the simulation process is to set the maximum time, 600, at which point we want the process to terminate. The tmax variable has this property.
1. $sim->setValue('tmax', 600);
We then tell the simulation to pause when it reaches time-step 200 and enter a loop until this time is met, sleeping briefly between each call to check status so that we do not affect the performance of our simulation.
1. $sim->setValue('pausetime', 200);
2. while ($sim->getValue('t')<200) {sleep 3}.
The value of g_cc is then set to zero, allowing the fluids to mix, and pausetime set to 400. The simulation process then restarts, and we again enter a loop until this new time is reached.
1. $sim->setValue('g_cc', 0);
2. $sim->setValue('pausetime', 400);
3. while ($sim->getValue('t')<400) {sleep 3}.
When this point is reached, the forces between the different fluids are reinstated by setting g_cc back to its default value of 2. The simulation is unpaused by setting pausetime to -1--meaning do not ever pause--and the simulation continues to process until it reaches the termination time we set at the beginning, time-step 600.
1. $sim->setValue('g_cc', 2);
2. $sim->setValue('pausetime', -1).

18. Summary
Lightweight middleware such as WEDS greatly facilitates deployment of applications on grids
We’re now working with several “computational user communities” from physics through to biology
All our middleware will be in the public domain: Download from

19. Acknowledgements
Prof. Peter Coveney, Radhika Saskena, Matt Harvey, Laurent Pedesseau, Mark Mc Keown, Stephen Pickles, Daniel Mason, Jonathan Chin
EPSRC
OMII