1. Reading e-Science Centre
VEIGA: Very Easy Interface to Grid Applications (formerly known as "Styx Grid Services")
Jon Blower
Technical Director
Reading e-Science Centre

2. Overview of VEIGA
Aim is to make running applications on a Grid as easy as running local applications
Apps can be run from remote locations using the same command as for running the app locally
Simple workflows can be created using simple shell scripts
Data can be streamed directly from app to app
Can run applications on Condor and Globus resources
Secure

3. Example: GULP General Utility Lattice Program (Julian Gale, IC)
Used extensively in mineral sciences (e.g. NERC e-Minerals project)
Takes a single input file (input.txt) describing a crystal lattice
Outputs a single file (gulp_0001.txt) with bond lengths, Helmholtz free energy etc
Having deployed GULP on a VEIGA server, it is run by clients like so:
veigarun <server> <port> gulp
Or can create simple wrapper script so that we can just type:
gulp

4. How it works GULP is installed on the VEIGA server
VEIGA server holds an XML description of the GULP program:
<gridservice name="gulp" command="c:\gulp\gulp_win.exe">
<inputs>
<input type="file" name="input.txt"/>
</inputs>
<outputs>
<output type="file" name="gulp_0001.txt"/>
</outputs>
</gridservice>
veigarun is a completely generic VEIGA client program and performs the following tasks:
Downloads the XML description of GULP
Uses it to parse the command line to find out what input and output files to expect (in case of GULP, these happen to have fixed names)
Uploads input file(s) to VEIGA server
Starts GULP, which runs on the VEIGA server
Redirects the standard streams (stdin, stdout, stderr)
downloads the output file(s) as they are produced (doesn't wait for finish)

5. Some Benefits
Very easy to deploy and run applications in the VEIGA environment
See
Useful when application is tied to a particular platform but you want to run it from elsewhere and pretend it's local
Can easily substitute the "Grid-enabled" GULP for your local GULP
There is a GUI for GULP that wraps the GULP executable. Can simply point this to the remote GULP instead of a local installation: no change to GUI code
Can use the remote application in place of local applications in shell scripts
Easy to create "workflows"
Can use all shell constructs such as pipes etc and combine with local apps
Can arrange for data to take most direct route between servers
Can do computational steering
Simple implementation but it works
Custom clients and server plug-ins can be created using a Java API
Documentation on this is not great yet…
WSRF bindings have been created (Andrew Harrison, Cardiff): VEIGA app modelled as WS-Resource

6. VEIGA and Condor Worker nodes VEIGA Server + Condor VEIGA client
If VEIGA server is a Condor submit host, can delegate jobs to a Condor pool
Helps to load balance
Not everything runs on VEIGA server
Multiple jobs can be run with a single command:
Upload directories instead of single files:
myprog –i inputs –o outputs
Easy high-throughput computing
Demo
VEIGA
Server +
Condor
VEIGA
client

7. VEIGA and Globus myproxy NGS server VEIGA server VEIGA client
VEIGA can be used as a portal to Globus resources
E.g. NGS
Users log on to VEIGA with MyProxy user/password
VEIGA retrieves proxy certificate
Helps with Globus firewall problem
VEIGA server takes pain on behalf of client
VEIGA server used as staging post for files
Could transfer files direct from client to destination but would require client to have GridFTP or similar
VEIGA
server
VEIGA
client

8. Some technical points of (possible) interest
VEIGA uses the Styx protocol for distributed systems
JStyx – Java implementation of Styx developed at ReSC
Styx is ~20 years old
Single server process handles all traffic
File upload/download, progress monitoring, control messages
Styx is a multiplexing protocol
Can tunnel traffic through SSH
Client-server connections are persistent
Servers only need one incoming port open through firewall
Clients need no incoming ports open
VEIGA only requires a JRE – includes all other dependencies
Whole system is a 3MB download

9. Security VEIGA can be set to permit access to registered users only
Contains set of usernames/passwords. Passwords stored as secure hash, not plain text
No client certificates! (my bank does not use client certs)
All traffic can be encrypted using SSL
Server has a certificate
Or, everything can be tunnelled down SSH
Because VEIGA is a single server process with single port
Keeps some sysadmins happy
For access to GSI-protected resources, users can log into a VEIGA server with MyProxy password
Then VEIGA retrieves proxy certificate for them
VEIGA behaves like a Grid portal

10. Possible test case: GCEP
Grid for Coupled Ensemble Prediction
Run ensembles of climate simulations and get stats of results
Use resources at Reading, RAL, British Antarctic Survey (BAS), NGS
Need to share results among project partners and allow partners to submit jobs
Need a resource broker (even if very simple)
Can we do the whole thing using SSH (and VEIGA)?
No Globus, no GSI, no proxy certs (except for NGS access)
SSH very well understood and trusted by all relevant sysadmins and already installed
Shared file system with sshfs (or SRB if not)
Give me a good reason why not!

11. Conclusions VEIGA favours simplicity over efficiency
A simple tool for simple jobs
Users already know how to use it!
Remote apps can be run just like local ones
Doesn't feel like job submission
Simple workflows are created with shell scripts
Can use VEIGA as an easy-to-use "portal" to Condor and Globus
Client interface is identical in all cases
Could use same principles to launch jobs using AHE / GridSAM
VEIGA client does not know about the back end
VEIGA as AHE client plug-in?
See
(as of 4/5/06)
Contains tutorials, links to papers etc
Current release (0.2.0) does not contain everything I've talked about today – next release (0.3.0)!