1. Test and Integration Current State
Santiago González de la Hoz
(on behalf of Integration Team)
Instituto de Física Corpuscular
Consejo Superior de Investigaciones Científicas, CSIC
Universitat de València
Valencia, SPAIN

2. Overview Goals For this purpose:
Implement the second-third tools-services and first application prototypes running on the international CrossGrid testbed
For this purpose:
A closed collaboration among WP1, WP2, WP3 and WP4 has been achieved (Jesus is leading the whole CrossGrid integration effort).
Since integration meeting in Poznan (July 2003)
The CrossGrid Integration team was formed.
WP2 and WP3 middleware (prototype 1) and a selective applications (prototype 0) were deployed on CrossGrid testbed
Integration Team (WP4, WP3, WP2 and WP1) is working to support of WP1 application prototype 1
Production testbed is based on LCG-1 middleware
Development testbed based on EDG2.x middleware
Now in Nicosia (January 2004): Integration Status
middleware
Integration among tasks
demo
Work for next months

3. Testbed evolution (see Jorge’s talk)
Production testbed:
From EDG to LCG-1
LCG-1 is:
VDT (Globus 2.2.4)
Information System (MDS)
Selected software from EDG 2.0
Workload Management System (RB)
EDG Data Management (RLS, LRC, …)
GLUE Schema LCG extensions
RB obtains the list of the existing gatekeepers from the MDS information tree
It uses the lcgpbs job manager. Remove the lcgpbs jobmanager and configure a plain pbs jobmanager to be compatible with MPI.
Development testbed
EDG 2.x release using LCFGng and RH7.3
The functionality of the Globus MDS has been replaced by RGMA

4. Integration Work (Integration Team)
Related task
Developer
WP4 integrator
Status
1.1 Biomedical appl.
D. Shamonin
A.Tirado
Prototyping
1.2 Flooding appl.
Branislav Simo
J.Astalos
Tested at several sites
1.3 HEP application
D.Rodriguez, C.Martinez-Rivero, I. González, M. Bluj
R.Marco
Installed and tested
1.4.a Meteo WAM-COAMPS
J. Gajeuski, M. Mogielnicki
A.Paddee
1.4.b Meteo SOM
A.Cofiño, J.M. Gutierrez
R. Marco
Tested at CSIC
1.4.c Air Pollution
J.Mouriño
J. Fontán
Tested at CESGA
2.2 MARMOT
B.Krammer
Installed and tested at CSIC(IFCA)
2.3 GridBench
G.Tsouloupas
C.Kanellopoulos
Installed and tested at AUTH
2.4.a GPM
W. Funika
P.Nycyk
Installed and tested at CYFRONET
2.4.b PPC
F.Rivera
C.Fernández
Installed and tested at CESGA
3.1.a RAS + MD
M. Plociennik
P.Wolniewicz
Installed and tested at PSNC
3.1.b Portal
Y.Perros, M. Kokkosoulis
J.Marco
Partially installed at CSIC (IFCA)
3.2 Scheduler
M.A.Senar
A.Fernandez, J.Sanchez
Prototype tested at CSIC (IFIC)
3.3.a OCM-G
R.Wismuller
A.Garcia
3.3.b SANTA-G
B.Coghlan, S.Kenny
M.David, D.O’Callaghan
Installed and tested at TCD/LIP
3.3.c Jiro Monitoring
K. Balos
M.Hardt
3.3.d Global
A.Padee
3.4 Data Access
Lukasz Dutka
S.Gonzalez
Installed at CSIC(IFIC)

5. Integration work Integration Document
It is based on the contributions received from integration team people more or less each month.
The document covers the following issues:
Couple of lines saying what is the software released (including version)
Pre-Form points (based in developers guide)
Code in CVS and how is organized
Compliance of QA rules
Packaging granularity
Level of autobuild experience
Explicit dependency list
Test-request (based in test and validation proposed by task 4.4)
Documentation: installation/management manual, user manual, development manual
Unit test
Compatibility with other software
Security issues
Usability
Previously reported issues
Plans for demo
Status
Use of other CrossGrid components
Being Updating in Cyprus (Appendix Deliverable D4.6)

6. Integration between Application and Services&Tools Prototypes
Interactive Job Scenario (see S. Beco’s talk)
To run interactive job for the simplified X# HEP application
X# middleware based on the EDG plus HEP (Task 1.3) executable
Computes a series of histogram for Gaussian Distribution
User interacts with the RAS (Task 3.1) via the X# MD (VNC mechanism is put in place in order to allow users to submit jobs and see the histogram)
In Cyprus demo, they don’t have a web service that allows the user to submit the interactive job. User launches the interactive submission service using a command line

7. Integration between Application and Services&Tools Prototypes
Application: WP1.1
Interactivity using MD
Working in integrating the application on the testbed
Submissions of jobs outside Amsterdam cluster:
Solver executable needs environment variables to run at other testbed sites

8. Integration between Application and Services&Tools Prototypes
Application: WP1.2
Running without problems with the MD and the Portal (Task 3.1)
Integration done with 3.1 (MD) and 3.2 (Scheduling Agent)
Demo presents:
jobs submission
visualization results
Integration with their portal based on Jetspeed with new components (workflow, collaborative tools)
Problems with job submission (authorities)
Data Management lacks some required functionalities????
Integration with Marmot (WP2) is OK
Working with MD new version
Demo:
Marmot + OCM-G (G-PM) + Portal

9. Integration between Application and Services&Tools Prototypes
Application: WP1.3
Running with MPICH-P4, submission time was long
Fixed using Replica Manager
Submit MPI jobs from the UI to the new EDG-2.x Resource Broker (To show MPICH-G2, in contact with Task 3.2).
Testing Santa-G possibilities
Integrated with the MD
DEMO:
Using Replica Manager
Inclusion of graphics in Portal

10. Integration between Application and Services&Tools Prototypes
Application: WP1.4
Atmospheric and Wave models:
Integrated with the MD and working to be integrated with Portal
Running in four processors in a local cluster.
Air pollution:
Integrated with the MD (old configuration)
Trying integrated the data coming from atmospheric prediction
Portal part was integrated for Poznan
They have installed their application in Valencia’s testbed and it runs ok.
DEMO:
They plan use the MD + PPC

11. Integration between Application and Services&Tools Prototypes
WP2
MARMOT:
Being integrated with MD
Integrated with WP 1.2 (Flooding application)
GridBenchmarking:
Progress with the GUI in the MD
They use applications for performance predictions
Benchmarking using different locations
Same Kernel as PPC (Vertlq)
In contact with Task 3.2 people about MPI jobs (Interested on MPICH-G2)
Tutorial ready
Being integrated with JIMS
Interested in Santa-G

12. Integration between Application and Services&Tools Prototypes
WP2
GP-M (OCM-G):
Show monitoring of Flooding applications with GPM tool
PPC (Performance Predictions):
The GUI on MD is working fine
Problem: they need monitoring information at WN
Working in this (Discarded NWS solution, waiting for JIMS)
Tutorial ready
Demo:
They will show the tool

13. Integration between Application and Services&Tools Prototypes
WP3
Task 3.1 a) RAS and MD:
RAS machine and services installed in Poznan, Lisbon and Nicosia (problems with JSS)
MD installed in Pawel’s Laptop (to assure functionality)
Medical, Air pollution and Flooding applications used as pilots to present functionality
MD presents MPI jobs in distributed environment
Interactivity issue is presented by Task 3.1, 3.2 and 1.3 (HEP)
Task 3.1 b) Portal:
Two portlets added to the portal (job-get-output, job-get-info)
Medical, HEP, Flooding and Air pollution applications ready with the portal to be shown in Demo
Show the generic portlets with all edg-commands

14. Integration between Application and Services&Tools Prototypes
WP3
Task 3.2:
New RB (EDG2.x and the same used in LCG)
Ready to work with MPICH-P4
Integrated with RAS and MD
Functionality is rather the same as presented in Poznan, the effort was dedicated to make it available under LCG1 and EDG2.x
Finishing MPICH-G2 support
Training application running HEP
Problems related to connectivity of WN’s.

15. Integration between Application and Services&Tools Prototypes
WP3
Task (Postprocessing, past 3.3 subtask)
RPM ready in CVS??? It’s being autobuild
Install in two clusters in Warsaw (but it must be installed on all sites driven by the scheduler)
The input data are taken from ganglia and iptables modules:
But it has to be integrated with infrastructure monitoring tools developed in CrossGrid (Santa-G and Jiro). Work to be done during integration meeting in Cyprus.
Task 3.3
OCM-G:
Investigating which applications are suitable for monitoring submitted from the MD (MD plug-in to support monitoring)
Demo shows integration of quite number of X# elements (MD, APPS, and G-PM)

16. Integration between Application and Services&Tools Prototypes
WP3
SANTA-G:
RPM’s available
Working on Demo with HEP application and snort tool
Santa-G works fine with EDG2.x RGM-A
JIMS:
It is working in local cluster at Cyfronet
RPM’s finished but….
Using web service commercial tool
Input to performance predictions has been discussed
Task 3.4
Functionality exploited with LCG-1 and EDG2.x
They are using four storage systems: (Cyfronet, Lisbon, Slovakia, and Karlsruhe)
Working in Integration with MD
Demo:
Estimation real access cost to these storage systems

17. Agenda 9:00 Integration Status Demos 10:30 Coffee Break
Integration setup at Cyprus (Wei 5’)
General description of integration work (Santi, 10’)
Testbed support for integration (Jorge, 10’)
Repository and tools (Marcus, 10’)
Demos
9:30 Flooding (Jan and Bettina, 60’)
Marmot + OCM-G + G-PM + Discussion
10:30 Coffee Break
11:00 Medical applications (Alfredo, 30’)
11:30 HEP (Celso, 15’)
11:45 Meteo Part (Carlos,15’)
12:30 Interactivity (Stefano, 30’)
13:00-14:00 Lunch
14:00 GridBenchmarking + JIMS (George, 25’)
14:25 Data Access Optimization (Lukasz, 15’)
14:40 Santa-G ( Stuart, 15’)
14:55 Postprocessing (Adam, 15’)
15:10 MD+Portal (20’)
15:30 Help Desk (Farida, 15’)
15:45 Discussion
IST Demo
Review