1. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 New Generation Environment for Grid Interactive MPI Applications M igrating D esktop / R oaming A ccess S erver Miroslaw Kupczyk -Rafal Lichwala - Bartek Palak miron - syriusz -bartek[@man.poznan.pl] Marcin Plociennik - Pawel Wolniewicz marcinp - pawelw[@man.poznan.pl] Stefano Beco - Marco Sottilaro stefano.beco-marco.sottilaro[@datamat.it] PSNC (Poland) - Datamat S.p.A. (Italy)

2. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Outline  Objectives  What is interactivity ?  CrossGrid architecture for interactive services  CrossGrid key components: Migrating Desktop & Roaming Access Server  Interactivity schema  at the end …. some conclusions

3. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Objectives !  To provide user-friendly framework for grid interactive and data intensive applications  To support parallel and real-time computation  To hide the complexity of data and resources handling  To deliver one uniform GUI that supports different HPC infrastructures on different platforms Application developers requirements Real user requirements Grid without real users is useless!!!

4. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Batch Jobs vs Interactive Jobs “I submit my job over the grid, and... ………… I wonder what I want ….” 1) „I want it to be executed now!”(priority and advance reservation) 2) „I want to monitor the execution” (checking the status) 3) „I want to know its registered events” (logging information) BATCH JOBS I want to see its final output I wait for the end of the job !!! INTERACTIVE JOBS While the ones are running I want to : 1) to receive any outputs 2) send my input data to it

5. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Requirements for interactivity When I want to submit an interactive job I need: the allocation of grid resources throughout the whole “interactive session” ; a bi-directional channel between my client and the application program running on a remote machine; the input and output streams are exchanged with my client via this channel (I can send my input data to the job and receive its intermediate output and error messages).

6. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Architecture JobSubmission Services CrossBroker EDG 2 LB Roaming Access Server Computing Element WN Node Java Visualisation/VNC plug-ins Migrating Desktop Interactive Session Services File Management Services Profile Management Services Application Management Services Virtual Directory II / LDAP SE

7. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Migrating Desktop  Key CrossGrid component  Integration point of most CrossGrid tools and application  Open system that integrates access to many grids all over the world  It provides work environment to users in any location, independently of a hardware platform and the operating system  A flexible system structure enables defining and adaptation requirements of individual groups of users  Advanced user-friendly Java application based on web services and Java CoG technology Interactive Application oriented framework giving the entire work environment anywhere in the grid

8. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 How does it look like?

9. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Migrating Desktop tools  Desktop tools - Job Wizard - Job Monitor - Application Container and Application Plugin - GridFTP Commander - User Profile Manager - Private Storage Management - VNC/SSH console

10. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Migrating Desktop functionality Main Features:  Single sign-on / authorisation  Platform independent  Batch jobs  MPI jobs  Running interactive applications using java plugins or VNC  Monitoring grid applications  Flexible Application framework  User profile management  Easy application add on  Local and grid file management

11. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Supported applications and tools  CrossGrid Applications: Medical application Air pollution HEP...  CrossGrid Tools: GridBench PPC Marmot OCM-G...

12. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Roaming Access Server  Well-defined set of web-services  An interface for accessing HPC systems and services (based on various technologies) in a common and standardised way  Interconnection between various grid middleware and applications  Additional features: Virtual Directory support Plug-in for various grid middleware JobSubmission Services Roaming Access Server Interactive Session Services File Management Services Profile Management Services Application Management Services

13. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Job Submission Services (JSS) a set of Web Services, which allows performing job submission and job monitoring: Job-Submission Job-Cancel (cancelling of one or more submitted jobs); Job-Status (bookkeeping information on the submitted jobs); Job-Logging-Info (logging information on the submitted jobs); GetUserJobs (list of the jobs “recently” submitted by users) ; Job-List-Match (list of resources fulfilling the user job requirements) JobSubmission Services Roaming Access Server Interactive Session Services File Management Services Profile Management Services Application Management Services

14. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Job Submission Services (JSS)  advanced services for MPI and interactive jobs  batch jobs : sequential and parallel (MPI) ? ok !  interactive jobs : sequential and parallel (MPI) ? ok !  two types of parallel jobs based on the MPICH library: “MPICH-P4” (single cluster) “MPICH-G2” (multiple clusters)  Interactive services based on Condor ByPass technology (to handle i/o streams between jobs on remote machine and user applications)

15. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Interactivity for MD-integrated applications Job Submission Services CrossBroker Logging & Bookkeeping Roaming Access Server Computing Element LRMS Gatekeeper JDL Migrating Desktop RAS shadow port, RAS shadow host Process Launched Interactive data Control data Submission flow In/Out/Err job data 010011000 WorkerNode Job Shadow stdin – stdout -stderr Console Agent Job Condor ByPass System Job Shadow Java Visualisation plug-in

16. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Conclusions  Supporting real grid interactive MPI application  Extending EDG interactivity approach – interactive application at user desktop  Providing trasparency of grid files and resource management  Integration of CrossGrid tools and applications  Easy to use in production environment not only grids easy to adopt for special purposes (computing on- demand)  MD/RAS already used by other grid projects (e.g. Progress project)

17. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Product availability- contact  Released under open source licence  Migrating Desktop website: http://ras.man.poznan.pl/crossgrid You can plug&play there the last MD/RAS server version

18. EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 Thanks for your attention l http://www.datamat.it www.man.poznan.pl if you need further information : cgrid@man.poznan.pl marco.sottilaro@datamat.it