Global Grid Access Cezary Mazurek, PSNC

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 2 Agenda Introduction PROGRESS grid-portal environment PROGRESS GSP (Grid Service Provider) Enabling access to global grid with open source GSP Conclusions Demo

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 3 R&D Center PSNC was established in 1993 and is an R&D Center in: –New Generation Networks POZMAN and PIONIER networks 6-NET, SEQUIN, ATRIUM projects –HPC and Grids GRIDLAB, CROSSGRID, VLAB, PROGRESS projects –Portals and Content Management Tools Polish Educational Portal Multimedia City Guide, Digital Library Framework, Interactive TV

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 4 SUN Center of Excellence PSNC became the Sun CoE in New Generation Networks, Grids and Portals in November 2002

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 5 PROGRESS (1) Duration: December 2001 – May 2003 (R&D) Budget: ~4,0 MEuro Project Partners –SUN Microsystems Poland –PSNC IBCh Poznań –Cyfronet AMM, Kraków –Technical University Łódź Co-funded by The State Committee for Scientific Research (KBN) and SUN Microsystems Poland

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 6 PROGRESS (2) Deployment: June 2003 –.... –Grid constructors –Computational applications developers –Computing portals operators Enabling access to global grid through deployment of PROGRESS open source packages

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 7 PROGRESS (3) Cluster of 80 processors Networked Storage of 1,3 TB Software: ORACLE, HPC Cluster Tools, Sun ONE, Sun Grid Engine, Globus Wrocław Gdańsk

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 8 PROGRESS GPE modules grid resource broker: enables the execution of PROGRESS grid jobs in a distributed cluster of Sun computers; uses XRSL language for grid job descriptions grid security: SSO authentication, authorisation checkpointing user interfaces –Grid Service Provider web portal: grid job management, application management, provider management, short news reading and editing, DMS file system management migrating desktop: grid job management and DMS file system management data management system: storing the input and output files, metadata, multiple media data containers visualisation

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 9 Motivation (1) grid access environments lacked flexibility: one grid -> one portal user must have relied on the grid (e.g. job execution history) user interfaces were not that much functional what if we need more than one user interface to grid services?

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 10 Motivation (2) Grid constructors –„We have to build a new portal to enable our grid resources to users...” Application developers –„We have to build a new portal to enable our grid- enabled applications to users...” Portal operators –„It might work but probably we have to redesign our portal to enable this new grid service...”

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 11 PROGRESS GPE PORTAL HPC RESOURCES GRID MANAGEMENT SYSTEM GRID SERVICE PROVIDER 4-tier new grid-portal environment PORTAL HPC RESOURCES GRID MANAGEMENT SYSTEM 3-tier classical grid-portal environment

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 12 PROGRESS GPE

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 13 PROGRESS GSP User environment (eg. Portal) HPC RESOURCES GRID MANAGEMENT SYSTEM GRID SERVICE PROVIDER HPC RESOURCES GRID MANAGEMENT SYSTEM GRID SERVICE PROVIDER User environment (eg. Portal) User environment (eg. Java GUI)

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 14 Grid Service Provider (1) the use of the grid resources most comfortable to the end users allows for easy building of numerous portals and other user interfaces; users can switch from one to another and use the same GSP services various web portals sharing the same distributed grid services and resources possibility of providing all clients (user interfaces) with computing resources belonging to two or more different grids

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 15 Grid Service Provider (2) Provided services : –job submission service managing the creation of user jobs, their submission to the grid and the monitoring of their execution (typically through reverse reporting performed by the Grid Management System about events connected with the execution of jobs) –application management service storing information about applications available for running in the grid assisting application developers in adding new applications to the application repository –provider management service keeping up‑to‑date information on the services available within the provider Portlet Webservice Session Bean Entity Beans

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 16 Job submission service computing job building, submitting them to the grid for execution and viewing the results job description is prepared using the XRSL language and transferred to the grid resource broker for the execution of the job grid resource broker reverse reports on grid events connected with the job „ workflowed ” jobs: sequences and parallels

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 17 Application management srv. application factory management application descriptor contains a reference to the application executable: a reference to a file stored in the DMS or a path to a binary on grid computing server filesystems also included in the application descriptor: available (required or optional) arguments, required environment variables and required input and output files applications in PROGRESS may be unconfigured or configured: one executable -> multiple configured applications virtual applications

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 18 Grid job management Application addition Application configuration Application configuration Job submission Job submission Job execution Job completion Abstract state Ready state Built state Active state Ghost state

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 19 Provider management service enables keeping up-to-date information on services available in the grid service provider a service descriptor contains information on the Web Service interface: URL at which the service is available, the service namespace reference (URN) and the service WSDL reference services may have multiple instances: informational services

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 20 Informational services examples of instance enabled services intended for use by web portals PROGRESS example: short news service other: document directory, discussion forum

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 21 PROGRESS Communication HPC Portal Grid Service Provider Data Management System Grid Resource Broker saveJob() getApplications() getTemplates() saveTaskOfJob() saveStdOfTask() submitJob() getUserJobs() getJobStatus() listUserDirectory() addUserFile() getUserFileLocation() submitJob() changeTaskStatus()

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 22 Web Services and Grid PORTLETS GRID SERVICE PROVIDER DATA MANAGEMENT GRID RESOURCE BROKER WS

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 23 GSP Release & Future The Grid Service Provider software has been released under an open source license (SC 2003) It is tested and deployed within PROGRESS GPE together with Grid Resource Broker, PROGRESS Portal and Migrating desktop, Data Management System, Test Applications and Visualisation We will continue the R&D on the Grid Service Provider –cooperation with multiple grid resource brokers/grid management systems (e.g. the GridLab one) –additional functionality (e.g. OGSA interface, cooperation with virtual laboratories) –apply the concept for access to visual grids, data grids and digital library tools and services

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 24 Enabling global grid

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 25 Enabling global grid

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 26 Enabling global grid

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 27 Enabling global grid Moreover, to enable global grid within a single GSP installation, we developed a concept of grid resource broker plug-ins The plug-in will be responsible for communication with the grid resource broker The job submission service will use the proper plug-in whenever there’s a need to submit a computing job to a particular grid

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 28 GRB plug-in A java class capable of communicating with and submitting computing jobs for execution to a particular grid To enable the GRB plug-in concept, there will be a set of entity beans prepared to manage the information about available plug-ins and grids

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 29 Available plug-ins The GRB plug-in mechanisms are under development New release will include a plug-in for the PROGRESS grid resource broker Next plug-ins to follow: –GridLab –SGE? –Globus?

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 30

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 31 PROGRESS HPC Portal

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 32 Conclusions Grid constructors –„Finally, we can forget about computing portals development” Application developers –„Now, we can easily make our application available to new computing portals” Portal operators –„Well, there are portlets available which we might include in our portal. It will work” End user –„Now, I really don’t care where is the grid service. I can access it from the portal I used to work with”

Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ Demo: SUPERCOMPUTING 2003, Phoenix, SUN Microsystems booth: Stand No 14