Grid Federation JXTA Jini etc. GGF7 Tokyo March 5 2003 PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http://www.naradabrokering.org http://grids.ucs.indiana.edu/ptliupages gcf@indiana.edu 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Portal such as “Jetspeed” Users Portal such as “Jetspeed” Hosting Environment Application/User Framework supporting development and deployment of OGSI compliant AWS (Application Web Services) AWS Hosting Environment Grid Computing or Programming Environments Generic Application Services Web Services “Core” Grid “Core” System Services Resource Grid Services e.g. DAI compliant database Database Resources 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Federation/Interoperability Problem? Have a collection of Web Services running in Grids defined by different suppliers? Interoperability – “particular application Web Service of supplier X” can utilize “core service of supplier Y” Federation– “core service of supplier X” can be integrated with “core service of supplier Y” to provide a integration/amalgam that is also a realization of core service. Need mediation to link different Grid Islands Standards Compliant InterGrids Federation Environment Jini Grid Resource1 GT3 Grid AWS2 Platform AWS3 Service1 JXTA AWS1 IBM Grid Resource2 Avaki Grid Service2 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

OGSI Interface Implications OGSI defines the features that all Web Services must have to be called Grid Services Using Web Services is equivalent roughly (in old-style computing lingo) to agreeing on a common (object-oriented) language It is equivalent to choosing C++ or Java or more precisely agreeing to use CORBA IDL or Java to define external interfaces (method calls) without prejudice as to implementation language For example, having a “Grid Information Port” is VERY roughly analogous to agreeing that all programs have a standard input and a standard output familiar from UNIX Agreeing that “Grid Information Ports” should communicate XML records (SDE Service Data Element) is roughly equivalent to saying standard output will use Fortan namelist syntax So this is not yet a complete interoperability or federation specification 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" InterGrids Develop Grid software including both the packaging of Grid Islands and federation between Grids Key underlying principles: Support small Grids (university departments, Private Grids) with easy deployment (e.g. JXTA Jini …) Support composability (P2P) and federation with itself and other Grids Use existing (Avaki, Globus JXTA ICENI ..) bits and pieces if possible – encourage such projects to produce modules useable in other Grid systems like InterGrids Good test/benchmark/tutorial material and good support 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of InterGrids I At outer levels of Grid, just need interoperable services but these services would need mediation as they access resources in a different grid Interoperable Services have interfaces allowing them to access appropriate other services in their own grid and through mediation the analogous services in other grids At inner levels of Grid, need to federate services Federated services produce results which need to be merged with other services of the same type when Grids are joined together In case of Jini based Grid, we keep Jini services as Jini and don’t wrap them Rather the OGSA wrapping of Jini occurs in the mediator Issues if multiple Grids overlap Can multiple brokers manage same resources (schedulers on computers) What happens if one shares resources between virtual organizations (Grids) 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of InterGrids II Mediation service integrates VPN with adaptable port Firewalls and security negotiation between domains Mediation Difference between PURL/URI and URL Mediation Service is distributed i.e. is not a single proxy server Each logical message (this could be several physical messages) in system is examined – destination URI is “just” mapped to URL if internal If external routed to “best mediator” in same way NaradaBrokering optimally routes messages Logical messages are self contained i.e. have enough information to fully specify any transformations needed in mediator i.e. they are “all” the Schema instance 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of InterGrids III When message arrives at mediator service, its source and destination Grids are examined Note Grids must register themselves in mediation service and in that registration (or a later update) give specification for service and any needed transformations) This registration involves introduction of concept of Grid Gridtype with specification attached to a particular Gridtype If they have same specification for this service, then message is routed on unchanged If specification different, the mediator looks to see if any special translations available for given source or destination If no special actions, the source mapper is used to convert message to FGSA (Federated Grid Service Architecture -- hopefully same as OGSA) and the destination inverse mapper is used to map FGSA to destination format This can generate an error returned to calling service and logged for both Grids 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of InterGrids IV This mediation is sufficient for an “interoperable” service Some services such as “search” or “look-up” are registered as federated i.e. requests to them are multi-cast to multiple grids and results merged Rules must be defined for federated services defining nature of result Is more than one answer allowed If >1 answer possible, then rules for merging results of services in each Grid must be specified Another complication is if a single service in one Grid corresponds to composition of multiple services in another Grid One sees this for look-up which could involve different levels of meta-data in different Grids This seems complicated but relatively clear what one has to do in composing dynamically services 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of InterGrids V There are two problems mentioned earlier Shared Resource Reference Services – this must be a standard issue in federation. This occurs when federation of say look up services leads to duplicate results. The look-up may not be quite the same and one would want to remove or combine duplicate responses Shared Resource Access Services – these are services in different Grids that access and affect the same backend resource. This issue comes up even inside a single Grid It is possible that mediator could be used to resolve this problem but an heuristic needs to be developed for this 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Federation Architecture Routing Node Mediation Node Resource Grid Instance Service R M 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" NaradaBrokering Publish/Subscribe Distributed Event/Message System http://www.naradabrokering.org “MQSeries/JMS” P/S applied to Collaboration, Grid, P2P(JXTA) Supports UDP, TCP/IP, Firewalls (actual transport  user call) Used in other projects: Collaboration, Portal and Handheld 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" NaradaBrokering Based on a network of cooperating broker nodes Cluster based architecture allows system to scale to arbitrary size Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. Now has five major core functions Message transport (based on performance measurement) in heterogeneous multi-link fashion General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing Distributed XML data-base using P/S XPATH metaphor Filtering for heterogeneous clients Federation of multiple instances of Grid services as illustrated by JXTA peer-group linkage 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering and JXTA Narada-JXTA provides JXTA guaranteed long distance delivery Request/Response Present if targeted at Particular peer Narada JXTA Event 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Federation of Services If you have a Service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing …. With a standard for client and server components Then can build a “server” that services all clients Alternatively can hierarchically consider collection of existing Server-client domains IBM or Globus OGSA islands Sun Grid Engine Schedulers Polycom/Access Grid A/V sessions Enterprise Search Engines Federation links servers for each island together JXTA Search has XML specified federation approach – will try to include and generalize as a NaradaBrokering federation framework DoD High Level Architecture HLA does this for simulation NB Hub 4/2/2019 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"