The Narada Event Brokering System: Overview and Extensions PDPTA Las Vegas June 24 2002 PTLIU Laboratory for Community Grids Geoffrey Fox, Shrideep Pallickara Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http://www.naradabrokering.org gcf@indiana.edu 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Implications of Web Services A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL) In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python Web Services separate the meaning of a port (message) interface from its implementation Web Services communicate with messages 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Raw Resources Clients Raw Data Raw Data Render to XML Display Format (Virtual) XML Data Interface Web Service (WS) WS WS WS etc. XML WS to WS Interfaces WS (Virtual) XML Knowledge (User) Interface Render to XML Display Format (Virtual) XML Rendering Interface Clients 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" XML Skin XML Skin Message Or Event Based Inter Connection Soft ware Resource Soft ware Resource Data base e-Science/Grid/P2P Networks are XML Specified Resources connected by XML specified messages Implementation of resource and connection may or may not be XML 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Different Web Service Organizations Everything is a resource implemented as a Web Service, whether it be: back end supercomputers and a petabyte data Microsoft PowerPoint and this file Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport ….. Gives a Peer-to-Peer Grid Here we discuss Event or Message Service linking web services together 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Classic Grid Architecture Resources Database Database Content Access Composition Middle Tier Brokers Service Providers Netsolve Security Collaboration Computing Middle Tier becomes Web Services Clients Users and Devices 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Peer to Peer Grid JXTA Integrate P2P and Grid/WS JXTA Database Database JXTA Web Service Interfaces Event/ Message Brokers Integrate P2P and Grid/WS Peer to Peer Grid Web Service Interfaces JXTA A democratic organization Peer to Peer Grid 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Role of Event/Message Brokers We will use events and messages interchangeably An event is a time stamped message Our systems are built from clients, servers and “event brokers” These are logical functions – a given computer can have one or more of these functions In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications NaradaBrokering is designed to provide these functionalities MPI provided such functionalities for all parallel computing 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering implements an Event Web Service (Virtual) Queue Web Service 2 Destination Source Matching Filter Routing workflow WSDL Ports Broker Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor Workflow implements message process Routing illustrated by JXTA Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of Event Service I MPI nowadays aims at a microsecond latency The Event Web Service aims at a millisecond latency Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites) Different performance/functionality trade-off Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S Synchronous systems: B acts as a real-time router/filterer Messages can be archived and software multicast Asynchronous systems: B acts as an XML database and workflow engine Subscription is in each case, roughly equivalent to a database query 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of Event Web Service II In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism All Web Services are specified in XML but can be implemented quite differently Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints One can sometimes compile message streams specified in GXOS to MPI or to local method call Event Service must support dynamic heterogeneous protocols 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Features of Event Web Service III The event web service is naturally implemented as a dynamic distributed network Required for fault tolerance and performance A new classroom joins my online lecture A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students Company X sets up a firewall The event service sets up brokers either side of firewall to optimize transport through the firewall Note all message based applications use same message service Web services imply ALL applications are (possibly virtual) message based 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" Broker Network (P2P) Community For message/events service Broker Broker (P2P) Community Resource Broker Broker Broker Data base (P2P) Community Software multicast Broker (P2P) Community 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" System Structure I Systems are a dynamic mix of structured and unstructured entities P2P systems like JXTA support unstructured systems realized by opportunistic messaging “broadcast locally” over a certain “network distance” Java Message Service JMS supports structured systems where clients (message endpoints) link to one of a known set of “central servers” Event system must support Advertise capability – Publish Advertise need – Subscribe both for type and form of messages Transport designated messages/events 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" System Structure II One could think that the world is a well defined structure of unstructured systems Unstructured dynamic systems are P2P (JXTA) Peer Groups Peer Groups could be cluster of students in a class for distance learning or cluster of Grid (OGSA) Web services generated to support running a job But maybe it is a set of structured communities with unstructured connection NaradaBrokering needs to support both models and those in between Currently has JMS mode, JXTA mode and Native (most powerful) mode P2P usually thought of as a set of “unruly dangerous clients” but can equally well be used securely as a middleware interaction mode between web services 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" Database Database MP Group Grid Middleware Grid Middleware Grid Middleware MP G r o u p MP G r o u p MP Group MP=Middleware Peer 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Community Grids Laboratory Activities I Core NaradaBrokering Event Service Operation in JMS (see IC paper) or JXTA mode to demonstrate integration of central and peer-to-peer mode Focus is Performance and Capabilities (see later) Garnet synchronous collaboration environment used for distance education and seminars Built first on commercial JMS but ported to Narada – shows that one can afford to use message service in synchronous application sharing Interface of Garnet to PDA with message size filtering and optimized HHMS message service (IC Paper) This filtering also needed for slow clients – mix of dial-ups and Internet2 clients in a collaboration Event system supports (XML) client profiles 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Single Server P2P Illusion Traditional Collaboration Architecture e.g. commercial WebEx Data base Collaboration Server 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering Performance Results 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering and JMS Low Rate; Small Messages 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering and JXTA Comparing Pure JXTA, Narada-JXTA and Direct P2P There is a bug in JXTA and this was only just fixed Narada-JXTA provides JXTA guaranteed long distance delivery Small Payload Larger Payload 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

uri="http://www.naradabrokering.org" email="gcf@indiana.edu" Batik Viewer on PC PowerPoint can be converted to SVG via Illustrator or Web export SVG Sharing PC to PDA 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

PDA Collaboration Event Filter GMS = JMS or Narada 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Community Grids Laboratory Activities II Use of JMS (Narada) to support asynchronous collaboration including early GXOS Schema (IKE Paper) XML based News Groups and Web Site management Integrated with Apache Slide and Jetspeed portals Audio-Video Conferencing as a Web service (IC Paper) H323 and SIP as Web services using XML Session Schema NaradaBrokering support of UDP Computing Portals as Web services (CIC Paper) NaradaBrokering could support events (status, performance, job flow) linking operational job to control servers and researchers 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

NaradaBrokering Futures Higher Performance – reduce minimum transit time to around one millisecond Substantial operational testing Security – allow Grid (Kerberos/PKI) security mechanisms Support of more protocols with dynamic switching as in JXTA – SOAP, RMI, RTP/UDP Integration of simple XML database model using JXTA Search to manage distributed archives More formal specification of “native mode” and dynamic instantiation of brokers General Collaborative Web services 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Collaborative Web Service Access Intercept and multicast messages produced by Web Service Web Service Interceptor Providing General Services Collaboration as a Web Service Collaborative Web Service Set Collaboration and Message Mode Master Client Web Service has a port on which collaborative modes set Web Service can be “front-end” (in middle tier) to complex back-end object Event (Message) Service Client Client 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

Collaborative Replicated Web Services Intercept and multicast messages SENT to Web Service Web Service Interceptor Providing General Services Set Collaboration Mode Web Service Master Object Display Object Viewer Object Web Service Object Viewer Object Display Event (Message) Service Web Service Object Viewer Object Display 12/3/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"