ESB tutorial Antoine Mensch

ESB tutorial - D2 - January 2004 Table of contents è ESB definition è Relevant standards è Experimentation scenarios è Main issues

ESB tutorial - D3 - January 2004 Enterprise Service Bus: a possible definition è ESB – the user view:  Connects applications not designed to work together  Provides enterprise-level quality of service  Provides a foundation for service-oriented and event-driven architectures  Relies on open standards  Promotes configuration rather than development  Must be platform- and language-agnostic è ESB – the technology view:  Provides an extensible platform for hosting integration components and services  Provides predefined integration services  Provides deployment and monitoring tools

ESB tutorial - D4 - January 2004 ESB requirements è Application connectivity  Message-based: applications interact with each other through messages  Non intrusive: the ESB must adapt itself to the existing applications, not the other way around  Loosely coupled: no requirements to integrate around a common data model or exchange format è Enterprise-level quality of service  Reliability of message delivery  Security  Deployment and monitoring capabilities  Services registry and directory  Scalability  Transaction support

ESB tutorial - D5 - January 2004 ESB requirements è Support for service-oriented and event-driven architectures  The ESB should help expose existing applications as services  The ESB should provide a publish-subscribe mechanism è Standards-based: the ESB should use standards for  Message representation  Message description  Message handling and processing è Declarative programming: the ESB should support XML-based languages for  Message and service descriptions (e.g. XML Schema or WSDL)  Configuration  Content-based message routing (e.g. XPath)  Message transformation and enrichment (e.g. XSL-T or XQuery)  Service orchestration (e.g. WS-BPEL)

ESB tutorial - D6 - January 2004 ESB platform requirements è Extensible platform  Defines a common message format within the ESB  Defines a common exchange protocol within the ESB for service invocation and publish/subscribe mechanisms  Provides a container for endpoints and pipelines è Endpoints  Represent entry and exit points for messages transiting in the ESB  Map messages to and from internal and external protocols and formats  Internal endpoints can also be used for routing è Pipelines  Provide configurable, extensible and dynamic endpoint connectivity  Execute the value-added services of the ESB

ESB tutorial - D7 - January 2004 ESB Asynchronous communications SOAP/XML messaging Content-based routing Message transformations Service-oriented and event- driven platform Request Response Notification Event Endpoint Pipeline Event Source Event Sink Service Provider Service Consumer

ESB tutorial - D8 - January 2004 Relevant standards è Two categories of relevant standards  XML and Web Services standards  Java and J2EE standards è XML and Web Services standards  Message representation and exchange  Message and service description  Enterprise-level quality of service  Message processing  Service orchestration è Java and J2EE standards  Platform extensibility framework  Enterprise-level quality of service  Message processing  Enterprise connectivity

ESB tutorial - D9 - January 2004 XML and Web services standards è Message representation and exchange  Two families of specifications SOAP 1.1 and related specifications SOAP 1.2 and related specifications  SOAP 1.1 (W3C note) SOAP with Attachments (W3C note) WS-I profiles (WS Interoperability organization): fix interoperability issues introduced by SOAP 1.1 and WSDL 1.1 specifications  SOAP 1.2 (W3C recommendation) MTOM (W3C proposed recommendation): attachments WS-Addressing (W3C working draft): standard SOAP headers for endpoint description and message routing WS-Eventing (non-standard): publish/subscribe mechanisms è Message and service description and registry  XML Schema (W3C recommendation)  WSDL 1.1 (W3C note) and WSDL 2.0 (W3C working draft)  UDDI (OASIS)

ESB tutorial - D10 - January 2004 WS-* specifications for enterprise-level WS è Endpoint capabilities and preferences  WS-Policy, WS-PolicyAttachments (non-standard) è Reliable messaging  WS-Reliability (OASIS draft): Defines reliable message exchange patterns  WS-ReliableMessaging (non-standard): Defines reliability properties for sequence of messages è Security  WS-Security (OASIS), plus related specs Integrity: XML Signature and XML Canonicalization (W3C) Confidentiality: XML Encryption (W3C) Security assertions: SAML (OASIS), provides support for SSO  WS-Trust, WS-SecureConversation… (non-standard) è Transaction support  WS-Coordination, WS-AtomicTransaction… (non-standard) è Management  WS-Management (non-standard): SNMP-like operations  MUWS and MOWS (OASIS drafts)

ESB tutorial - D11 - January 2004 XML standards for integration services è Message processing  Untyped languages: XPath 1.0 (W3C): XML filtering XSL-T 1.0 (W3C): XML transformation  Typed languages, based on XML Schema: XPath 2.0, XSL-T 2.0 (W3C draft) XQuery 1.0 (W3C draft): XML query language  DOM (W3C): In-memory representation è Service orchestration and choreography  WSBPEL (OASIS draft)  WS-CDL (W3C draft)

ESB tutorial - D12 - January 2004 Java and J2EE standards è Platform extensibility framework  Java Business Integration (JSR-208) è Enterprise-level services  Reliable messaging: JMS, provides publish/subscribe mechanisms  Security: JSSE (SSL/TLS), JAAS (authentication), JCE (cryptography)  Transaction support: JTA  Management: JMX è Message processing  JAXP supports DOM, SAX and XSL-T  JAX-M and JAX-RPC provides Web Services invocation support è Enterprise connectivity  JCA

ESB tutorial - D13 - January 2004 Overview of JBI (JSR-208) è Status  Early draft released in october 2004  BEA and IBM (market leaders in J2EE servers) have withdrawn from the expert group è Goals  Establish an environment for hosting integration components, through standard Service Provider Interfaces  Define an abstract normalized message mechanism to connect integration components  Define a standard for the packaging and deployment of integration components  Define administration and management hooks

ESB tutorial - D14 - January 2004 JBI architecture è Two types of integration components  Service Engines  Binding Components è Service Engine  Provides business logic and transformation services è Binding Component  Provides connectivity to external services è Normalized Message Service  Represents message content in WSDL-compliant format  Represents message context information  Routes messages between integration components

ESB tutorial - D15 - January 2004 Service invocation through JBI 1. Message sent through Client 1 protocol 2. Normalized message forwarded via NMS to SE 1 3. SE 1 invokes SE 2 through the internal « loopback » interface 4. SE 2 returns transformed message to SE 1 5. SE 1 forwards message via NMS 6. Message sent through Client 2 protocol

ESB tutorial - D16 - January 2004 Web services vs. Java/J2EE standards è Common features  XML/SOAP as normalized message format  XML Schema and WSDL for message and service description è Web Services standards  Context information (routing, reliability, security profile, transaction…) is passed as SOAP headers  Language-agnostic  Many specifications are immature and/or proprietary è Java/J2EE standards  Java SPI for plug-in integration components  Context information is passed out-of-band (through API/SPI)  Mature underlying specifications (JMS, JTA…)

ESB tutorial - D17 - January 2004 Some usage scenarios è Goals:  Prototype ESB functionality using open-source components  Identify missing features in existing components è Open-source components  Tomcat: Web server, servlet container  Axis: Web Services engine (server and client sides)  Joram: JMS-compliant reliable messaging implementation  Orbeon: XML pipeline engine è Scenarios  Web service invocation  Mail-based Web service invocation  Web service adapter  Content-based routing  Data transfer  Secure Web service invocation

ESB tutorial - D18 - January 2004 Tomcat Scenario 1: Web Service invocation è Description  WS client and server are unchanged  SOAP messages are wrapped into JMS envelopes  A JMS QueueRequestor is used to retrieve the reply è Advantages  Server location transparency  Reliability  Load balancing potential è Issues  Threading: servlet is multi- threaded while JMS Listener is not  Need for container concepts (e.g. thread pool) to run connectors ESB Tomcat+JORAM Axis WS client Servlet HTTPConnector JMS Listener HTTPInvoker Axis WS server Tmp Q Q SOAP HTTP SOAP/JMS

ESB tutorial - D19 - January 2004 Tomcat Scenario 2: Mail-based Web Service invocation è Description  WS server is unchanged  POP3 server is polled  SOAP messages are received and sent as mail attachments  The WS invocation is similar to Use Case 1 è Advantages  Use of a different, unforeseen protocol to access WS server è Issues  Need to add destination information (in this case mail ReplyTo) in the SOAP message  Need to manage MIME multipart messages ESB Tomcat+JORAM POP3 Reader JMS Listener HTTPInvoker Axis WS server Q2 Q1 SOAP HTTP Mail server JMS Listener SMTPSender SOAP Attach.

ESB tutorial - D20 - January 2004 Tomcat Scenario 3: Web Service adapter è Description  Transform an incoming SOAP message into another one, compliant with expected input of WS server  Transform reply accordingly è Advantages  Connection of incompatible client and server  Possibility of independent evolution of client and server è Issues  Need for a single interface for internal endpoints (queues or connectors), to avoid useless intermediate queues  Need for a component model to combine basic components into more complex reusable components ESB Tomcat+JORAM Axis WS client HTTPConnectorHTTPInvoker Axis WS server Tmp Q Q SOAP HTTP Q1Q2 JMSListener Adapter SOAP HTTP

ESB tutorial - D21 - January 2004 Tomcat Scenario 4: Content-based routing (CBR) è Description  Incoming SOAP messages are dispatched to several WS servers, based on content  Servers are assumed to have the same interface (possibly using WS Adapters as in Use Case 3) è Advantages  Additional step in server location transparency è Issues  Need for a rule language to dispatch messages to appropriate internal endpoints ESB Tomcat+JORAM Axis WS client HTTPConnector HTTPInvoker2 Axis WS server 2 HTTPInvoker1 Axis WS server 1 Tmp Q Q SOAP HTTP Q1Q2 JMSListener CBR

ESB tutorial - D22 - January 2004 Scenario 5: Data transfer è Description  Incoming XML files are wrapped into SOAP messages, large or binary files as attachments  The connector polls for incoming files è Advantages  Homogeneous view of internal messages  Value-added components can be used è Issues  Performances: large attachments should not be copied from one internal endpoint to another, but rather stored once and retrieved when needed ESB Tomcat+JORAM FileConnector JMS Listener FileWriter Q SOAP/JMS Attachments File

ESB tutorial - D23 - January 2004 Tomcat Scenario 6: Secure Web Services invocation è Description  Same as Scenario 1  Use HTTP-Auth+SSL for authentication è Advantages  Same as scenario 1 è Issues  Same as scenario 1  Security information must be passed as context data through the ESB ESB Tomcat+JORAM Axis WS client Servlet HTTPConnector JMS Listener HTTPInvoker Axis WS server Tmp Q Q SOAP HTTPS HTTP-Auth SOAP HTTPS HTTP-Auth SOAP HTTPS SOAP/JMS

ESB tutorial - D24 - January 2004 Main issues è The ESB platform should be a container for endpoints (connectors) and pipelines è The ESB should define a pluggable component model to ease integration and reuse of integration components è Predefined components supporting basic ESB functionality are required  Reliable messaging  Standard exchange protocols  Message routing and transformations è The internal message structure must support  Attachments  Context information (routing, security profile…)