Enterprise Service Bus Operating System for Distributed Software Design Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Axel Angeli - Building SOA City - Enterprise Service Bus Enterprise Service Bus is a software architecture model to support and control communication of widely distributed and disparate applications An Enterprise Service Bus is basically a set of public services that assist the execution of services in a distributed environment. The main duty is to make a decision which server should execute the service when a request arrives from a client. It is an essential design concept of an ESB that every client directs all its requests through the ESB instead of passing it directly to a potential server. This indirection allows to ESB to reveal its real power and apply special functionality. The ESB will then be able to monitor the traffic, store the message in a staging area and automatically deliver it when the receiver is ready. The ESB will also take over the very important task to monitor the traffic, write logs and intervene to avoid collision. An ESB takes the role of an operating system across disparate independent computers Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Duties of an Enterprise Service Bus Axel Angeli - Building SOA City - Enterprise Service Bus Manage service invocation Assigns tasks to available servers for execution Host Commodity Services Hosts public services that are convenient and needed by other services Monitor and control collisions Controls message traffic and guarantee accident free delivery Protocol Conversion Provides centralized and harmonic protocol conversion Scripting automatic tasks Allows easy automation of tasks and embed rules engines Queue Message Allows staging and automatic forwarding of messages to avoid loss Duties of an Enterprise Service Bus ESB is a large collection of public services Allows dynamic plugging of services Assigns tasks to available servers Manage service invocation Controls message traffic and free delivery Monitor/control collisions Hosts commonly needed public services Host Commodity Services Provides centralized protocol conversion Protocol Conversion Allows easy automation of tasks Scripting and rules Allows staging of messages to avoid data loss Message Queue An ESB hosts a large collection of services. There will be many commodity services that are useful and regularly needed by other services. Most services deal with directing and marshalling the routing of messages, doing common and often needed data transformations. Popular commodity service are compressing and encrypting data, splitting data into smaller chunks, filter unwanted data, extract routing information from the content via a rule engine. A very special service rendered by an ESB is the transformation and conversion of multiple protocols to allow services of many different legacies to easily plug into the bus system, no matter which protocol it decides to use to initiate a request or deliver the response. That way even older and exotic legacy systems can be easily hooked up into the SOA without requiring the service client to adapt itself to it. A good ESB is not a static construct. The power of an ESB lies also in the ability to dynamically change its behavior. This is normally achieved by adding a powerful scripting language that allows ad hoc changing code that is used to control services and a rule engine to give the accidental user the ability to add business rules. A mandatory component of an ESB is a message queue that can be controlled and used by internal and external services. Nearly every middleware has a message queue. Without the ability to stage messages in strict sequential order it is not possible to design complex real time multitasking programs across a large number of distributed execution platforms. An ESB is incomplete when the MQ is hidden and only accessible to services that are built into the ESB. For an ESB to be useful it must provide a good Message Queuing system for any service that plugs into the bus. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

An Enterprise Service Bus is basically a set of public services that assist the execution of services in a distributed environment. The main duty is to make a decision which server should execute the service when a request arrives from a client. It is an essential design concept of an ESB that every client directs all its requests through the ESB instead of passing it directly to a potential server. This indirection allows to ESB to reveal its real power and apply special functionality. The ESB will then be able to monitor the traffic, store the message in a staging area and automatically deliver it when the receiver is ready. The ESB will also take over the very important task to monitor the traffic, write logs and intervene to avoid collision. An ESB hosts a large collection of services. There will be many commodity services that are useful and regularly needed by other services. Most services deal with directing and marshalling the routing of messages, doing common and often needed data transformations. Popular commodity service are compressing and encrypting data, splitting data into smaller chunks, filter unwanted data, extract routing information from the content via a rule engine. A very special service rendered by an ESB is the transformation and conversion of multiple protocols to allow services of many different legacies to easily plug into the bus system, no matter which protocol it decides to use to initiate a request or deliver the response. That way even older and exotic legacy systems can be easily hooked up into the SOA without requiring the service client to adapt itself to it. A good ESB is not a static construct. The power of an ESB lies also in the ability to dynamically change ist behavior. This is normally achieved by adding a powerful scripting language that allows ad hoc changing code that is used to control services and a rule engine to give the accidental user the ability to add business rules. A mandatory component of an ESB is a message queue that can be controlled and used by internal and external services. Nearly every middleware has a message queue. Without the ability to stage messages in strict sequential order it is not possible to design complex real time multitasking programs across a large number of distributed execution platforms. An ESB is incomplete when the MQ is hidden and only accessible to services that are built into the ESB. For an ESB to be useful it must provide a good Message Queuing system for any service that plugs into the bus. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Axel Angeli - Building SOA City - Enterprise Service Bus Service Plugs Add and remove services Message Queue Buffer messages until they can be processed Process Director Control the process flow Message Router Dynamically decide the sequence of service execution Process Monitor Monitor the behavior of the processes Exception Handler Handle errors and exceptions dynamically Adapter Engine Interface to outside orbit Core Elements of ESB An ESB is extensible by services and other ESB Other basic services can be added at any time Add and remove services Service Plugs Buffer messages until they can be processed Message Queue Control the process flow Process Director Dynamically decide the sequence of service execution Message Router Monitor the behavior of the processes Process Monitor Handle errors and exceptions dynamically Exception Handler Interface to outside orbit Adapter Engine New services can be plugged into the service bus. The main element of an ESB is the ability to allow “hot plugging” of service components. This means that any program can be activated and deactivated at any time without suspending or inhibiting the execution of non related processes. In a distributed environment it is the normal case that one service needs to wait for another service. An ESB takes over the common duty to buffer the pending requests until they can be handled properly. The buffers are normally designed as message queues to allow a proper sequencing of processing. An ESB is designed to handle millions of messages at the same time. They are coordinated by one or several process directors. They are central processes that watch the service execution and make use of sophisticated algorithms to decide which service should execute next. The process director works closely together with the message router. This is normally a rule engine that allows to decide the sequence and activation of a service depending on the current state of the system. All processes and services are monitored at any time. Every service is executed in a save habitat to reduce the possibility that one service can impact another service negatively. The service monitor can alert the process director and marshal a misbehaving service. In a sense it is the police squad of the ESB. Misbehaving services or an exceptional situations are handled by a standard exception handling mechanism. No message or service should be allowed to run into a situation that cannot be gracefully ended. The ESB is a closed habitat. Communication with the outside orbits is done exclusively via specially designed gateways and adapters. Adapters take external data formats and convert them into a harmonized and canonical internal format. Any message processed by the ESB is always delivered in a standard container. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

An enterprise service bus is primarily an abstract pattern to allow execution of services that adhere to certain communication patterns. It is defined as being extensible in the sense that new services can be added at any time and multiple regional Enterprise Service Buses can be joined to form a bigger and Global ESB. New services can be plugged into the service bus. The main element of an ESB is the ability to allow “hot plugging” of service components. This means that any program can be activated and deactivated at any time without suspending or inhibiting the execution of non related processes. In a distributed environment it is the normal case that one service needs to wait for another service. An ESB takes over the common duty to buffer the pending requests until they can be handled properly. The buffers are normally designed as message queues to allow a proper sequencing of processing. An ESB is designed to handle millions of messages at the same time. They are coordinated by one or several process directors. They are central processes that watch the service execution and make use of sophisticated algorithms to decide which service should execute next. The process director work closely together with the message router. This is normally a rule engine that allows to decide the sequence and activation of a service depending on the current state of the system. All processes and services are monitored at any time. Every service is executed in a save habitat to reduce the possibility that one service can impact another service negatively. The service monitor can alert the process director and marshal a misbehaving service. In a sense it is the police squad of the ESB. Misbehaving services or an exceptional situations are handled by a standard exception handling mechanism. No message or service should be allowed to run into a situation that cannot be gracefully ended. The ESB is a closed habitat. Communication with the outside orbits is done exclusively via specially designed gateways and adapters. Adapters take external data formats and convert them into a harmonized and canonical internal format. Any message processed by the ESB is always delivered in a standard container. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Common Processing Pattern Axel Angeli - Building SOA City - Enterprise Service Bus Common Processing Pattern Interface Section to Outside Orbit Harmonization Section Internal Pipes Q Q Q Q Process Transform Canonical Transform Canonical Adapter Adapter Adapter Q Transform Process All communication via message queues Data is delivered via an adapter to queue Message is transformed to canonical format Business process is invoked by the ESB Result converted back and delivered by adapter An enterprise service bus is primarily an abstract pattern to allow execution of services that adhere to certain communication patterns. It is defined as being extensible in the sense that new services can be added at any time and multiple regional Enterprise Service Buses can be joined to form a bigger and Global ESB. Any processing on service-oriented architecture follows a common processing pattern. Understanding that pattern is the fundament for making sense to processing in distributed environments. Communication on a service-oriented Enterprise Service Bus takes place with messages that are pushed forward from message queue to message queue. A general process receives messages from the outside orbit. Since data is delivered in many different ways they are all exchanged with the outside world via adapters. An adapter takes the message and stores it in raw format in a message queue or another convenient staging area. From now on the Enterprise Service Bus mechanism takes over control. The ESB will pick the message from the common message queue and convert the data into a harmonized internal format, the Canonical Message Format and leaves the resulting message in another queue like you drop a letter into a mailbox. That way the business process can pick and process the message, only to deliver the result to another message queue. Now the messages are converted back from the internal canonical format into an adapter format, while the adapter takes care of physical delivery of the message to the designated receiver. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Any processing on service-oriented architecture follows a common processing pattern. Understanding that pattern is the fundament for making sense to processing in distributed environments. Communication on a service-oriented Enterprise Service Bus takes place with messages that are pushed forward from message queue to message queue. A general process receives messages from the outside orbit. Since data is delivered in many different ways they are all exchanged with the outside world via adapters. An adapter takes the message and stores it in raw format in a message queue or another convenient staging area. From now on the Enterprise Service Bus mechanism takes over control. The ESB will pick the message from the common message queue and convert the data into a harmonized internal format, the Canonical Message Format and leaves the resulting message in another queue like you drop a letter into a mailbox. That way the business process can pick and process the message, only to deliver the result to another message queue. Now the messages are converted back from the internal canonical format into an adapter format, while the adapter takes care of physical delivery of the message to the designated receiver. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Communication has three layers Communication via Bus Devices connector connector Adapter Logic proc proc Transport R S R S R S Communication has three layers Logic, Data, Control The central idea of an ESB is the strict segregation of the abstract layers in a communication scenario. When a message is sent from a sender to receiver it is subject to three different treatments: logical processing of its semantics, transportation of the message from sender to receiver and the rules that decide which logic needs to be applied and who should receive the message Communication with the outside world is done strictly through gateways that are maintained by adapters. An adapter converts arbitrary external formats into a standard internal message format before it is allowed to leave the quarantine area and is given access to the ESB. Control C C C Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Message queues Queues are the central element of an ESB Queues are mandatory in distributed computing Services work at different speeds Messages are held until they can be handled Queues store undeliverable messages (dead letters) Queues can be database, files or memory arrays Message queues are the core element of an ESB. It is the normal case a distributed computing environment that services work at different speed. So there is always a need for a mechanism to balance out the waiting time that happens between the moment when a request arrives and the ability of a service to handle it. Message queues are a convenient way to handle this problem and takes away the complicated and receptive task from the service down to ESB level. Messages are held in the message and the process director monitors that they are routed properly to and from a service. A message that cannot be processed or delivered is called „dead letter“. The ESB reports all dead letters in a dedicated dead letter queue. A message is not necessarily a persistent database store. It can be implemented in thousand ways. Usually they are folders in a file system with the messages as individual files. If speed is an issue and loosing a message in transit due to machine failure is acceptable the queue can also be a simple queue in memory. Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

(c) 2010 Logosworld Donnerstag, 29. November 2018 Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018

Pipes (c) 2010 Logosworld Donnerstag, 29. November 2018 Thursday, 29 November 2018 Axel Angeli - (c) 2010 logosworld.com - Building the SOA City (c) 2010 Logosworld Donnerstag, 29. November 2018