1. l BeJUG – Enterprise SOA’07 Conference
l Belgium French Community goes SOA Open Source with Servicemix and BULL
l Guillaume Nodet - IONA
l Anne Noseda - BULL

2. Guillaume Nodet Anne Noseda IONA Principal Engineer
l Speakers
l 09/10/2007
Guillaume Nodet
IONA Principal Engineer
PMC Chair of the Apache ServiceMix project
Involved in several Open Source projects (Apache ActiveMQ, Apache CXF, Apache Ode, Apache Geronimo)
Previously involved in XFire, Mule
Anne Noseda
Bull Open Source BU
Worked with OW2 projects (Bonita and Orchestra)
Technical Director of the Open Source ESB project at the French Community of Belgium

3. l First part : Apache Servicemix
l BeJUG – Enterprise SOA’07 Conference
l 09/10/2007
l First part : Apache Servicemix
l Guillaume Nodet - IONA

4. The Agile ESB Standards based Flexible Reliable
l The Agile ESB
l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
The Agile ESB
Standards based
Flexible
Reliable
Breadth of Connectivity
Standards based
While standards-based support is marketed by many ESB vendors, the support is provided externally, requiring developers to work with proprietary APIs when directly interacting with internal APIs. ServiceMix was designed with the requirement to eliminate product API lock-in, by being built from the ground up to support the Java Business Integration specification (JSR 208). Our agile ESB needs to use JBI as a first class citizen, but also support POJO deployment for ease of use and testing.
Flexible
Another characteristic of an agile ESB is the flexibility with which it can be deployed within enterprise application integration framework: standalone, embedded in an application component, or as part of the services supported by an application server. This allows for component re-use throughout the enterprise. For example, the binding for a real-time data feed might be aggregated as a web-service running within an application server, or streamed directly into a fat client on a traders desk. An agile ESB should be able to run both types of configurations seamlessly.
To provide rapid prototyping, an agile ESB should support both scripting languages and embedded rule engines, allowing business processes to be modeled and deployed quickly.
Reliable
Our ESB needs to handle network outages and system failures and to be able to reroute message flows and requests to circumvent failures. Hence the need to support transactions, persistent exchanges, distributed and clustered services.
Breadth of Connectivity
An agile ESB must support both two way reliable Web-services and Message Oriented-Middleware and needs to co-operate seamlessly with EIS and custom components, such as files, s or RSS feeds.

5. Java Business Integration
l Java Business Integration
l Apache Servicemix u
Standards Based
Flexible
Reliable
Breadth of connectivity
Java Business Integration
Integration Systems
Plug-In Components
WSDL
Decoupling
JBI is a java based standard to build integrations systems by using plug-in components which interoperates through mediated normalized message exchanges.
The message exchange model is based on the web services description language (WSDL).
The figure on the left a high level concept of JBI of the plug-in framework. The JBI environment provides interfaces to be used by plug-in components, while the components provides interfaces used by the JBI environment. Components do not interact with each other directly. Instead, as shown in the bottom figure, JBI acts as an intermediary to route messages between components. This separation is the key to decoupling between service providers and consumers, which is one of the goal of service oriented architectures. In addition, it provides a key point for message processing and monitoring.
In this WSDL-based, service-oriented model, JBI plug-in components are responsible for providing and consuming services. By providing a service, a component is making available a function or functions that can be consumed by other components (or even itself). Such functions are modeled as WSDL 2.0 operations, which involve the exchange of one or more messages. A set of four WSDL-defined, basic message exchange patterns (MEPs) crisply defines the sequence of messages allowed during execution of an operation. This shared understanding, between consumer and provider components, of the message exchange pattern is the foundation of interoperability of such components in JBI.
The services provided by components (if any) are described to JBI by the component, using WSDL 1.1 or This provides an abstract, technology-neutral model of services using XML-based message exchanges. WSDL also provides a mechanism for declaring additional service metadata of interest to service consumers and JBI itself. Components can query JBI for the for the WSDL describing available services.

6. JBI 1.0: architecture l Java Business Integration l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
JBI 1.0: architecture
This picture describes the architecture of the JBI system.
The JBI environment provides deployment, control & monitoring features through JMX based administration tools
The Normalized Message Router or NMR, which provides the mediated message exchange infrastructure
Components (in white rectangles)
Components are divided in two categories:
Service Engines (on top) provides business logic and transformation services and also consumes such services.
Binding Components provide connectivity to services external to the JBI environment.
Service engines and binding components can function as service providers, consumers, or both. Note that the distinction between SEs and BCs is purely pragmatic, but the separation of business logic from communications logic reduces complexity and increases flexibility.
The JBI environment, in addition to the messaging system, defines a management structure based on JMX which provides standard mechanisms for:
Installing components
Managing a component’s life cycle (start / stop)
Deploying service artifacts to components
JBI components often function as a type of container to which artifacts can be deployed to add new service or provider logic. For example, a service engine that provides XSLT-based transformation services would have XSLT style sheets deployed to it, in order to add new transformation operations. This process of adding such component-specific artifacts to an installed component is called deployment, to distinguish it from component installation. Such an artifact is called a service unit, while collection of deployment artifacts and associated metadata is called a service assembly.
The core message exchange concept implements WSDL messaging. Service requests are generated by consumer components, routed by the NMR, and delivered to a provider component. For example, the BPEL SE may generate a request, which happens to be provided by the external service provider connected to the WS-I BC. The NMR will route the request to the WS-I binding. The SE in this case is a service consumer, and the BC a provider.

7. JBI 1.0: WSDL messaging model
l Java Business Integration
l Apache Servicemix u
Standards Based
Flexible
Reliable
Breadth of connectivity
JBI 1.0: WSDL messaging model
Abstract model
Concrete model
Service providers
Service Consumers
JBI models services produced and consumed by components using Web Service Description Language 1.1 or When the terminology differs between the two version, the 2.0 one is used. This is mainly the case for the interface and endpoint terms, which are called respectively “port type” and “port” in WSDL 1.1.
WSDL provides a declarative model of message-based services on two levels:
The abstract model defines services without references to a particular protocol or wire encoding
The concrete model is bound to a particular communication protocol and communications endpoint
JBI uses the abstract service model as the main basis for component interactions. Components play one of two roles in such interactions:
Service provider. The component that performs the given service (either directly or as a proxy for an external provider)
Service consumer. The component that invokes a given service (either directly or as a proxy for an external consumer)

8. JBI 1.0: Normalized Message Router
l Java Business Integration
l Apache Servicemix u
Standards Based
Flexible
Reliable
Breadth of connectivity
JBI 1.0: Normalized Message Router
Normalized Message
Runtime Endpoint Activation
Delivery Channel
Service Invocation using MEPs
Addressing Endpoints
The Normalized Message Router (or NMR) receives message exchanges from JBI components and routes them to the appropriate component for processing. This decouples the service producers from consumers and enable additional processing.
Normalized Message
JBI uses the concept of a normalized message for interactions between consumers and providers.
A normalized message consists of three parts:
A payload, which is an xml document that conforms to a WSDL message type without any protocol or encoding
Properties (or metadata) holds extra data associated with the message (security, transactions …)
Attachments are arbitrary (non-xml) data that contain a data handler to manipulate them
Endpoint Activation
Endpoint activation is the process by which a service provider tells the NMR that it provides services, making them known to the NMR so that it can route service invocations to that service.
The activation is split in two steps”
Declaring a service endpoint (service QName + endpoint name)
Providing metadata: the component has to provide a WSDL description of the activate endpoint
The WSDL description will be used by the NMR to know which interfaces and operations are implemented by the activated endpoint.
Delivery Channel
A Delivery Channel is a bidirectional asynchronous communication pipes between a component and the NMR.
A service consumer uses its delivery channel to initiate a service invocation, while the provider uses its delivery channel to receive such invocations.
Each component is provided with a single delivery channel, so the same channel is used for both inbound and outbound communications.
Service Invocation
Service invocation refers to an instance of an end-to-end interaction between a service consumer and a service provider.
Four patterns are supported by JBI:
In-Only: consumer issues a request to provider with no error (fault) path provided.
Robust-In-Only: consumer issues a request to provider. Provider may respond with a fault if it fails to process request.
In-Out: consumer issues a request to provider, with expectation of response. Provider may respond with a fault if it fails to process request.
In-Optional-Out: consumer issues a request to provider, which may result in a response. Consumer and provider both have the option of generating a fault in response to a message received during the interaction.
Addressing Endpoints
JBI uses the same concept of endpoints as defined in WSDL 2.0.
Endpoints refers to a particular address, accessible by a particular protocol, used to access a particular service.
Endpoints have two distinct types:
External endpoints are endpoints outside the JBI environment. They are endpoints exposed by binding component acting as service consumers to expose an internal endpoint for the use of external service consumers
Internal endpoints are exposed by service providers inside the JBI environment. They are accessed using the NMR APIs.
Binding components serve to map between internal and external endpoints. For example, an internal endpoint exposed outside the JBI environment by a binding component is mapped by the BC to an external provider endpoint for the use of external consumers.
In JBI, endpoints are referred to (or addressed) in three distinct ways:
Implicitly: the NMR selects the endpoint based on the service type required
Explicitly: a consumer chooses the endpoint based on its own logic and configuration
Dynamically: an endpoint reference (EPR) is used within a message exchange to provide a “ call-back” address that the service provider should use to send further message exchanges. EPRs are xml fragment created and resolved by JBI components.

9. JBI 1.0: Invocation example
l Java Business Integration
l Apache Servicemix u
Standards Based
Flexible
Reliable
Breadth of connectivity
JBI 1.0: Invocation example
This sequence diagram shows a InOut service invocation between a consumer and a provider.
The consumer creates an InOut message exchange, populates the “in” message (the request) and send it to the NMR
The provider component polls the delivery channel for an exchange
It processes the request, populates the “out” message and send it back to the NMR
The consumer polls the delivery channel for the response
It processes the response and mark the exchange as “done”
The provider receives the “done” status from the delivery channel
This example is an asynchronous invocation, but JBI also handles synchronous calls. In such a case, the consumer call to send would be a sendSync and would block the thread until the response is received. On the provider side, if the provider wish to synchronously deliver the response, the call to send (#3) would block until the consumer acks the response.
When an exchange involves a binding component, either as a consumer or a provider, the same diagram applies. The difference is that the exchange creation would be triggered by an external request (a JMS message, or an HTTP request) before step #1 and send the response after step #4. If the provider is a binding component which proxies a remote service, it would invoke the service between the steps #2 and #3.
People may wonder why there is a need for a “done” status… In JBI, all exchanges are terminated by either a “done” or an “error” status. Errors are different from faults which are parts of the normal exchange processing. So, the done status is very useful to implement reliable message, transactions or passing streams inside the exchange.
Let’s take the example a file binding component, which would poll for files, send the content as an InOnly request, and delete them afterwards. For performance reasons, it will send an opened stream on the file, but it has to close the stream and delete the file once processed. If the consumer is asynchronous (which is better because it saves threads), it needs a way to know when the file has to be deleted, hence the need for the “done” status.

10. JBI 1.0: Packaging META-INF/jbi.xml Components Shared Libraries
l Java Business Integration
l Apache Servicemix u
Standards Based
Flexible
Reliable
Breadth of connectivity
JBI 1.0: Packaging
META-INF/jbi.xml
Components
Shared Libraries
Service Units
Service Assemblies
JBI defines a standard packaging for installing new components and deploying artifacts onto these components that can function as “containers”.
All these artifacts contain a JBI descriptor in the META-INF directory called jbi.xml.
We have already discussed some of these artifacts, but let’s dive in them a bit more. There are four types of artifacts, all packaged as zip or jar files:
Components installers contain the libraries and resources needed to run the component. Components can references Shared Libraries.
Shared Libraries are collection of jars that can be shared by several components
Service Units are artifacts meant to be deploy to a specific component. Aside from the JBI descriptor, the service unit packaging is specific to a component and can take various forms: a single XSLT style sheet, a BPEL process, or some java classes …
Service assemblies are collections of Service Units. Service units can not be deployed directly and must be packaged in a service assembly which contains the service units and associated metadata.
A service assembly is thus a zip of zips. It can be compared to an EAR in the J2EE world. To ease the pain of packaging the JBI artifacts, ServiceMix provides a powerful maven based tooling which can package all the JBI artifacts and generate their JBI descriptors automatically. To ease the development more, we provide maven archetypes for JBI components and service units: archetypes are templates that can generate a project in one command line.

11. Reliable QoS attributes Flows Sync / Async ST Persistent SEDA
l Reliable
l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
Reliable u
QoS attributes
Sync / Async
Persistent
Transactional
Clustered
Flows ST
SEDA
JMS
JCA
ServiceMix can be configured for reliability. It uses Apache ActiveMQ under the cover, a well-known JMS broker, to achieve most of its reliable messaging.
When a JBI exchange is sent through the NMR, four attributes are used to define the required QoS.
Synchronous / Asynchronous: if the exchange has been sent using send or sendSync
Persistent: the exchange can be configured to be persistent
Transactional: a JBI exchange may be used to convey a transaction or to be itself enlisted in a transaction
Clustered: the NMR can be clustered (where the same JBI endpoints are activated on multiple containers) or distributed (where JBI endpoints are activate on a remote container)
ServiceMix uses the concept of flows, which affects how the NMR will route and distribute exchanges to JBI components. The combination of these QoS parameters will be used by the NMR to choose a flow for a given exchange, as not all flows can handle the same QoS. Multiple flows can be configured and allow to handle a variety of QoS, depending on the exchanges.
Four different flows are defined.
Straight through flow. A JBI exchange is routed straight to it's destination with no staging or buffering. A straight through flow is best suited for the cases where the ServiceMix container is deployed with simple flows (no state) or embedded, or where latency needs to be as low as possible. The ST flow can not handle persistent, transactional or clustered exchanges.
SEDA flow. The SEDA flow introduces a simple event staging between the internal processes in the NMR Broker. A SEDA flow (the default) is suited for general deployment, as the additional staging is well suited buffering exchanges between heavily routed to components (where state may be being used) for example. The SEDA flow can handle transactional exchanges.
JMS flow. A JMS Flow is used for cases where you need collaboration between more than one ServiceMix container (for fail-over or scalability). Component deployment happens in the same way as a normal ServiceMix container (both for POJO and standard JBI artifacts) but all the containers in the cluster are notified of a deployment, and the JMS Flow will handle automatic routing (and failover) of JBI exchanges between the container instances. A Message Queue is used for each JBI endpoint, so multiple instances of the same named Component will have requests load balanced across them. The JMS flow can handle persistent or clustered exchanges that are not transactional.
JCA flow. The JCA Flow is very similar to the JMS Flow, in that it allow to cluster ServiceMix containers together. The main difference is that it uses JCA to provide support for XA transactions when sending and receiving JBI exchanges. The JCA flow can handle all asynchronous exchanges (persistent, transactional or clustered).
JMS / JCA flows use an underlying Apache ActiveMQ JMS broker which can be configured to provide full replication for fail-over.

12. Breadth of Connectivity
l Breadth of Connectivity
l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
Breadth of Connectivity u
Binding Components
http (http/soap), jms (jms/soap)
file, ftp, xmpp
, RSS, Jabber…
Service Engines
Bean, EIP, Camel, JAX-WS, Drools, Quartz, Saxon, Script, WS-Notification
Third Party Components
ODE, CICS, Corba
Breadth of Connectivity
An agile ESB must support both two way reliable Web-services and Message Oriented-Middleware and needs to co-operate seamlessly with EIS and custom components, such as files, s or RSS feeds.

13. Roadmap ServiceMix 3.1.2 released in December
l Roadmap
l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
Roadmap
ServiceMix released in December
ServiceMix 3.2 in October
Work has began on ServiceMix 4.0
Simplified internal API
Based on OSGi
Built on top of Apache Camel
Breadth of Connectivity
An agile ESB must support both two way reliable Web-services and Message Oriented-Middleware and needs to co-operate seamlessly with EIS and custom components, such as files, s or RSS feeds.

14. ServiceMix 4.0: routing DSL
l ServiceMix 4.0: routing DSL
l Apache Servicemix
Standards Based
Flexible
Reliable
Breadth of connectivity
ServiceMix 4.0: routing DSL
from(“
tryBlock().
to(“activemq:queue:requests”).
 setOutBody(constant(“<ack/>”)).
handle(Throwable.class).
setFaultBody(constant(“<nack/>”));
from((“activemq:queue:requests?transacted=true”).
process(requestTransformer).
to(“
filter(xpath(“//nack”)).
process(nackTransformer).
to(“jdbc:store”);
from(“
to(“activemq:queue:responses”).
setOutBody(constant(“<ack/>”)).
from(“activemq:queue:responses?transacted=true”).
process(responseTransformer).
from(“
to(“jdbc:load”);
from(“
tryBlock().
to(“activemq:queue:requests”).
setOutBody(constant(“<ack/>”)).
handle(Throwable.class).
setFaultBody(constant(“<nack/>”)); 
Breadth of Connectivity
An agile ESB must support both two way reliable Web-services and Message Oriented-Middleware and needs to co-operate seamlessly with EIS and custom components, such as files, s or RSS feeds.

15. l Second part : Belgian Success Story
l BeJUG – Enterprise SOA’07 Conference
l 09/10/2007
l Second part : Belgian Success Story
l Anne Noseda - BULL

16. Belgium’s French Community
l Context of the project – Actors
l Belgian Success Story u
Context
Architecture
Development
Administration
Stress Tests
Belgium’s French Community
culture
education
research and training
health (preventive)
youth
sports
ETNIC
Information Technology agency of the Belgium’s French Community
founded in 2002
130 IT specialists
budget of  € 24 million

17. Use Case : pupils registration
l Context of the project – Goals
l Belgian Success Story u
Context
Architecture
Development
Administration
Stress Tests
Goals
Improve communication between Belgium’s French Community and other actors
Implement SOA
Use Case : pupils registration
Before : various communication ways (paper documents, mails, excel sheets, …)

18. Different actors  coordination
l Context of the project – Difficulties
l Belgian Success Story u
Context
Architecture
Development
Administration
Stress Tests
Difficulties :
Different actors  coordination
3,500 schools PO
Belgium’s French Community …
Different systems  interoperability
8,000 local applications
Delphi
.NET
Java
Access
Enterprise internal flows
Cobol
FTP
Web Services

19. Technical obligations
l Context of the project – Needs
l Belgian Success Story u
Context
Architecture
Development
Administration
Stress Tests
Needs
Loose coupling, transport of messages, interoperability  ESB
Security (authentication & authorisation)
Synchronous & asynchronous mode
Good performance (20 transactions / second)
Robustness
Overall management of errors
Development tools
Administration, monitoring and audit tools
Technical obligations
ESB JBI standard
BPEL engine
WS-Security & X509 Certificate Token
WS-Addressing

20. Development of the solution
l Context of the project – Timeline
l Belgian Success Story u
Context
Architecture
Development
Administration
Stress Tests
Project Timeline
3 steps
2006
2007
01/10
01/01
01/07
01/09
time
POC
Development of the solution
Tools
+ regular synchronisation points & testing with all the actors
 Evolution of the architecture
Proof Of Concept
performance
stability, robustness
functionalities

21. General architecture of the solution
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
General architecture of the solution
Servicemix Centralized Authentication Service : security
Servicemix Work : business
Broker JMS ActiveMQ
Servicemix tools (development & administration tools) u

22. Components used BC servicemix-http ws-addressing ws-security
l Components
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Components used
BC servicemix-http
ws-addressing
ws-security
specific security module
BC servicemix-jms
transaction
SE servicemix-eip
pipeline for MEP transformation
SE / BC servicemix-bean
specific needs  java development
JBI concerns
id generation, orchestration, schema validation, polling u

23. Components used SE / BC servicemix-jsr181
l Components
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Components used
SE / BC servicemix-jsr181
specific needs  java development
no JBI concerns (XFire)
security enricher, response writer, fault handler
SE Apache ODE
BPEL engine
SMX-WORK orchestration
heart of loose coupling u

24. Asynchronous communication
l Asynchronous mode
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Asynchronous communication
JMS between SMX-CAS and SMX-WORK
polling to retrieve responses u

25. Polling – correlation <wsa:MessageID> <wsa:RelatesTo>
l Asynchronous mode - polling
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Polling – correlation
<wsa:MessageID>
<wsa:RelatesTo> u

26. Synchronous communication
l Synchronous mode
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Synchronous communication
same SMX-CAS
HTTP between SMX-CAS and SMX-WORK
no Responses DB u

27. Cimero Open Source plug-in Eclipse developed by BULL drag and drop
l JBI process development
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Cimero
Open Source
plug-in Eclipse
developed by BULL
drag and drop u

28. Cimero Properties (icons, colours) Generic SU  open
l JBI process development
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Cimero
Properties (icons, colours)
Generic SU  open
xbean.xml generation
ant task to rebuild SU & SA after xbean.xml modifications u

29. Intalio Designer Open Source plug-in Eclipse
l BPEL process development
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Intalio Designer
Open Source
plug-in Eclipse
drag and drop, mapper, data editor u

30. Servicemix administration
l Administration tools
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Servicemix administration
Web application
Installation, deployment, list, start, stop of components, SA, SU u

31. ActiveMQ administration
l Administration tools
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
ActiveMQ administration
Web application
Queues and topics administration (view, purge, …)
Sending JMS messages u

32. OpenNMS Open Source 3 parts Java web application – console
l Monitoring tools
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
OpenNMS
Open Source
3 parts
Java web application – console
daemons (collecting data, …)
DB PostgreSQL u

33. Spagic Open Source 4 parts Console - Java web application
l Audit tools
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Spagic
Open Source
4 parts
Console - Java web application
Listeners : collecting messages
Monitors : messages
Meta DB u

34. Spagic console Process instances Messages XML l Audit tools
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Spagic console
Process instances
Messages XML u

35. Bi-processor quad core (8 CPU) Servicemix 3.1.1
l Stress Tests
l Belgian Success Story
Context
Architecture
Development
Administration
Stress Tests
Environment
Linux Redhat
Bi-processor quad core (8 CPU)
Servicemix 3.1.1
Stress Tests
1 to 1,000 threads
100 to 100,000 messages
throughput SMX-CAS : 40 to 75 msg/s
throughput SMX-WORK : 20 msg/s
Stability & Reliability Tests
2,300,000 messages in 30 hours u

36. ESB and SOA will be a standard
l Conclusion
l Belgian Success Story
Solution :
100% Open Source
Scalable
Apache Servicemix
able to cover all needs
high-performance
robust
Future at ETNIC
ESB and SOA will be a standard
solution will be extended to all communications between Belgium’s French Community and other actors

37. Blog: http://gnodet.blogspot.com
l Questions ?
Guillaume Nodet
Blog:
Anne Noseda

38. l Project's partners