Agile SOA Agile EAI How do we achieve agility in Enterprise Integration?

What is Enterprise Integration?  Enterprise Integration is the main activity of an in-house software development organization. Building software products is not our main activity.  Why? DTE is in a mature industry and a relatively stable environment. As a result we can buy almost all of the software we need cheaper than we can build it.  But we still need to create interfaces for almost everything we buy.  Enterprise Integration means building application-to-application (A2A) and business-to-business (B2B) interfaces.

Contents Context Analysis Planning Managing Testing Deploying What is Enterprise Integration? The message is the feature-set. Release planning is a multi-player game. Concurrent engineering requires a different approach to project management. Re-usable components and services require automated test suites. Deployment requires configuration control.

The message is the feature.  The message is the feature in EAI and SOA.  Analysis: Enterprise Patterns.  How to identify the messages quickly and simply.  Messages are speech acts.  Identify messages in complete sentences.  Nouns identify business objects (data entities).

The message is the feature … because:  Enterprise Integration implements communications between business processes.  The simple way to plan the integration features is to think about the messages that are exchanged between business processes.  A major application is the IT proxy or interface for some business process.  A2A or B2B -- It is the same thing. What if your enterprise decides to out-source a business process?

Analysis: What to look for … In-coming Messages  Request  Work Order Request  Question  Work Order Status  Entity State (data)  Subscribed Event  External Event of Interest to Me  Data Subscribed Out-going Messages  Response  Work Order Complete  Answer  Progress / Exception  Business Object (data)  Published Event  Event I Observe or Validate of Interest to Others  Data Published

Messages are speech acts. Express each message as a complete sentence.  Request: Please install a new service S at location L. ervice Slocation L  Response: Service S installed at location L.  Question: What is the status of work order W?  Notice: Inventory asset A moved from location L1 to location L2.

Noun ==> Business Object Write each message as a complete sentence. Use nouns in a consistent way. Avoid application-specific jargon. service S location L  Request: Please install a new service S at location L. Service Slocation L  Response: Service S installed at location L. work order W  Question: What is the status of work order W? asset A location L1location L2  Notice: Inventory asset A moved from location L1 to location L2.

Message Patterns ==> Design Patterns  Request-Response (work order … publish-subscribe)  Client: Please replace meter M at location L with a meter of type T.  Service: Meter M1 with reading R1 replaced with meter M2 reading R2.  Question-Answer (query … remote procedure call)  Client: Order status S = What is the status of work order W?  Service: Work order W has status S.  Event Notice (event … publish-subscribe)  Publisher: Work order W will be delayed due to problem P.  Subscriber: (updates local work management)

Example: Metering Service In-coming Messages  Request: (We accept the request.) Please install a meter of type T for service S at location L.  Query: (We answer the question.) Metric M = How much energy has been used at meter M?  Response: (When the work is done, the delegate responds with work order results.) Work order W completed normally with results R  Notice: (We subscribe to notice.) Meter M needs a new battery before date D. Out-going Messages  Request: (We delegate work to …) Please install a meter of type T for service S at location L.  Query: (We ask the delegate.) Status S = What is status of work order W?  Publish: (We publish the event.) Meter M is monitoring service S at location L.

Business Collaboration - a Use Case Scenario FromToMessage ClientMetering Service Please replace meter M at location L with a meter of type T. (This is work order WO-A received from the Client.) Metering Service Field Service Please replace meter M at location L with a meter of type T. (This is work order WO-B delegating work to Field Service.) Field Service Metering Service Meter M1 reading R1 replaced with meter M2 reading R2. (Work order WO-B completion report received from Field Service.) Metering Service Client Meter M1 reading R1 replaced with meter M2 reading R2. (This is work order WO-A completion report sent to Client.) Metering Service Subscribers Meter M1 reading R1 replaced with meter M2 reading R2. (This event notice is published for all interested subscribers.) Subscriber(self)Each subscriber updates its world-model (stored in a database) to reflect the business event. There may be additional side-effects.

Release planning is a multi-player game. Planning and Budgeting  Business Process -- owner / governor  Application Portfolio -- owner / governor  Component Library -- architect / planner Design and Construction  Software Architect -- designer / planner / coordinator  Project Manager -- planner / coordinator  Software Developer -- builder / tester / operator / maintainer

How do you plan a release? A release is a coherent set of features  for a specific audience (UI feature set)  for a specific process (EAI feature set) Ask which features are necessary and sufficient.  Is data replication necessary and sufficient?  Is passive notification of exceptions sufficient?  Is workflow needed?

Interface Packaging ==> Wrappers within wrappers publish subscribe Business Process A Application 1 Application 2 Application 3 Business Process B Application 4 Application 5 Application 6 Message Delivery Service Question Answer Create services by building wrappers on business services, legacy applications, and the message delivery system.

Business Service Wrapper Business Process == Business Service Application A Application B Application C Business Process Workflow Request Response Published Event Subscribed Event

Legacy Application Wrappers Business Process == Business Service Application A Application B Application C Request Response Published Event Subscribed Event Database Question-Answer

Work Breakdown Structure Sender  Detect the event / need.  Assemble the business objects into the message.  Send the message. What about workflow?  Workflow is essential for handling business-process exceptions.  Each process manages its own workflow and no other workflow. Receiver  Receive the request.  Translate the business objects into local form.  Call the equivalent method. Workflow Responsibilities  Delegate work as needed.  Respond to process exceptions.  Publish events.  Feed back results.  Collect statistics.

Concurrent Engineering Who should do which components?  Each application’s maintenance team has the local knowledge to …  Write logic to detect events.  Write logic to translate between local representation and standard representation.  Write application wrappers (API).  Each business-process team has the knowledge to …  Define and implement local workflows.  Define and implement end-to-end tests.  Design business-process monitoring (dashboard displays and reports).  A tooling team has the skill and focus to …  Create commonly-used components and services.  Create and maintain a repertoire of common business objects.  The project coordinators have the skills and tools to …  Identify project-to-project dependencies.  Coordinate release plans. Messages and business objects provide clear and succinct points of contact for design discussions and release coordination

SOA requires automated test suites. Why do you need automated test suites?  If you have a service with clients, and you are about to deploy a new release … You can ask all of the clients to re-test. -OR- You can exhaustively test the service to assure backward compatibility.  For a service of any complexity, automated tests are the only robust solution.  By definition, integration tests cross system boundaries.  Business process tests tend to require set-up, tear-down, etc. Some services have 100’s of clients

Configuration Control Challenges  Where is each component used?  To whom do we send release notices?  Where is this service used?  When the service is down, who is affected?  Which version is used where?  When a method is deprecated, who is at risk?

Summary  Business processes communicate via messages.  That means the message is the feature (the user story) for agile EAI and agile SOA.  Secret Sauce: Identify each message with a complete sentence.  Secret Organization: Concurrent engineering reduces time to solution.

To achieve EAI / SOA agility: Focus on the business-process message exchanges.