2. EMEA 2 Enterprise Integration Erik Doernenburg ThoughtWorks, Inc. edoernen@thoughtworks.com

3. EMEA © Copyright ThoughtWorks, Inc.® 2005 3 Agenda  Challenges in enterprise application development  Design patterns  Enterprise Integration patterns  Interop technologies  Conclusion & Resources

4. EMEA © Copyright ThoughtWorks, Inc.® 2005 4 The challenge  Enterprises depend on a growing number of IT systems  The systems must provide an integrated solution for the enterprise  The systems must interoperate with each other  Architectural trends and “waves” of technologies  Changing business needs and requirements

5. EMEA © Copyright ThoughtWorks, Inc.® 2005 5 Existing Systems New System Finance Operator Application  Workflow items are stored on Tibco RV/CM queue  Tibco Integration Manager  organises workflow  handles imports from document recognition  handles interaction with existing ERP systems  Reference data imported using linked servers fax/telex email maildocument recognition app web ERP ref. data

6. EMEA © Copyright ThoughtWorks, Inc.® 2005 6 New application Existing Portfolio Management Tool  Front-end uses Office 2003 code-behind technology  communicates with server using WebSphere MQ queues  Message format defined with XSD schema  Application uses existing services  accesses analytics library with COM bridge  accesses stored deals through deal manager service  accesses service bus through messaging abstraction library analytics library Existing deal manager Existing messaging library technology adaptor Excel front-end application logic

7. EMEA 7 Design Patterns

8. EMEA © Copyright ThoughtWorks, Inc.® 2005 8 Why design patterns?  Code reuse remains difficult but…  Knowledge reuse can be very valuable  Patterns encapsulate knowledge of successful designs

9. EMEA © Copyright ThoughtWorks, Inc.® 2005 9 Using technology successfully  Syntax  Basic language mechanism  A necessity but not a guarantee  Constructs  Classes, Interfaces, Inheritance, Polymorphism, etc.  Helpful but not sufficient for good design  Principles  Separation of concerns, Dependency Injection, etc.  Steer us towards a better solution  Patterns  Model-View-Controller, Observer, Decorator  Concrete design guidance

10. EMEA © Copyright ThoughtWorks, Inc.® 2005 10 Patterns  “Mind sized” chunk of information (Ward Cunningham)  Shows a good solution to a common problem within a specific context  Observed from actual experience  Has a distinct name  Not copy-paste  ThoughtWorks collaborates with Microsoft to document design patterns for the.NET platform

11. EMEA 11 Designing with Enterprise Integration Patterns

12. EMEA © Copyright ThoughtWorks, Inc.® 2005 12 Enterprise Integration Patterns  Gregor Hohpe defined a visual pattern language describing message-based enterprise integration solutions  Pattern language comprises 65 patterns in 6 categories Application A B Message Channel RouterTranslator Endpoint Monitoring Messaging Endpoints Messaging Channels Message Construction Message Routing Message Transformation System Management

13. EMEA 13 Request/Response patterns

14. EMEA © Copyright ThoughtWorks, Inc.® 2005 14 Pattern: Request-Response  Service Consumer and Provider (similar to RPC)  Channels are unidirectional  Two asynchronous point-to-point channels  Separate request and response messages Request Channel Response Request Reply Channel ProviderConsumer

15. EMEA © Copyright ThoughtWorks, Inc.® 2005 15 Multiple consumers  Each consumer has its own reply queue  But how does the provider send the response?  Could send to all consumers (very inefficient)  Hard code (violates principle of context-free service) Responses Requests ? ? Request Channel Reply Channel 1 Reply Channel 2 Provider Consumer 1 Consumer 2

16. EMEA © Copyright ThoughtWorks, Inc.® 2005 16 Pattern: Return Address  Consumer specifies Return Address (the reply channel) in the request message  Service provider sends response message to specified channel Responses Reply Channel 1 Reply Channel 2 Request Channel Reply Channel 1 Reply Channel 2 Provider Consumer 1 Consumer 2

17. EMEA © Copyright ThoughtWorks, Inc.® 2005 17 Load-balanced service providers  Request message can be handled by multiple providers  Point-to-point channel supports competing services  Only one service receives each request message  But what if the response messages are out of order? Request Channel Reply Channel Provider 1 Provider 2 Consumer

18. EMEA © Copyright ThoughtWorks, Inc.® 2005 18 Pattern: Correlation Identifier  Consumer assigns a unique identifier to each message  Identifier can be an arbitrary ID, a GUID, a business key  Provider copies the ID to the response message  Consumer can match request and response Message Identifier 12121212 12 12 Correlation Identifier Request Channel Reply Channel Provider 1 Provider 2 Consumer

19. EMEA 19 An order processing system

20. EMEA © Copyright ThoughtWorks, Inc.® 2005 20 Multiple specialised providers  Each provider can only handle a specific type of message  Route the request to the "appropriate" provider. But how?  Do not want to burden sender with decision  Letting providers "pick out" messages requires coordination Order Entry Order Entry Widget Inv. ? ? Order Messages Gadget Inv.

21. EMEA © Copyright ThoughtWorks, Inc.® 2005 21 Pattern: Content-Based Router  Insert a content-based router  Routers forward incoming messages to different channels  Message content not changed  Mostly stateless, but can be stateful, e.g. de-duper Order Entry Order Entry Widget Inv. Order Messages Gadget Inv. Content Based Router

22. EMEA © Copyright ThoughtWorks, Inc.® 2005 22 Composite messages  How can we process a message that contains multiple elements? Order Entry Order Entry ? ? Order Message Widget Inv. Gadget Inv.

23. EMEA © Copyright ThoughtWorks, Inc.® 2005 23 Pattern: Splitter & Router  Use a splitter to break out the composite message into a series of individual messages  Then use a router to route the individual messages as before  Note that two patterns are composed Widget Inv. Gadget Inv. Order Entry Order Entry RouterSplitter Order Message Order Items Order Item 1 Order Item 2

24. EMEA © Copyright ThoughtWorks, Inc.® 2005 24 Producing a single response  How to combine the results of individual but related messages?  Messages can be out-of-order, delayed  Multiple conversations can be intermixed Widget Inv. Gadget Inv. Billing Order Item 1 Response 1 Order Item 2 Response 2 Confirmed Order ? ?

25. EMEA © Copyright ThoughtWorks, Inc.® 2005 25 Pattern: Aggregator  Use a stateful filter, an Aggregator  Collects and stores messages until a complete set has been received (completeness condition)  Publishes a single message created from the individual messages (aggregation algorithm) Widget Inv. Gadget Inv. Billing Aggregator Order Item 1 Response 1 Order Item 2 Response 2 Confirmed Order

26. EMEA © Copyright ThoughtWorks, Inc.® 2005 26 Communicating with multiple parties  How to send a message to a dynamic set of recipients?  And return a single response message? Request For Quote Vendor A Vendor B Quotes Aggregator Best Quote Vendor C ? ?

27. EMEA © Copyright ThoughtWorks, Inc.® 2005 27 Pattern: Scatter-Gather  Send message to a pub-sub channel  Interested recipients subscribe to a "topic"  Aggregator collects individual response messages  may not wait for all quotes, only returns one quote Request For Quote Vendor A Vendor B Quotes Aggregator Best Quote Vendor C Pub-Sub channel

28. EMEA © Copyright ThoughtWorks, Inc.® 2005 28 Complex composition  Receive an order message  Use splitter to create one message per item  Send to scatter/gather which returns "best quote" message  Aggregate to create quoted order message Quote request for each item Vendor A Vendor B Quotes Aggregator Best Quote for each item Vendor C Aggregator Splitter Pub-Sub channel

29. EMEA 29 Interop technologies

30. EMEA © Copyright ThoughtWorks, Inc.® 2005 30  Resource based  RPC style / distributed objects  Message based / document-oriented Major interop technologies DBfiles RMI-IIOP (CORBA) MOM in-process bridge WS-* WS Remoting (DCOM) WS

31. EMEA © Copyright ThoughtWorks, Inc.® 2005 31 Interop technology characteristics point-to-point transient messages durable message server lifetime- management bridge Remoting (DCOM) RMI-IIOP (CORBA) WSMOMDB/files routable WS-* clustering in-proc.NET server J2EE server platform neutral WSMOMWS-*

32. EMEA © Copyright ThoughtWorks, Inc.® 2005 32 Messaging  Channels are separate from applications  Removes location dependencies  Channels are asynchronous & reliable  Removes temporal dependencies  Data is exchanged in self-contained messages  Removes data format dependencies  Loosely coupled integration enables independent variation

33. EMEA © Copyright ThoughtWorks, Inc.® 2005 33 Why Web Services?  Web services provide all benefits of messaging solution  loose-coupling  service oriented  reliable communication  Why web services?  composable protocol  vendor neutral  suitable for access through Internet

34. EMEA © Copyright ThoughtWorks, Inc.® 2005 34 Web Services development  Web Services are expected to become the default Messaging solution in the future  WS-* standards are evolving rapidly, most important  WS-Security  WS-Policy  WS-Addressing  WS-I defines profiles, sample applications and testing tools  Profiles are guidelines for using WS specifications  Use Basic profile 1.1 to build interoperable solutions  Further research on WS is being carried out

35. EMEA 35 Conclusion

36. © Copyright ThoughtWorks, Inc.® 2005 36 Conclusion  Enterprise systems are becoming more complex  We have patterns to help us with the design  We have technologies to implement our designs  Building for interoperability and integration is key

37. EMEA © Copyright ThoughtWorks, Inc.® 2005 37 Enterprise Solution Patterns using Microsoft.NET using Microsoft.NET Microsoft Patterns & Practices 2003 Recommended books Patterns of Enterprise Application Architecture Patterns of Enterprise Application Architecture Martin Fowler Addison-Wesley, 2003 Enterprise Integration Patterns Enterprise Integration Patterns Gregor Hohpe, Bobby Woolf Addison-Wesley, 2004 Integration Patterns Microsoft Patterns & Practices 2004 Enterprise SOA Enterprise SOA Dirk Krafzig, Karl Banke, Dirk Slama Prentice Hall, 2004

38. Please complete your session feedback form THANK YOU