1. Choosing an application integration strategy Anko Duizer Class-A

2. Overview Why integration? What is integration? Challenges Common scenario’s Which technologies does Microsoft offer? Messaging Design considerations A different viewpoint: The Ring(s) The Matrix

3. Why? Enterprises are typically comprised of hundreds applications –Custom built, acquired by a third party, legacy systems Reasons –Creating a single, big application to run a complete business is impossible –Flexibility to select “the best” –Use a collaborative partner model

4. Those who can not remember the past are condemned to repeat it!

5. The current situation F B E A D C

6. What? Connecting –Computer systems –Companies –People Application integration is the secure and orchestrated sharing of processes and/ or data between applications

7. Integration scenarios Information portals Data replication Shared business functions Service-oriented architectures Distributed business processes Business-to-business integration

8. Basic challenges Networks are unreliable Networks are slow Any two applications are different Change is inevitable Organizational problems

9. Integration approaches Simple association Central hub The big database approach Message bus Service integration Hybrid

10. Simple association UI BO Data Application UI BO Data Application Endpoint

11. Central hub UI BO Data Application UI BO Data Application Endpoint ERP System

12. The big database approach UI BO Data Application UI BO Application

13. Message bus UI BO Data Application UI BO Data Application Endpoint Message bus

14. Service integration (1) UI BO Data Application UI BO Data Application Endpoint Service

15. Service integration (2) Application Endpoint Service Endpoint Application Endpoint Application Endpoint Application

16. Microsoft technology BizTalk Server 2004 Web Services MSMQ DTS Indigo (not available yet) SQL Service Broker (not available yet)

17. BizTalk Server 2004 action orchestration designer biztalk editor biztalk mapper messaging management developmentoperation management administration tracking messaging services orchestration services config tracking port pipeline designer applications monitoring Business rules

18. Web Services Host Component Client Application Host Page Browser XML over HTTP HTTP request.NET COM EJB Perl … listener proxy Internet

19. MSMQ Queue msg Sender Receiver

20. DTS OLE DB ODBC Fixed field ASCII delimited Source OLE DB ODBC Fixed field ASCII delimited Repl. publication Destination DTS Data Pump In Out ActiveX Script Copy Trim String … Custom Transforms

21. Indigo Port Service Message Channel Service Channel

22. SQL service broker SQL Server objectDescription Message TypeDefines valid messages for exchange between services ContractSpecifies type of messages and their direction in a conversation, initiator or target QueueStores messages before sending and after receipt as result set Service ProgramThe part of a service broker application that reads messages from a queue and processes them. ServiceAddressable endpoint for service communication msg Service Contract Service program Queue Message type

23. Messaging Loosely coupled Communicating asynchronously Reliable Makes the messaging responsible for transferring data from one application to another

24. Sync or async? time Process A Process B blocked time Process A Process B

25. Basic messaging concepts Application Channel Endpoint Message Translation Routing

26. Messaging patterns Y X Y X 1) Fire and Forget No Response required 2) Request / Reply System makes request and gets response 3) Needs ability to cache response from multiple requests to respond to one message Y X EAI B X 4A/B) Multiple Applications needed to satisfy request 4A: XREF Look-up 4B: XREF Update AC XREF EAI A X 5) Publish Subscribe A, B and C subscribe to message B C EAI 6A) Needs data from Z to be able to update Y Y X EAI Z

27. The message is the message!

28. Main design considerations Timing/ Latency Identification –Industry keys, Internal keys, Chained keys & Matching algorithms Sharing entities, activities, processes and utilities Designing endpoints Message transformation

29. Main design considerations Routing Security System management ACID Transactions Service-oriented architectures

30. Shared data ownership E-Commerce Inventory E-Commerce

31. Choosing

32. To EAI or to ETL? Data (ETL) Data Transformation Services Messages (EAI) BizTalk Server 2004 Unit of modeling “Package” – an XML definition of “workflow” and data flow Schema that capture structure/ Contracts that capture behavior Unit of interaction “Synchronous” DTS pipeline for data movement Asynchronous message passing Instancing DTS Service controls instancesWeak/opaque URI-based references Extensibility Scripts, SQL, custom components Intermediaries and message routing Platform DatabasesMultiple

33. Evaluation criteria CriterionDescription LatencyHow quickly is the data to be transferred? TransformationComplexity of the transformation ScopeApplication, organization or partner? VolumeQuantity of exchange by a transfer IntrusionDegree of change to existing applications in order to effect transfer EffortEffort required to build and maintain the solution OrganizationWhich solution fits best with the organization?

34. The Ring(s) 1. Application 2. Enterprise 3. Collaborative partners 4. Everybody else

35. The Matrix BizTalkSSBIndigoMSMQDTS LatencyNear real time Real timeNear real time Batch TransformationHigh N/A High ScopeEnterprise & partners App. App. & Enterprise VolumeMedium LowMediumHigh IntrusionMediumLowHigh Low EffortHighMediumHighMedium Organizationtbd

36. The Matrix - Reloaded IndigoMSMQSSB Environment Any WS-* compliant WindowsSQL Server 2005 (both) Application Any distributed application AsynchronousDatabase application Message store In-memory or database NT File system SQL Server 2005 Type of message Persistent & Non-peristent Reliable, Express, Transaction Transactional only Protocol Various TCP only

37. The Matrix - Revolutions Ring0123 CommunicationSynchronousAsynchronous FocusPerformanceSecurity Integration Security Integration Firewall IndigoYesMaybeNo BizTalk ServerNoYes WS-*MaybeYes MSMQNoYesNo SSBYesNo

38. Conclusions Integration is hard! Integration requires good thinking! Different technologies available, use them smartly! Start thinking asynchronously and message based!

39. References Books –Enterprise integration Patterns, Hohpe & Wools –Enterprise Integration Solutions, Spackman & Speaker Web –http://www.class-a.nl –http://www.objectwatch.com –http://msdn.microsoft.com/architecture

40. Questions ?

41. The Rings(s) 1.Synchronous,.NET or J2EE?, Performance, Indigo, SSB, Web Services 2.Asynchronous, MSMQ or MQSeries?, DTS, Security, WS-* 3.Asynchronous, BizTalk Server, Security, WS-* 4.Firewalls, for the rest you don’t care!