2. EMEA 2 Evolutionary Enterprise Enablement using BizTalk 2004 Nick Hines ThoughtWorks, Inc. nphines@thoughtworks.com

3. EMEA © Copyright ThoughtWorks, Inc.® 2004 3 Agenda  Client Context  Standards and guidelines  Design Patterns and BizTalk 2004  Test Driven Development in Integration Projects  Automated deployment using BizTalk 2004

4. EMEA © Copyright ThoughtWorks, Inc.® 2004 4 Client Context

5. EMEA © Copyright ThoughtWorks, Inc.® 2004 5 Initial Client State  Enterprise formed from many smaller companies  Large number of technologically disparate systems  Existing integrations ad-hoc and point to point  600+ point to point interfaces, 200+ content types, 50+ transport types  Lack of resilience in existing integrations  Difficult for IT support to monitor integrations

6. EMEA © Copyright ThoughtWorks, Inc.® 2004 6 Aspiration and Vision  Align integration strategy with business strategy  Migrate towards Event Driven Enterprise/Service Orientated Architecture  Reduce integration complexity  Single platform for integration infrastructure

7. EMEA © Copyright ThoughtWorks, Inc.® 2004 7 Challenges  Limited business buy-in  Difficult to calculate business value  Limited internal IT resources  Resistance from current project teams  Limited experience of BizTalk 2004

8. EMEA © Copyright ThoughtWorks, Inc.® 2004 8 The Prize  Potential Cost Savings associated with Enterprise Architecture (Source: Object Architects/Generali Informatik) Consolidating the project portfolio Consolidating the application portfolio Reduced project costs (~10%) Reduced operation costs (~5->15%) Optimizing IT infrastructure Reduced infrastructure costs (~5->10%) Optimizing resource management Reduced personnel costs (~10->20%) 20-35% cost reduction when all 4 levers are used

9. EMEA © Copyright ThoughtWorks, Inc.® 2004 9 Standards and Guidelines

10. EMEA © Copyright ThoughtWorks, Inc.® 2004 10 Evolving standards and guidelines  Big Design Up Front - protracted and expensive process  Little business or IT traction => shelf-ware  Real systems = real standards + direct business value  “Living” documents reflect current reality  Generate technical documentation wherever possible

11. EMEA © Copyright ThoughtWorks, Inc.® 2004 11 Best Practices  Design Patterns to aid communication and guide implementation  Use Test Driven Development to facilitate and support change  Use automated build and deployment to ensure consistency

12. EMEA © Copyright ThoughtWorks, Inc.® 2004 12 Engagement Model  Determines how proposed new applications approach integration  Covers all stages of lifecycle  Determines required artefacts/tasks  Will evolve over time

13. EMEA © Copyright ThoughtWorks, Inc.® 2004 13 Design Patterns

14. EMEA © Copyright ThoughtWorks, Inc.® 2004 14 Design Patterns  “Nuggets of advice that describe solutions to frequently recurring problems” – Gregor Hohpe  Most reusable software artefact  Capture best practice in implementation neutral manner  Facilitates higher level of communication

15. EMEA © Copyright ThoughtWorks, Inc.® 2004 15 Enterprise Integration Patterns  No simple answers for enterprise integration  Few established standards – XML/XSD, Web Service  Limited control over participating applications  Disparate application technologies, O/S, platforms, etc.

16. EMEA © Copyright ThoughtWorks, Inc.® 2004 16 “Seed” application – Territory Sales  Ongoing project to improve MI  40+ Territory Sales systems  Combination of proprietary and bespoke systems  Proposed project to integrate Sales with Budget and Forecasting

17. EMEA © Copyright ThoughtWorks, Inc.® 2004 17 Territory Sales Integration

18. EMEA © Copyright ThoughtWorks, Inc.® 2004 18 Messaging Problem  How do we share sales information between multiple applications? Alternative  File transfer, Shared database, RPC Solution  Decoupled, asynchronous, disconnected, point-to-point, publish/subscribe, etc. Cost  Asynchronous not “natural”, testing and debugging complex

19. EMEA © Copyright ThoughtWorks, Inc.® 2004 19 Canonical Data Model Problem  How do we reduce dependencies on individual territory LOBs data formats? Alternative  Translate messages between each pair of applications Solution  Create data format independent of any application Cost  Double translation, difficult to design and agree! Point to point Canonical

20. EMEA © Copyright ThoughtWorks, Inc.® 2004 20 Channel Adapter Problem  How do we connect territory LOBs to messaging? Alternative  Modify custom applications. What about packages? Solution  Invokes application supplied interface, acts as client to messaging system BizTalk  SAP, J D Edwards, PeopleSoft, CORBA, SNA, JMS, etc. Cost  Message formatting driven by application internal structure

21. EMEA © Copyright ThoughtWorks, Inc.® 2004 21 Guaranteed Delivery Problem  How can we ensure messages will be sent if messaging system fails? Alternative  Resend messages if not received Solution  Store messages to persistent store, don’t delete until consumed BizTalk  Message Box “guaranteed”, (even clustering not 100%!) Cost  Performance, need large amounts of disk if high volume

22. EMEA © Copyright ThoughtWorks, Inc.® 2004 22 Message Translator Problem  How do we cope with different sales message formats? Alternative  Modify applications to use common format Solution  Use component to convert one format to another BizTalk  Extensive translation support via maps, (XSLT) Cost  Performance, especially if many levels of translation

23. EMEA © Copyright ThoughtWorks, Inc.® 2004 23 Message Splitter Problem  How do we process sales data delivered in batches? Alternative  Modify subscribers to process batch and single messages Solution  Break composite message into individual messages BizTalk  “Interchange” processing slow => custom component Cost  Processing overhead, calling out of message system

24. EMEA © Copyright ThoughtWorks, Inc.® 2004 24 Content Enricher Problem  How do we add Enterprise Header to all sales messages? Alternative  Modify sender, have subscriber add missing data Solution  Specialised translator to add “missing” fields BizTalk  Supports default values, DB lookups, HTTP, etc. via maps Cost  Increased processing time, lookups introduce additional dependencies

25. EMEA © Copyright ThoughtWorks, Inc.® 2004 25 BizTalk Map Editor  Provides extensive support for Message Translator  Provides extensive support for Content Enricher

26. EMEA © Copyright ThoughtWorks, Inc.® 2004 26 Publish-Subscribe Problem  How can the territory LOBs send messages to all parties? Alternative  Sender needs to know and wait for all consumers Solution  Send copy of each message to all interested applications BizTalk  Supported by Send/Receive Ports and Orchestrations Cost  More difficult to track message dependencies, more complex message flow

27. EMEA © Copyright ThoughtWorks, Inc.® 2004 27 Durable Subscriber Problem  How do inactive subscribers avoid missing messages? Alternative  Publisher stores and resends Solution  Use messaging system to save published messages while subscriber inactive BizTalk  Message Box provides this functionality Cost  Disk space if subscriber inactive for long periods

28. EMEA © Copyright ThoughtWorks, Inc.® 2004 28 Test Driven Development in Integration Projects

29. EMEA © Copyright ThoughtWorks, Inc.® 2004 29 Test Driven Development in EAI  Test normal applications, more so enterprise applications  Traditional integration testing manual and sporadic  Testing difficult, complex scenarios, hard to achieve “base state”  Need unit/subsystem testing  Challenging because integration solutions often “black- boxes”  Need a testing framework

30. EMEA © Copyright ThoughtWorks, Inc.® 2004 30 Components of Testing Framework  Generator – creates test data and passes to component under test  Verifier – Compares actual with expected results  Stub – Simulates part of system not under test

31. EMEA © Copyright ThoughtWorks, Inc.® 2004 31 Unit/Component Testing  Focus on specific component of system  Use stubs to eliminate outside dependencies  No substitute for end to end, (integration) testing

32. EMEA © Copyright ThoughtWorks, Inc.® 2004 32 Integration Testing  Focus on interaction between components  Bottom up testing approach  Failure indicates problem at that level or interaction

33. EMEA © Copyright ThoughtWorks, Inc.® 2004 33 BizTalk Test Framework  No test facilities built in to BizTalk  Developed custom adapter to allow creation of stubs and verifiers  Creates stub send and receive ports  Zero impact on system under test  API via NUnit test framework  BTSUnit – http://gforge.public.thoughtworks.org/projects/btsunit

34. EMEA © Copyright ThoughtWorks, Inc.® 2004 34 Automated deployment using BizTalk 2004

35. EMEA © Copyright ThoughtWorks, Inc.® 2004 35 Deployment challenges  “Out of the box” deployment manual and limited  Many configuration parameters; port bindings, hosting, URI’s, etc.  Need different configuration for development, test, stage, production  Need to maintain parameters in single easily modifiable location

36. EMEA © Copyright ThoughtWorks, Inc.® 2004 36 BizTalk deployment tools and techniques  Combination of NAnt, WIX, and WMI scripts  Developed by Scott Colestock, (www.traceofthought.com) extended at clientwww.traceofthought.com  Supports parameterization of multiple environments via Excel sheet  Provides MSI based deployment with automatic dependency detection

37. EMEA © Copyright ThoughtWorks, Inc.® 2004 37 Conclusion Iterative approach + Best practices + Patterns + Testing = Success!

38. Please complete your session feedback form THANK YOU