A Software AG Perspective (?) Jim Wallace - Senior Architect EAI & Enterprise Data Availability

IRMAC Presentation – 21/NOV/ The IT Paradox There is nothing new under the sun …..

IRMAC Presentation – 21/NOV/ Agenda Introduction & Agenda Maximizing Data Utility in the Enterprise An Alternative Approach (EAI) Enterprise Integration Patterns Enterprise Integration Toolsets Summary and Questions

IRMAC Presentation – 21/NOV/ Maximizing Data Utility in the Enterprise Expansion of the IS domain – especially the rise of Electronic Business – has greatly accelerated the need and the usage of data across and between enterprises Total Data Availability Means:  Information where you want it  When you need it to be there  In the form and state satisfies your needs In short, everything that we’ve been taught but never quite been able to achieve

IRMAC Presentation – 21/NOV/ My Definition of Electronic Business Electronic Business: Ability to co-operatively initiate and complete transactions through the exchange of information over the network, preferably without having to resort to other modes of communication, and possibly without human intervention.

IRMAC Presentation – 21/NOV/ Historical Strategy & Tactics Structure the data as flexibly as possible Centralize information wherever possible Leverage data replication Build distributed databases Separate transactional and analytical stores of data Real time processing of data Data cleansing and restructuring We have eagerly seized on technological advances to do this more cost effectively

IRMAC Presentation – 21/NOV/ The Aim – The One Big Database Data Access Modules Data Access Modules Application Layer Data Access Modules Persistence Layer

IRMAC Presentation – 21/NOV/ Complexity Rules! Considered in their own terms, all of these approaches were successful Just the problem became bigger  Domain Complexity  Solution Complexity  Technological Complexity It keeps changing faster than we can keep up

IRMAC Presentation – 21/NOV/ Even As Enterprises Try to Reduce It !! Control

IRMAC Presentation – 21/NOV/ One Big Database – Not! In practical terms, it has proven to be impossible to develop and to implement all-inclusive, enterprise wide data models While the effort should and must continue, additional approaches are required This is where Enterprise Application Integration (EAI) has a contribution to make It yields some of the advantages of the O(h)BD, in a more cost effective manner

IRMAC Presentation – 21/NOV/ A Digression – Process & Applications The role of EAI in the enterprise has been driven by the need to support higher level business processes. Things like -  “Straight Through” processing  Customer Self Service  Unified (WEB) Interfaces Offering Multiple Services (Portals)  Greater movement across enterprise boundaries (Electronic Business) The application of EAI neither exclusively nor primarily a data level focus  Although the end result both depends upon and results in increased data availability EAI normally makes data more available – and more useful – by “sharing” it among collaborating applications

IRMAC Presentation – 21/NOV/ Multiple Levels Of Integration Essentially a classification of techniques used for integration between applications or app components  Data Level  Method Level  Application Programming Interface (API) Level  User Interface Level Data tends to be shared or exchanged regardless of what level integration is occurring Often, data can not be shared or transferred without understanding the process or application

IRMAC Presentation – 21/NOV/ OS 390 Solaris Oracle SAP C++ Windows 2000 Windows NT Application View As Complex As Data View Natural Apps SQL Server VB Apps Pivotal MQ Series CICS SQL Server Java Apps IIS HTML EDI Trns The Enterprise EDI HL7 XML SOAP ….. RPC MOM FTP CORBA …... COM HTTP Clients Partners Suppliers Clients DB2 VSAM Cobol Apps Adabas

Warehouse Mgmt. Logistics Order Entry Financials Bank Parcel Service Call Center E-Commerce Portal Customer The Enterprise 1) “I want to buy it.” 2) “I don’t have it yet!” 3) “When are you going to get it delivered?” 4) “Please pay for it.” B2B B2C A2A Electronic Business = Zero-Latency Business {Gartner}

IRMAC Presentation – 21/NOV/ Zero or Low Data Latency Just the old concept of timeliness with an additional twist Refers to the real time (zero) or near-real time (low) transfer of information between applications Zero data latency tends to imply transactional solutions, while low data latency or near-real time allows for assured delivery (messaging) options Low data latency within the enterprise is no longer enough; it must be enabled across the enterprise perimeter

IRMAC Presentation – 21/NOV/ An Affirmation; Modeling the Problem Adding EAI oriented solutions to the problem of data availability does not reduce the modeling effort required – it intensifies it Each application view of the data must be defined and understood Additionally the dependencies involved in sharing and transferring information must be supported I find that use case modeling, at the level of the inter- application interfaces, is valuable for scoping the problem, and understanding it at a high level

IRMAC Presentation – 21/NOV/ Modes of Integration? A Functional View! Data Consistency Multistep Process Composite Application Batch or immediate transfers Multiple processes Multiple steps One-way, asynchronous interactions (loosely coupled) Systems are physically and logically independent Manual or straight-through flows Batch or immediate, individual transfers One business process Multiple steps One-way, asynchronous interactions (loosely coupled) Systems are physically and logically independent  Potential long transactions Immediate interaction One business process One step Two-way, synchronous interactions (Tightly Coupled) Systems are physically and logically dependent Usually a client server (request reply) interaction

IRMAC Presentation – 21/NOV/ Front End versus Back End Integration Front End Integration  Making Enterprise Resources available by means of tailored interfaces  Managing interaction between the enterprise and the user community  Providing gateways for information to and from enterprise systems Back End Integration  Integration of various types among different enterprises resources (especially applications)  Back end applications collaborate to complete or realize a business process

IRMAC Presentation – 21/NOV/ Enterprise Integration (Internal & External) Specialized Application Interfaces Backend Applications User Interfaces B2C A2A Specialized Application Interfaces Specialized Application Interfaces Automated Interfaces B2B WEB Services Trend To XML Trend To XML

IRMAC Presentation – 21/NOV/ Front End Integration Browser Wireless Device Presentation & Communications Portal Business Logic Portal DB Order Entry Shipping Order Fulfillment Inventory “Typical” N-Tier Architecture Connected To Multiple Back Ends With Different Middleware

IRMAC Presentation – 21/NOV/ Critical Problems For Front End Integration All the pieces of the n-tier application have to talk to one another The various components may have different views of the information associated with the transaction Must manage security (authentication and authorization); and e access to all of the back end resources involved Typically, units of work must be supported over multiple resources (2 phase commit) The solution as a whole must perform and scale …….

IRMAC Presentation – 21/NOV/ Integration Engine Back End Integration SAP PeopleSoft Order Entry WMS Customer Service Invoicing

IRMAC Presentation – 21/NOV/ Characteristics Of Back End Integration Connectivity more of an issue at the back end Complexity of interaction may be greater Security, while always critical, is more easily managed Timing is not as much of an issue; near real time is often good enough Consequently, suitable for multi-step processes Transactions (units of work) are often reduced in scope …...

IRMAC Presentation – 21/NOV/ Integration: The Enterprise View Web Based Services

IRMAC Presentation – 21/NOV/ Integration Technologies & Techniques Conventional Techniques  File Transfers & Distributed File Systems, RPC, Teleprocessing Monitors, Database Replication, RJE, Screen Scraping, ….. Contemporary Techniques  Message Oriented Middleware  Web Enablement (Browser Based)  Distributed Objects  Integration Brokers  Document Centric Integration (XML)

IRMAC Presentation – 21/NOV/ Message Oriented Middleware (MOM) Products that pass messages from a source to a destination Fundamentally asynchronous in nature Based on queues, explicitly or implicitly; requiring programming to load and unload msgs at either end Rely on assured delivery, rather than transactions Principally a point-to-point solution ….. Not a lot of value added features originally ….. MQ Series, MSMQ, EntireX Broker, Sonic MQ …..

IRMAC Presentation – 21/NOV/ MOM Example: MQ Series This diagram from IBM shows MQ Series communicating between two platforms. MQ Series itself manages the flow of msgs between the local queue and the remote queues; and does low level marshalling. The OS on the sending and receiving platforms may vary.

IRMAC Presentation – 21/NOV/ Distributed Objects (really Components) COM/DCOM, CORBA, and Java RMI/IIOP Enables processes to invoke methods on external components (perhaps on remote platforms) Supports Component Based Development (CBD); with access to individual components only via interfaces Method level, point to point, synchronous integration par excellence Unmatched flexibility, but complex to implement Supports strongly typed interfaces

IRMAC Presentation – 21/NOV/ RMI Stub A Trivial Distributed Object Implementation RMI Skeleton Customer Remote Reference Layer RMI Transport Layer Internet Protocol (IP) RMI Server RMI Interface Customer Account Manager RMI Client Sybase JDBC Distributed Objects Customer object calls the Account Manger with an Account ID. Manager retrieves the balance, and returns it to the Customer Object.

IRMAC Presentation – 21/NOV/ Integration Brokers Use the Broker architectural pattern to implement and manage flows of msgs between apps and resources Uses a MOM (often a JMS) Messaging Backbone Have sophisticated capabilities for:  Connecting to applications (connectors or intelligent adapters)  Transforming and augmenting messages (applies business rules)  Content Based Routing Have an enterprise scale orientation & focus  Connects multiple apps in multi-step integration flows Offer assured delivery; are not transactional oriented

IRMAC Presentation – 21/NOV/ Broker Architectural Pattern Integration Broker ApplicationAPI ApplicationScreenInterface DistributedComponent ApplicationInterfaceObject Application DirectoryService Database MiddlewareService

IRMAC Presentation – 21/NOV/ Integration Brokers - Continued Provide a number of value added features centered on scalability, reliability, connectivity and graphical implementation tools Semantically operate very similarly to workflow engines or Business Process Management products Goal is to engineer a connection between multiple enterprise resources, performing whatever processing necessary to ensure that the connection adheres to business rules In other words, they implement what is in effect an integrated application, with a minimum of effort Especially suited for back end integration Low data latency solution

IRMAC Presentation – 21/NOV/ Simple Integration Broker Implementation

IRMAC Presentation – 21/NOV/ Document Centric Integration XML Focus - SOAP/Biztalk; X-Bridge; Rosetta Net; …. Integration through the transfer of XML “documents” between Enterprises, platforms, applications, …. HTTP or HTTP/S is often used to transmit the documents over TCP/IP connections Additional protocols or processing standards often used to manage the communication  Defined XML dialects, like BiZtalk, Rosetta Net, ebXML, ….. Eliminates much of the technical complexity in communicating between environments and domains

IRMAC Presentation – 21/NOV/ Orchestrator/XML Architecture Consumer (e.g. Browser) Service HTTP XML Consumer (e.g. Browser) XML World Wide Web Consumer (e.g. Business Partner) Rules Content Based Routing Style Sheets XML Gateway Log Transform Replicate/ Aggregate Log

IRMAC Presentation – 21/NOV/ The Orchestrator/XML Solutions Enables customers to dynamically compose new services in order to react to new business requirements Provides a document driven reaction framework Enables applications to be integrated in an XML way Intelligent, flexible & scalable XML Routing Hub  Obtains an XML document  May transform it (based on style sheet)  Route document based on content (Content Based Routing) Leverages XML-based and distributed-object technologies to provide enterprises with highly configurable and adaptable solution for integrating XML documents

IRMAC Presentation – 21/NOV/ Document Centric Integration: Order Entry Web & Application Server Orchestrator /XML Java Server Order Entry Customer B Customer A Customer C Can configure this as a Web Service Java Server Order Tracking Customers Submit XML Orders via HTTPS Server Redirects Orders To /XML /XML Analyses Document, Transforms It, and Routes It to Services Server Converts & Edits Data, and Forwards It to Order Fulfillment Server Submits Tracking Request & Returns Reply Tamino Audit DB

IRMAC Presentation – 21/NOV/ A Multi Step Integration Process Price Policy Verify Credit (Java) Issue Policy Invoice Policy (COM) >> Workflow Engine Check Balance (Natural) Update Account (Natural) Credit Bureau Web Service Adabas EntireX RPC SOAP HTTP/S

IRMAC Presentation – 21/NOV/ Integration & Web Enablement

IRMAC Presentation – 21/NOV/ N-Tier Application For Product Distribution

IRMAC Presentation – 21/NOV/ Conclusions Multiple EAI Techniques and Technolgies Available In order to properly use them, modeling and planning is necessary Properly used, they can substantially enhance data availability within the enterprise Thank you!