Enterprise Application Integration An architecture for integration

Structure of enterprise applications Enterprise Applications keep track of (data) information related to the operations of the enterprise e.g. Inventory, sales ledger And execute the core processes that create and manipulate this information Process from ordering of products to cash received Enterprise may also have supply chain management systems or customer relations managements systems which also requires keeping track of information from outside the organisation Data in the organisation is captured in a number of data models: relational data model; star scheme….

An example of a business process: Purchase-to-Pay 1. Determine requirements, complete purchase requisition. Automatically generate the purchase requisition based on quantity on-hand, quantity-on-order, and expected demand. © L. Gray, CNU

Purchase-to-Pay 2. Prepare and record purchase order. Assists the buyer in identifying sources of supply for the requested item, preparing Request for quotation to vendors, analyzing vendor quotes, comparing vendor prices, terms, and past performance

Purchase-to-Pay 3. Receive and record goods. Compare quantity ordered to quantity received. Routes goods to the function that requested them or directs them the warehouse for immediate sale. It also records vendor performance data.

Purchase-to-Pay 4. Receive vendor invoice, match with PO and receiving report; record payable. If the three-way match fails, the enterprise system notifies the proper personnel to ensure timely reconciliation of differences.

Purchase-to-Pay 5. Prepare and record cash disbursement and update accounts. Uses vendor and AP data to schedule payments in accordance with vendor terms and to receive discounts.

Enterprise process models Each business process model “must” capture the series of steps and interactions required to complete a specific business process: The execution of the process will impact on the enterprise data across one or more applications A process may be executed within a single application or more often through collaboration between several applications In the semi-automated (Islands of computing) system: Originally, the process and data models were implemented in a wide variety of functional system (related to organisational departments) Each system consisting of its own database, application logic and user interface

IT reality – Islands of Computing Purchased Application Package(s) – etc. Program Program Legacy Application System(s) Program New Web-based application(s) Program Program Program Program Program Program Enterprise IT reflects the structure and history of each enterprise not the business processes required. Departments have their own IT systems Legacy systems are left in place and new systems built separately.

Two architectural solutions Integration Leave existing applications in place Minimises risk of disruption and change in data models/process models Link existing applications, integration of systems, which map between existing data and process models and add ‘missing’ process models where required Effective when most of the functionality is already implemented in systems and the integration is primarily creating linkages and processes between these applications When significant additional functionality is required beyond what is already in the applications,: may need to replace existing systems or redesign the data (inconsistencies) and business process models

Tactical approach to integrating the Business information systems Application System B Trans action File Program Extract Program Program Down Down load load Program Extract File File Program Load Program Load Screen Message Program Application Scrape Queue Application Database Replicator System A System C Extract Load Program Program Down Program load File Program Program Program Trans Program action File Program Message Queue Each requirement is addressed with a point to point solution Typically consisting of a data transfer mechanism and a data format converter.

Strengths and weaknesses of “islands of Computing” tactical approach Focused on getting the job done Each individual solution can be created quickly Technical Weaknesses Numerous Point-to-Point Interfaces Sharing of data between systems Data duplication Data inconsistency Inconsistent Qualities of Service Limited or lack of integrated information Inconsistent Business Processes solutions as they are not integrated

Strengths and weaknesses of “islands of Computing” tactical approach Business Weaknesses Cost of maintenance Isolated decisions lead to overall inefficiencies Increased expenses Requirement to retain knowledge of the solution in-house. Potential security or loss of service/process failure due to inconsistencies. Therefore, there is a general requirement for integration to support organisational system capabilities For instance Customer Relationship Management – tracking all information about a customer across the enterprise

Solution 2: Enterprise “information systems” integration EAI can help resolve the following issues: Unrestricted sharing of data and business processes (information) across an organization and can be extend to customers, suppliers and regulators The linking of data, business processes and applications to automate business processes While ensuring that there are consistent qualities of service (security, reliability etc) Reduce the on-going cost of maintenance and reduce the cost of rolling out new systems.

Enterprise integration requirement Unrestricted sharing of data and business processes throughout the networked applications or data sources in an organization The Linking of Data, Business Processes, Applications to automate business processes While ensuring that there are consistent qualities of service (security, reliability etc) Reduce the on-going cost of maintenance and reduce the cost of rolling out new systems. Solution: Devised ways to efficiently reuse legacy and existing apps and add new ones

Layers of an EAI Stack Adapter Adapter Provides real-time and historical data on performance of processes and assists in making decisions. EAI Business Activity Monitoring Manages and tracks business transactions that might span multiple systems and last minutes to days. Business Process Orchestration Data Transformation Ensures the data is the correct format for delivery to The next system. Message Storage & Routing Adapter Ensures the reliability of data delivery between systems. Adapter Provides “open” connectivity between data sources: programs that interact with the underlying business application

Adapters Also known as “Connectors”, “Translators”, “Wrappers” or “Bridges” Provide seamless connectivity to the underlying application or data Convert information and events into data (a form) that can be utilized by the “enterprise’s” I.T. infrastructure Business Activity Monitoring Business Process Orchestration Data Transformation Message Storage & Routing Adapter Adapter

Message storage and routing EAI Reliable messaging Connect applications running on different systems by sending and receiving application data as messages Message storage Central repository for temporary storage of transactions until they can be delivered Also called “Message Warehousing” Message queuing and routing Asynchronous communications Set of tools that route messages between sources and destination (consumers) based on pre-defined “business” rules Business Activity Monitoring Business Process Orchestration Data Transformation Message Storage & Routing Adapter Adapter

Data Transformation Data Transformation Transformation to and from formats used by Different Systems Handling simple or complex data structures GUI Configuration tools to help define the transformations EAI Business Activity Monitoring Business Process Orchestration Data Transformation Message Storage & Routing Adapter Adapter

Intermediate (canonical) common format Data Transformation Source Target F. name: denis L. name: manley Affiliation: DIT, Dublin, Ireland Name: Denis Manley Company: DIT City: Dublin Country: Ireland Intermediate (canonical) common format EAI typically converts the source data into an intermediate format and then convert its into the target format .

EAI: Building a data model Create a common data model including a set of terms for the concepts in the domain of the data sources being integrated e.g., Employee, Customer, Patient, weight, height, body Temperature, … Mappings relate data items in data sources to terms in data model Background knowledge about terms essential for transformations e.g., Employee subClass Of Person, 2 people with the same last name, first name and street address are likely to be the same,

Issues in data transformation Challenge 1: Coping with software from different vendors Oracle vs. SQL Server Challenge 2: Coping with different “data” formats Relational vs. XML Challenge 3: Coping with different data models (schemas) Solvable for the smaller scale cases one person understands all the schemas As the complexity and scale of the data models increases it becomes harder and harder Challenge 4: Coping with data from new sources such as new systems, suppliers, customers

Business Process Orchestration Process Modeling Describing the flow of information in the context of business processes Using the input/output of processes as the Integration points between diverse business processes Process Brokering Execution of discrete steps within a business process Ability to recover from failed steps Process Management Monitor business processes Correlate metrics to specific business process steps EAI Business Activity Monitoring Business Process Orchestration Data Transformation Message Storage & Routing Adapter Adapter

Business Activity Monitoring Business Activity Monitoring: Using your data assets (sales figures) to make better decisions BAM is used to gather information about what is occurring in the EAI deployment Status of current processes Identification of problem areas E.g. Spotting applications that are not responding Monitoring of unusual activity E.g. Unusually large orders The collected information is displayed for operators or managers to diagnose and determine the solution EAI Business Activity Monitoring Business Process Orchestration Data Transformation Message Storage & Routing Adapter Adapter 24 24

Potential EAI benefits Lower development costs Integration is simpler because systems are more loosely coupled. Lower opportunity costs Integration is done more quickly corresponding cost savings reachieved sooner Lower maintenance effort adapters extract the interaction with external systems significant advantage from the software engineering point of view

Potential EAI Challenges Architecture concentrates all of the processing into a single server/cluster. Often became hard to maintain and evolve efficiently Hard to extend to integrate 3rd parties on other technology platforms The data transformation model introduces an intermediary step Added complexity and additional processing effort EAI products typified by Heavy customisation required to implement the solution Lock-In: Often built using proprietary technology (specific to a vendor) and required specialist skills Lack of flexibility: Hard to extend or to integrate with other EAI products! Requires organisations to be ready or prepared to adapt EAI: EAI readiness

What is EAI readiness? Much of the challenge associated with adopting EAI is not related to the technology but to how prepared the organisation is to implementing it. EAI readiness is a state of business maturity in which the enterprise’s: Business processes, employees, Technology (systems) structure, and culture, are prepared for the successful introduction of both EAI technology and the business disciplines that drive its introduction. All five of these factors must be properly situated and aligned before an EAI implementation can succeed.

Business process readiness Business strategic goals for project (i.e. process efficiency, customer satisfaction, supply-chain optimization) clearly defined and agreed upon by company executive leadership. Desired business processes designed, documented, and valuated. Gap between current and desired business processes well understood.

Employee readiness Staff accustomed to using information technology in their day-to-day work. Staff accustomed to adaptation of new roles, responsibilities, processes, and knowledge … ; i.e. to business change. Staff have broad understanding of the business and their contribution to it. Executives understand the nature of investing time, staff and budgets into the business infrastructure in order to reap future returns. (financial v strategic)

Technology readiness No systems are to be integrated using EAI tools that will be retired/ decommissioning soon. There is no point integrating with a system which will soon be removed Applications portfolio has been evaluated for semantic compatibility. Check it is possible to transform from 1 data model to the other Implementation of EAI is not defined as the purpose of the effort. The project should be about the business objective, not about the technology

“organisational” Structural readiness Keep it simple by ensuring that the EAI system involves as few departments/ organisations as possible. Each organisation involved adds to the complexity of the project as more people have to be engaged with and there are potentially greater variety of technologies and architectures. Stakeholder analysis performed to determine overall potential for business resistance to the planned change. EAI projects change the way that each department/organisation works. Therefore, EAI projects need the cooperation of each department/organisations. Business “steering” committee is established and ready to provide leadership to the business change.

Cultural readiness Corporate culture embraces the importance of well-defined business processes. Staff accustomed to adaptation of new roles, responsibilities, processes, and knowledge …. Individuals and departments willing to change the way the applications and data is defined.

Questions Many technologies involved in business difference business process can not effectivley communicate and share data. Enterpirse application integration is one approach to ensure limiting this problem: Describe the 4 main steps in the EAI framework (8 marks) Explain, using a suitable example, why EAI is preferred to the to the “islands of computing” approach to developing enterprise I.T. systems. (12 marks) Discuss why an organisation must be prepared in order to correctly implement the EAI approach. (10 marks)