black marble the strategic IT asset for your organisation PRAGMATIC BUSINESS VALUE OF USING SERVICE ORIENTATED ARCHITECTURE AND IMPLEMENTING AN ENTERPRISE SERVICE BUS

black marble FBCS, CEng Senior Architect Black Marble LTD Robert Hogg

black marble Agenda What is SOA? Connecting Systems Demo Governance Getting the Return

black marble

Key Business System Drivers Businesses need: Flexibility Agile Systems Rapid Development Cycle Extensibility Collaboration – local/national/global Rigidity Reliability Concrete Guarantees and Risk Management Compliance with governance

black marble Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business. SOA Terminology Two Definitions of SOA Service Oriented Architecture: a software design approach in which key functions are built as reusable components which implement industry standards for interoperable communications. Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

black marble State of SOA PerceptionsPerceptions –SOA = IT centralization/standardization –“SOA is dead” (Anne Thomas Manes, Burton Group; January 2009) –Cloud computing will replace SOA RealityReality –SOA does not solve all enterprise IT issues –SOA is complex, and difficult –Core value of SOA is agility, but needs vary –SOA demands change, but organizational resistance is often the biggest impediment –Need to focus on architecture, not technology

black marble SOA Manifesto Service orientation is a paradigm that frames what you do. Service-oriented architecture (SOA) is a type of architecture that results from applying service orientation. Business value over technical strategy Strategic goals over project-specific benefits Intrinsic interoperability over custom integration Shared services over specific-purpose implementations Flexibility over optimization Evolutionary refinement over pursuit of initial perfection.

black marble The Benefits of SOA Align Business and IT Increase Agility Reduce Costs Source: Dr Arvindra Sehmi

black marble Common Myths about SOA MythsFacts 1. SOA is a technology1. SOA is a design philosophy independent of any product technology or industry trend. 2. SOAs require web services2. SOAs may be realized via web services but using web services will not necessarily result in a SOA. 3. SOA is new and revolutionary.3. EDI, CORBA, and DCOM were conceptual examples of SOA. 4. SOA ensures the alignment of IT and business 4. SOA is not a methodology 5. A SOA Reference Architecture reduces implementation risk 5. SOAs are like snowflakes, not two are the same. 6. SOA requires a complete technology and business process overhaul 6. SOA should be incremental and built on your current investments. 7. We need to build a SOA7. SOA is a means, not an end.

black marble Common Myths about SOA #2 MythsFacts SOA begins at the firewall SOA Aligns the Business With IT SOA Governance Fixes Everything SOA begins at the firewall People Align Business With IT Governance needs to span all of IT, not just SOA

black marble Service Oriented Development Tools Service Oriented Development Tools Service Composition Service Composition What does it look like? Business Process Functional Needs Service Oriented Management Service Oriented Management Service Registry Service Health Monitoring Service Management Exception Management SLA Management Service Consumers Web 2.0 Composite Applications Forms Mobile User Experience Managed Service Connectivity Change & Control Mediation Connection Business Process Team Workflow Analytics & Reporting Process Optimization Business Intelligence Business Processes Rules Management Service Enablement and Messaging WS* Standards Legacy Systems Adaptation Contracts Federated Identity Federated Data Transform Tools Composition Modeling Composition Testing Service Enablement and Messaging Service Enablement and Messaging Process Modeling Service Modeling Service Consumers Service Consumers Service Testing Portals UX Tools Service Virtualization

black marble the strategic IT asset for your organisation CONNECTING SYSTEMS

black marble Connecting Application is Not Enough Point-to-Point Integration Inhibits Agility Point-to-Point –Complex Interface –Redundant Logic –Doesn’t Scale –Lacks Visibility Business Impact –Delays Response to changing business needs JD Edwards Service JD Edwards Service SAP Service SAP Service Java Appl. Service Java Appl. Service.Net Appl. Service.Net Appl. Service CICS Service AS/400 Service AS/400 Service MS CRM Service Oracle Service Oracle Service

black marble “End Point/Service Oriented Spaghetti” 15 Service Providers 22 Service Consumers Guess how many Service Providers and Consumers are in this Spaghetti? Even a few dozens of services pose quite a plumbing challenge!

black marble Introduction to ESB “A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components.” - Gartner Group

black marble Enterprise Service Bus (ESB) CRM BizTalk was positioned as a Hub and spoke Static Receive Port Static Send Port Hard Coded Map Name Static Schema Static Receive Port Static Send Port Hard Coded Map Name Static Schema Static Receive Port Static Send Port Hard Coded Map Name Static Schema Most decisions are made and locked in at Design time or at Deployment not at Run time! Any change is a re-development or a system re-configuration! We’ve allowed the easy to use tools to define our architecture Now we’re saying it can be an Enterprise Service Bus?

black marble We need to agree on what an Enterprise Service Bus is and what it does. Message Broker Message Transformation Message Validation Message-Oriented Middleware Service Orchestration Adaption

black marble ESB Core Configurable, reusable architecture Addresses common concerns generically - Routing, Transformation, Exception Handling, Monitoring etc. Enforces Consistency and Unification Leverages modern Standards (WS-*), still supports Legacy Central point for Service Monitoring and Administration Increases agility (through added Abstraction, Loose coupling) Faster implementation times (through Configuration vs. Coding)

black marble Agility thru Service Composition Service Consumer Service Provider Invocation & Orchestration Transport Protocol Conversion Data Format Transformation Location & Version Transparency Message Interactions Support Enterprise Service Bus Error Handling & Repair

black marble ESB Design Patterns The ESB Toolkit promotes pattern-based development Message Routing Patterns –Message Router –Content-Based Router –Routing Slip –Scatter-Gather –Recipient List –Splitter Message Transformation Patterns –Message Translator –Normalizer –Content Enricher Service Mediation Patterns –VETO/VETRO –Request-Response Service Management Patterns –Repair and Resubmit

black marble Relevant Pattern Implementation ESB Guidance Policy Centralization Rules Centralization Rules Centralization Event-Driven Messaging Data Format Transformation Protocol Bridge Protocol Bridge Message Router Message Router Message Broker Message Broker Legacy Wrapper Architectural Design Patterns Message Routing Message Routing Scatter-Gather Routing Slip Routing Slip Recipient List Recipient List Message Transformation Content Enricher Content Enricher Data Model Transformation ESB Usage Patterns Repair and Resubmit VETO/VETRO GatewayGateway ?? Reply Forward Reply Forward ?? Metadata Centralization Perimeter Service Router Content-Based Router Content-Based Router

black marble ESB Toolkit - New Abstraction Layer on Top of BizTalk Declarative, Meta-data, Policy and Configuration –Driven. Transform Service RoutingRouting Process Orchestration ProtocolAdaptationProtocolAdaptation End Point Resolution Pub/Sub Service Service Consumers Service Providers 1.Transform my message 2.Determine which endpoint I need 3.Route my message 4.Route the response to a second service 5.Return the final result to me 1.Transform my message 2.Determine which endpoint I need 3.Route my message 4.Route the response to a second service 5.Return the final result to me On Ramp Off Ramp Transform my message Resolve a service end point address for me

black marble Web Services BizTalk On-ramps Off-ramps Itinerary Services Resolvers Adapters Exception Microsoft ESB Guidance Architecture Management Portal

black marble ESB Toolkit Core BizTalk ESB Toolkit Components Adapters Dynamic Ports Dynamic Ports Pub Sub Engine Pub Sub Engine Transformation Engine Transformation Engine Business Rules Engine Business Rules Engine Host Environment Host Environment Itinerary Services Resolvers BizTalk Components On/Off-Ramps Management Portal Management Portal Orchestration Engine Orchestration Engine Adapter Providers Core Web Services Core Web Services UDDI 3.0 UDDI 3.0 Exception Management BAM ESBToolkit

black marble Core Web Services Resolver Web Service Transformation Web Service UDDI Web Service Exception Web Service Operations Web Service BizTalk Send Ports Off-Ramps BizTalk Receive Ports On-Ramps BizTalk ESB Toolkit Architecture Exception Management Store Exception Management Store ESB Management Portal Provisioning Framework Reports Alerts ESB Toolkit Core Itinerary Services Exception Management Framework Exception Logger Exception Handler Fault Processor Resolver-Adapter Provider Framework Resolvers (…) Adapter Providers(…) UDDI 3.0 BAM Generic SOAP Send Generic WCF Send Generic JMS Send Custom Send Pipeline Custom Pipeline Itinerary Store Itinerary Store Generic SOAP Receive Generic WCF Receive Generic JMS Receive Custom Receive Pipeline Custom Pipeline Route Service Custom Service Transform Service Transformation Engine Business Rules Engine Orchestration Engine BizTalk Pub/Sub Engine

black marble Itinerary Services: Message Processor Resolvers: Context Finder Adapter Providers: Adapter Properties The Core ESB Toolkit Components Itinerary Mediation Policy

black marble Itinerary Concept This is the heart of ESBG –Allows us to implement the ESB pattern as opposed to a Hub/Spoke pattern It provides the runtime flexibility that BizTalk doesn’t have by default

black marble Itinerary Mechanism Itinerary Concept Logical overview Orchestration Based ESB Services Route Transf orm Messaging Base ESB Services Route Transfor m Itinerary Cache MessageBox Itinerary Processing Generic On- Ramps Generic On- Ramps Itinerary Message body

black marble Three Ways to Assign Itineraries Itinerary is resolved on the server’s ESB On-Ramp receiving port via configurable resolver Service Proxy Service Proxy Client resolves itinerary via resolver service and then sends it in a WCF/SOAP Header request Adaptive Service Client Client sends an itinerary in a WCF/SOAP Header request Advanced Service Client

black marble BizTalk Dynamic Send Port Itinerary Based Routing & Processing Light-weight service sequencing & composition –Invokes itinerary (internal) and external services –Dynamic service context resolution at runtime Maps internal service invocation to BizTalk service containers –pipelines in ports and orchestrations. Pub/Sub Engine BizTalk Receive Port Routing Service On Ramp Receive Pipeline Resolver Adapter Provider Off Ramp Send Pipeline The “heart” of the ESB Toolkit BizTalk Orchestration ItineraryService1 Resolver1 Pub/Sub Engine

black marble Itineraries Deployment Design Time XML File Itinerary Repository Test/Run Time ItineraryItinerary OnRampOnRamp Itinerary Selector ESBProcessingESBProcessing Itinerary & Message MessageMessage Itinerary Repository BizTalk Deployment Tools

black marble Resolvers For runtime flexibility services are nothardcoded to specific endpoints or maps Service end-point metadata can bedetermined at runtime Resolvers are called by services to setan artifact (map, itinerary) or endpointmetadata

black marble ESB Toolkit Resolver Can resolve: –Message delivery endpoints –Maps for transformations –Endpoint configuration –Custom service metadata –Itinerary to be associated with a message Endpoint/Map resolution with: UDDI 2.0/3.0 XPATH Static Business Rules Engine (BRE) Itinerary resolution with: Itinerary (BRE) Itinerary Static

black marble Resolver Mechanism ESB Services A resolver configuration string contained within the itinerary is provided to the resolver manager Resolver Mechanism StaticStaticUDDIUDDI SQLSQLXPathXPath BREBRE WSMEXWSMEX Resolver Manager The resolver mechanism is able to load different resolvers based on where the runtime data is stored Rules Engine Service Registry DatabaseDatabase

black marble ESB Resolvers Endpoint Resolution UDDI 2.0 Artifact Resolution ESB Resolvers Static BRE Itinerary - Static XPATH Itinerary - BRE UDDI 3.0 ESB Toolkit V2 ESB Guidance V1

black marble ESB Adapter Provider The following Adapter Providers supplied: –WCF-BasicHTTP –WCF-WsHTTP –WCF-Custom WCF-SQL –FTP –WebSphere MQ –File –SMTP Custom –any BizTalk adapter (that support dynamic ports) can be exposed as an adapter provider

black marble ESB Adapter Providers FTP ESB Adapter Providers WebSphere MQ File Custom WCF-BasicHTTP WCF-WsHTTP WCF-Custom ESB Toolkit V2 ESB Guidance V1 SMTP WCF-SQL

black marble ESB Exception Management Portal

black marble BizTalk ESB Toolkit Extensibility Extensibility PointDescription Adapter ProviderCreate new Adapter Provider component, manifest file Orchestration ServiceImplement orchestration, update configuration Messaging ServiceImplement service and designer extender ResolverImplement resolver component Composite ResolverDefine Unity container, fact providers, fact translators Cache ManagerSupport Enterprise Library 4.1 API Itinerary ExporterImplement custom exporter component

black marble DEMO

black marble GOVERNANCE

black marble SOA Governance Architecture Publish/Subscribe Infrastructure ESB Core Services Virtualize d Service Off-ramp IIS Web Service On-ramp Management Point Governance Portal Governance Server Security Enforcement SLA Enforcement Service Registry Service Metrics Collection Service Configuration Metrics Reporting Service Lifecycle Management Physical Service

black marble ESB Governance Integration Integrated with governance solutions from AmberPoint and SOA Software Specific functionality varies by governance provider, but typically include: –Service metrics –Policy-driven security –Policy-driven service-level agreement –Alerting

black marble Defining Solutions As A Set Of Capabilities Mapping Service RoutingRouting Process Orchestration ProtocolAdaptationProtocolAdaptation End Point Resolution Pub/Sub Service WSDL Schema Service Consumer 1.Transform my message 2.Determine which endpoint I need 3.Route my message 4.Route the response to a second service 5.Return the final result to me 1.Transform my message 2.Determine which endpoint I need 3.Route my message 4.Route the response to a second service 5.Return the final result to me

black marble the strategic IT asset for your organisation GETTING THE RETURN

black marble Business Value Broaden availability of Information and Capability Exception based processing Shorten business process cycles Regulatory compliance Alignment of systems with the business

black marble Limitations Not all Capabilities are modular Issues over Transfer of Control and Ownership Modelled process do not implement all activities, e.g. Human intervention Difference between managing documents and business processes

black marble Road to failure about for Implementing SOA Typically a one or more of: –Lack of Comprehension of Business needs. –IT first Design. –Poor Management. –Poor implementation. –No Need for SOA

black marble Key Lessons Critical to have a single shared vision by business and IT Enterprise logging and monitoring critical for finding and resolving problems Apply Solid change management –Test Change Management before initial rollout Governance, Governance, Governance Consider an Agile rollout but do it well. After Rollout keep talking to the business and keep the value of the investment visible by showing agility.

black marble Loosely coupled messaging environment Policy driven message level routing and mediation Itinerary based processing Enterprise Service Bus - Summary Higher levels of reuse Lower operational costs Dynamic business changes Business and exception analytics Business requirements in the drivers seat What it gives you… Why you want it…

black marble Where Next? Online content – – –

black marble How to Contact Us

black marble the strategic IT asset for your organisation APPENDIX

black marble Endpoint A UDDI Repository Version Policy Schema Metadata Endpoint Metadata

black marble Endpoint A UDDI Repository Endpoint BEndpoint C Intelligent Routing

black marble Endpoint A UDDI Repository Endpoint BEndpoint C Transformation

black marble Endpoint A UDDI Repository Endpoint BEndpoint CEndpoint D Orchestration

black marble Vertical Market Application Database Mainframe Application J2EE Application.NET Application Web Service Endpoint Adaption

black marble Service Consumer Asynchronous Service Synchronous Service Message Exchange Patterns Publishing/Subscribe Service