Bottom-Up Methodology using Integration Patterns Matt Rothera Director, Customer Centric Engineering SOA-11: SOA Design Best Practices

© 2006 Progress Software Corporation2 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Agenda  The “Organic Approach”  Qualification Criteria for the First ESB Project  Patterns and Integration Scenarios  Scoping a SOA Project “Bottom-Up”

© 2006 Progress Software Corporation3 SOA-11: SOA Design Best Practices: Bottom-Up Methodology The Path to SOA Your ESB: Reality meets Nirvana Do both: Top-down and Bottom-up

© 2006 Progress Software Corporation4 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Parallel Tracks Business Process Analysis Canonical Identification Project “2” Project “1” Service Identification Adapt and Evolve Enterprise SOA Adapt and Evolve Projects Feedback Iteration Evolution over time is the key

© 2006 Progress Software Corporation5 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Agenda  The “Organic Approach”  Qualification Criteria for the First ESB Project  Patterns and Integration Scenarios  Scoping a SOA Project “Bottom-Up”

© 2006 Progress Software Corporation6 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Goals for your first SOA projects  Business Qualification Prove incremental nature of SOA with immediate ROI  Technical Qualification Expose the ESB’s unique characteristics to solve tough integration problems  Service and Event Reuse Potential Simplify follow-on projects by reusing services and infrastructure Produce reusable events, data and services

© 2006 Progress Software Corporation7 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Selection of Project Category Forward Facing Systems (Front-end to Back-end Integration) Extended Enterprise Partner Networks (B2B) Flow-Through Process (EAI, STP, ERP, OSS, etc.) Highly Distributed Networks (e.g. branch offices) Pick projects from these categories, then correlate

© 2006 Progress Software Corporation8 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Qualification Criteria for your first SOA Project  Agility  Scale  Distributed or Federated  High Availability  Through the Firewall The need to integrate drives your first SOA Project

© 2006 Progress Software Corporation9 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Categories of Business Impacts  Automating a Process  Improving Productivity for certain stakeholders in the organization  Improving Quality or Consistency by reducing the chance for human errors  Visibility to Information, allowing better decisions  Reduced Maintenance Costs  Ability to staff new integration projects more quickly  Ability to construct new integrations in significantly less time  Ability to respond to changes in business requirements more quickly Line of BusinessIT Related

© 2006 Progress Software Corporation10 SOA-11: SOA Design Best Practices: Bottom-Up Methodology New integration or replacement?  Automating a Process  Improving Productivity  Improving Quality or Consistency by reducing the chance for human errors  Visibility to Information, allowing better decisions  Reduce Latency, Making processes more efficient or reducing windows of error  Improve reliability, reducing exception conditions that can cost the company money  Provide Real-Time access to information instead of Batch Your questions change depending on which NEW Replacement

© 2006 Progress Software Corporation11 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Agenda  The “Organic Approach”  Qualification Criteria for the First ESB Project  Patterns and Integration Scenarios  Scoping a SOA Project “Bottom-Up”

© 2006 Progress Software Corporation12 SOA-11: SOA Design Best Practices: Bottom-Up Methodology What’s an Integration Pattern?  A proven method of capturing an “expert’s” knowledge of an integration approach  A response to a specific, recurring problem that occurs in the integration space  Fairly standard and well defined term in the industry – we will adopt the same approach and use for the term  See “Enterprise Integration Patterns.com”  Examples: Splitter Aggregator Resequencer Claim Check

© 2006 Progress Software Corporation13 SOA-11: SOA Design Best Practices: Bottom-Up Methodology What’s an Integration Scenario?  What is an Integration Scenario? Consists of at least one Source System and one or more Target Systems Represents the flow of data in typically one direction Description of the connection methods  What is Mediation? Integration logic that is required between the source and target to smooth out the differences between the data –Protocol –Recovery –Format –Destination –Sequence  Discuss a scenario in terms of: An On-Ramp Mediation Logic One or More Off-Ramps

© 2006 Progress Software Corporation14 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Sample Integration Patterns Routing Pipes and Filters/Routing Slip Content Based Router Splitter Transformation Canonical Data Model Envelope Wrapper Content Enricher Interaction Models Produce Fire and Forget Request-Reply Async Request-Reply Bulk Read Operate/Aggregate/Correlate Cache Claim Check Process Manager System Interaction Adapter Emitter Bridges Messaging Application Server (SSB or Servlet) Interaction Models Consume Event Driven Consumer Selective Consumer Polling Consumer Replier Bulk Load System Interaction Adapter Requestor/Sender Bridges Messaging Application Server (MDB) Resequencer Gather

© 2006 Progress Software Corporation15 SOA-11: SOA Design Best Practices: Bottom-Up Methodology 1 Multiple on/off ramps to the bus Screen Scraping Hardware – Serial Port or Custom TCP Socket, IPC, Shared Memory Other custom protocol File – Shared Disk, FTP, or File Services in ESB Service Container Database –Stored Procedure, SQL, Trigger, Table Polling Standard Protocol – HTTP, FTP, SMTP Messaging – MQ Series, Tib Native Sonic JMS Clients.NET™, Java™, COM, C/C++ ESB Custom Service to App Logic Adapter to App Logic Web Service (over JMS preferred) Connect in the most effective manner Business Event Technology Event

© 2006 Progress Software Corporation16 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Best Practice: Canonical XML Message Transform from Native Format to “XML Canonical” NOTE: Move from Native directly to “Canonical” if Possible!

© 2006 Progress Software Corporation17 SOA-11: SOA Design Best Practices: Bottom-Up Methodology “On-Ramps” and “Off-Ramps” for OpenEdge®  On-Ramp Use OpenEdge SonicMQ® Adapter for Emitting Real- Time Events from OpenEdge 9.1x or OpenEdge 10.x Environments File Drop for Others – Not Ideal, but last resort!  Off-Ramp Use OpenEdge SonicMQ Adapter for 9.1x Use OpenEdge Adapter for Sonic ESB® for 10.X Java Open Client/ESB Service for OE 9.1x or 10.X for Complex Interactions between ESB and OE Environment

© 2006 Progress Software Corporation18 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Agenda  The “Organic Approach”  Qualification Criteria for the First ESB Project  Patterns and Integration Scenarios  Scoping a SOA Project “Bottom-Up”

© 2006 Progress Software Corporation19 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Four steps for scenario analysis  Step 1: Define the Basic Patterns  Step 2: Draw out the Details of Mediation  Step 3: Correlate the Scenario to Other Scenarios  Step 4: Extend the Scenario

© 2006 Progress Software Corporation20 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Global purchasing example ERP Purchase Order Purchasing System ERP Purchase Order ERP Purchase Order Manufacturer consolidates purchasing to reduce costs

© 2006 Progress Software Corporation21 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 1: Define the Basic Patterns  Declare the basic on-ramps, off-ramps, and a first pass flow of the use case At this point, the flow of the use case will probably look very basic Steps 2, 3, and 4 will begin to flesh out the scenario and draw out additional requirements for mediation in the ESB.  Iterate over the use case and continually refine it as you learn more. On-rampOff-ramp Flow Start with simple drawing

© 2006 Progress Software Corporation22 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 2. Draw out the Details of Mediation  Are there syntactic differences in the data? Ask detailed questions about the data

© 2006 Progress Software Corporation23 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 2. Draw out the Details of Mediation  Are there syntactic differences in the data?  Is the data presented in “bulk”, but must be manipulated on an individual record basis? Or Vice Versa? Ask detailed questions about the data

© 2006 Progress Software Corporation24 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 2. Draw out the Details of Mediation  Are there syntactic differences in the data?  Is the data presented in “bulk”, but must be manipulated on an individual record basis? Or Vice Versa?  Are there semantic differences with the data? Ask detailed questions about the data

© 2006 Progress Software Corporation25 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 2. Draw out the Details of Mediation  Are there syntactic differences in the data?  Is the data presented in “bulk”, but must be manipulated on an individual record basis? Or Vice Versa?  Are there semantic differences with the data?  Are there pieces of data that are missing? Ask detailed questions about the data

© 2006 Progress Software Corporation26 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Add in real-time elements over time Batch Real-Time Reusable Components Example

© 2006 Progress Software Corporation27 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 3: Correlating the Scenarios  The power of the ESB becomes evident when the same services, events, and data are reused in other contexts  Use these probing questions: Are there other scenarios that could benefit from services, events, or data? Could other departments, organizations, or partners benefit from any of the information? Is this related to a “master business process” that needs to be tracked and managed? Look for 2 nd order interactions

© 2006 Progress Software Corporation28 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Reuse of Events and Services Purchase Order Event Product Cross Referencing Service Product Lookup Service Consolidate Purchase Order Service Can these be used elsewhere?

© 2006 Progress Software Corporation29 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Reuse Comes in Many Forms  Reuse of Deployed Business Events and Services (Previous Slide)  Reuse of Integration Service Capabilities Reuse general purpose mediation services other integration scenarios  CSV to XML Service Reuse On Ramps and Off Ramps for other Integration Scenarios  OE Adapter Reuse processes (composite services)  Batch to Real-Time example

© 2006 Progress Software Corporation30 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Look for the Master Business Process  What is the process? How is this integration related to an overall business process? What steps is this integration fulfilling in that process?  Who are the stakeholders? Who in the organization is involved in the specific steps in the process?  Can the stakeholders view, react, and fix? Measure how long a process is taking? Know when exceptions occur in that business process? Ask these probing questions

© 2006 Progress Software Corporation31 SOA-11: SOA Design Best Practices: Bottom-Up Methodology The Bigger Picture: Procurement Process ERP Purchase Order Internal Requisition External Requisition Purchasing System Re-Key Order Why? Eliminate Manual Tasks Reduce Latency of overall process Provide visibility into status of overall process Can this process be automated?

© 2006 Progress Software Corporation32 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Step 4: Extend the Scenario  The purpose of this step is to demonstrate additional value for an ESB enabled integration and how easy it is to enable  Additional uses revolve around the following topics: Audit + Logging Business Activity Monitoring What about administration and control

© 2006 Progress Software Corporation33 SOA-11: SOA Design Best Practices: Bottom-Up Methodology In Summary  Evolve to a SOA using Top Down and Bottom- Up approaches  Focus on Reuse of events and services  Embrace change and Evolve through configuration of mediation logic in the ESB

© 2006 Progress Software Corporation34 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Relevant Exchange Sessions  SOA-1: Fundamentals of Service Oriented Architecture  ARCH-5: Service Interfaces in Practice  SOA-7: Designing Sonic ESB Services and Processes for the ESB Developer

© 2006 Progress Software Corporation35 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Education / Documentation References   Sonic Icons

© 2006 Progress Software Corporation36 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Questions?

© 2006 Progress Software Corporation37 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Thank you for your time

© 2006 Progress Software Corporation38 SOA-11: SOA Design Best Practices: Bottom-Up Methodology

© 2006 Progress Software Corporation39 SOA-11: SOA Design Best Practices: Bottom-Up Methodology Benefits of moving mediation into the ESB Format Protocol Destination Sequence Recovery Core Business Logic Application with Mediation Logic ESB Supports 100’s of protocols, and will evolve with the latest WS-* standards for maximum interoperability ESB’S messaging foundation provides reliable mechanisms for guaranteed delivery available today ESB’s configurable routing mechanism provides an easy method of making routing changes Embed multiple format handling and extend the range of the information by pushing it into the bus Addition of new systems or change in routes does not require a system or application change