© 2006, iTKO, Inc. | All rights reserved. Achieving 99.99% Quality in SOA Applications – The “Headless” Dilemma John Michelsen StickyMinds Webinar November 7, 2006

Agenda SOA development and QA trends Strategy/best practices - the Three C’s “Headless” Testing Fundamentals LISA Intro, and LISA WS-Testing Offer

Who are we?  1999 – iTKO Incorporates (Privately Held)  2001 – LISA under development – (4) Patents Pending  2003 – LISA 1.0 Released to Market  2004 – LISA 2.0  LISA 3 & 3.5 A History of Quality Leadership Entered the market to deliver an SOA testing solution, LISA Current version 3.5 (2006) Key value propositions: Increase Test Coverage Lower Cost per Test Faster Discovery & Resolution Everyone should own quality™.

Platform Evolution Causes Testing Evolution

The Promise of SOA Reduce integration cost Through loosely-coupled forms of integration and industry standards Increase asset reuse Build new business workflows from existing Services to form composite applications Increase business agility Better control business process definition and management to meet customer needs Reduce business risk Governance, compliance, and risk reduction through increased business visibility and agility

Server (database) Browser UIs Standard enterprise apps Client UIs …before SOA You had a standard delivery platform. And a standard infrastructure.

Transaction service Data warehouse BI tools Core app Partner Reseller Customer buyer Business rules Order management Outsourced supplier Transaction provider Partner Reseller Customer buyer Your extended enterprise Division 1 Division 2 SOA promises a composite workflow SOA was supposed to simplify extending workflows.

Transaction service Data warehouse BI tools Your App Workflow Channel Partner RMI objects Customer company CRM Web App Business Rules Outsourced firm Legacy App SOAP objects Web Services.NET Ordering Service Content Database Legacy App Web interface Messaging service File System Legacy Data Division2 Financials Mainframe Division 1 Your Company ESB But heterogeneous technology is complex Standards and components are still evolving Composite apps have multiple owners It’s Continuously Changing

Why A New Strategy? Few changes at the database level, but multiplies issues downstream For every middle tier change, there are hundreds of front-end impacts possible at the UI layer Limitless front end options Any client, multiple customers Mix of technologies – XML/RPC, Swing, AWT, JS, DOM, now AJAX Services consume business logic Database/ mainframe = 1 change UI level =1000x Middle-tier level = 100x

Best Practices: “Three C’s” of SOA Testing Complete Testing Every heterogeneous layer of architecture Test UI verify in system of record Reuse functional test for performance testing Collaborative Testing Test early before UIs are created Not just dev, business analysts and QA should verify processes Continuously Test Regress on existing functionality Add testing of new services to existing testing workflows Complete > single component  all technologies Continuous > Phased tests  constant testing Collaborative > Dev/QA test silos  everyone tests, every phase

Data / Applications Process and services layer Complete: SOA Testing across every layer Presentation layer Integration layer Web UI RMI DatabaseLegacy AppFile System ASP.NET MQ Custom API Swing J2EE SOAP/XML CORBA ESB BPM.NET WS

Additional Apps Now & Future Additional Apps Now & Future Additional Apps Now & Future CORBA App Typical Trading System Example Ordering Swing App ESB Processing Web App Business Process App A Business Process App B Additional Apps Now & Future Database DB RMI Data is correlated in here. 1 9 App Server EJB, WS, POJO

Collaborative: SOA Requires Agile QA expands dev tests and create functional and load tests Complete Test Coverage Lower Cost per Test Faster Discovery & Resolution Developers unit test and jump-start QA Business Analysts outline a business process and ‘not implemented’ test cases Production test their implementation and report issues to support Support rapidly debug issues and communicates appropriately

Collaborative Testing in an SOA environment BPM Validation Orchestration Tests Service Validation Component Tests Service 3 Dev/QE Order Acquisition Check Inv Check Credit Place Order QA/BA Place Order Service 1 Service 3 Service 2 QE QA Continuous SOA Test Platform

Continuous: SOA is never done SOA development is constantly evolving Project-based development had a “test phase” Database Create components Create UI Acceptance Test Deploy App Unit testIntegration New service Database Create UI Database Create UI New service Service changed New service Service retired

ESB Continuous: Minimize Unintended Consequences APP 1 Database RegistryMDMTransactions Internal Service Legacy App Partner Service APP 2 ??? ??? ? ? NEW

ESB Continuous: Minimize Unintended Consequences APP 1 Database RegistryMDMTransactions Internal Service Legacy App Partner Service APP 2 A data service is currently failing due to a change in the internal service Continuous Testing

Continuous: SOA at 5-9’s Business Investment Availability 100% % system uptime Physical Infrastructure Scalability 100% % application availability Integrity 100% ?????% functional integrity Application Infrastructure Business Applications Fault Tolerant HW Diesel Power Redundant Bandwidth App Servers Transaction Monitors Clustered Services SOA-aware Dev/QA Process Comprehensive Testing Tools Continuous Testing Service Availability to the business is the product of all three

Continuous Levels: Build & Deploy Component/ Service Teams Development QA/QE Pre-production Production productcomponentproduct component Dev build test bed Staging Production Dev build test bed Continuous SOA Test Platform Deployment Level Build Level

So how do we defeat the “Headless” dilemma?

Requirements are King -- On-line Music Scenario Use Case: (UC1) Customer with existing account does a search for ‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1- click purchase feature to buy. Business: (BR1) We need a flexible mechanism to change these criteria at our whim. After discussion: for now partners get 20% of net proceeds on their product sold on the first $100,000 / mo, 10% thereafter. PSR: (PSR1) Search can’t take any longer than 3 seconds for a user on a typical broadband connection. Policy: Punt for now. This is security, interop standards, corp dev standards, etc. Functional: (FR1) Search must include results from CDs, DVDs, Books, and Memorabilia, with CDs on the top (UC1). Technical: (TR1) Create a search service that aggregates CD, DVD, Book, and Mem data (FR2). (TR2) Search service execution must be under 2 seconds (PSR1).

Service Level Guidelines Service Validation Component Tests Service 3 Dev/QE UC & BR: Validation mechanism PSR: Load/performance test at the service level FR: Likely 80% of your requirements can be nailed right here TR: Dev test at the component level UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

Orchestration Level Testing Guidelines UC & BR: Not usually PSR: Often the best way to prove PSR FR: Invoke a behavior, validate outcomes TR: Simulate unstable/unavailable services Orchestration Tests Place Order Service 1 Service 3 Service 2 QE QA UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

Composite/BPM Level Testing Guidelines UC: Primary focus, performs a cause that we validate at the service/component level BR: Many times same as UC, sometimes not relevant PSR: A few PSR scenarios proved at this level, but continuous testing is the key here FR & TR: Hopefully very few BPM Validation Order Acquisition Check Inv Check Credit Place Order QA/BA UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

Typical SOA Test Map -- Dramatically Simplified Content Presentation Layer Content Mgmnt Offers & Specials Search Order Mgmnt Commission Rules Svc CDDVD BooksMem CRM System Media Delivery Billing Media Repo AP CRM Facade Batch Feeds UC1 UC1: Search & Buy

Typical SOA Test Map -- Dramatically Simplified Content Presentation Layer Content Mgmnt Offers & Specials Search Order Mgmnt Commission Rules Svc CDDVD BooksMem CRM System Media Delivery Billing Media Repo AP CRM Facade Batch Feeds BR1* BR1 BR1: Commissions

Typical SOA Test Map -- Dramatically Simplified Content Presentation Layer Content Mgmnt Offers & Specials Search Order Mgmnt Commission Rules Svc CDDVD BooksMem CRM System Media Delivery Billing Media Repo AP CRM Facade Batch Feeds TR2* PSR1PSR1: 3 sec search

Typical SOA Test Map -- Dramatically Simplified Content Presentation Layer Content Mgmnt Offers & Specials Search Order Mgmnt Commission Rules Svc CDDVD BooksMem CRM System Media Delivery Billing Media Repo AP CRM Facade Batch Feeds FR1: Search is aggregate, CDs top results FR1

Typical SOA Test Map -- Dramatically Simplified Content Presentation Layer Content Mgmnt Offers & Specials Search Order Mgmnt Commission Rules Svc CDDVD BooksMem CRM System Media Delivery Billing Media Repo AP CRM Facade Batch Feeds TR1 & 2 TR2: 2 sec search TR1: Create search on 4 repos TR1

The Three Dimensions of Quality maturity Complete > single component  all technologies Continuous > Phased tests  constant testing Collaborative > Dev/QA test silos  everyone tests, every phase

LISA meets SOA Testing Challenges LISA is Complete - Test heterogeneous SOA LISA can test every tier of an application, regardless of location LISA can test every service technology with one tool, one test LISA’s framework approach creates bridges to legacy services that means ‘you can get there from here’ LISA is Collaborative – Everyone owns quality LISA shines at testing all components before UI even exists, as well as integrating with the team’s ALM lifecycle and process Unit, regression, system, load, and monitoring in one tool Leverages diverse skill sets in Dev, QA/QE, and BA roles LISA is Continuous – Enables Agile SOA Tests constantly evolving, interdependent systems during and after deployment Traps for the “what-if” unintended system-wide consequences of making changes or corrections to components.

LISA WS-Testing $5,000 Off for next 50k users LISA WS-Testing Edition is THE tool for Web Services testing All our web services goodies are in there: WSDL-based, point and click WS-I Interop testing Functional testing and service simulation UI and Command Line execution

Thank you for your interest! ?? ? iTKO Web Site Resources URL: or White papers Support FAQs, Tech Notes, and Forums (once registered) Download WS-Testing or start a larger evaluation More from me See above Blog: Start with I’m