1. System Testing and Quality: How Real Partnerships Deliver Accelerated Time-to-Market & Increased Revenue David Gehringer VP Marketing, Fanfare

2. About Fanfare Fanfare provides software solutions to 70 global high-tech equipment manufacturers and service providers that simplify and accelerate system and device testing. Our technology enables QA teams and developers to build and run automated tests, measure pass-fail criteria and automatically generate documentation that can be easily shared. We help accelerate time to market while increasing product quality.

3. Quality Costs Quality may not be job 1, but it should be Poor quality…. – Costs in development cycles – Lengthens testing – Delays releases – Create customer support costs – Deteriorates brand equity – Cost 100x to fix once in the field

4. Defect Case Study Global leading Carrier and Manufacturer Realize these costs are for each company Deep breath… – None of these costs resolve the defects ActionTime Communication about defect (Time Zones / Language barriers) 2 Days Average # of defects discussed (mostly bugs but some are not) 400 Time reporting defects800 Man Days (~4 man years) Cost to discuss defects / release~$500k

5. Product Release Schedule 0 3 6 9 12 15 18 24 Device Testing Service Provider Device Testing & Feature Validation Service Provider Device Testing & Feature Validation Interoperability and System Testing Deployment Months to Market from first production prototype

6. Quality is Really Simple Developers Customer Advocacy Support Automation Service Provider Feature testing Regression Testing Abstraction Response map ownership Testbed maintenance Only Legacy on tcl/perl Testers System Testing Personal Regression Legend TC – Test case TB – Testbed TR – Test Report RM – Response Map CR – Capture Report Proc - Procedures System Testing Interoperability testing Device testing Regression Systems Unit Testing Build Regression

7. Quality Workflow Testing assets should flow from the manufacturer to the service provider – The tests should have already passed – Assets reduce the work of the service provider – Speeds identification of defects – Triage defect causes quickly

8. What Assets to Share Test Cases Test Reports Setup & tear down procedures Test bed / topology files Abstraction – Output, such as device response, GUI elements, Data sets, etc.. – Device centric variables

9. What is Failing In-house systems – Information and assets hard to move and share – Sapping the best talent to maintain – Creates a center of the universe problem Scripting – Similar to coding and is hard to understand – Small talent pool – Brittle and costly to maintain

10. Testing Relationship

11. Value Chain of Quality

12. Quality is Simple Developers Customer Advocacy Support Automation Service Provider Feature testing Regression Testing Abstraction Response map ownership Testbed maintenance Only Legacy on tcl/perl Testers Version Control Response Maps Procedures System Testing TC RM Procs RM, Procs TC TR CR TR TC TC TR TC TR TB Procs Personal Regression Local RM, TB, Proc s Legend TC – Test case TB – Testbed TR – Test Report RM – Response Map CR – Capture Report Proc - Procedures System Testing Interoperability testing Device testing Regression Systems Unit Testing Build Regression Asset Repository

13. Quality Business Equation It is in the financial best interest of both manufactures and service provider to work as partners for quality improvement. Quality Issues – Delay sales, delay deployment, and raise support – Reduce both the top line and bottom line – Customer loss and brand erosion

14. Service Provider Trends – Business advantages to NEMs who provide assets – Better relationships drive ongoing costs down – RFPs that require test assets from NEMs

15. The Good Samaritan It is not greening the planet, but high quality makes good business sense. Those who figure this out will have significant market advantage.

16. Q & A David Gehringer. dgehringer@fanfaresoftware.com

17. Sharing Quality Both the manufacturer and service provider should establish testing communication platform to: – Allows test to flow bi-directionally. – Speeds defect understanding and reproduction – Accelerates resolution and verification – Reduces quality time line for faster time to market