Testing as a Driver for Development Change Wall Street Systems Graham Thomas

2 Abstract  You have established a testing process but reached the point where to make further improvements you have to change the way development works.  So how do you go about changing the development lifecycle? What are the key areas to address, and how do you cope with the traditional problems of the relationship between the developers and the testers, organisational priorities assigned to testing, and the commercial realities of a software house.  This presentation is case study of work over the last year, kicking-off a change program, and the initiatives that have resulted.

3 Agenda  Improving the testing process  Analysis of the results  Development Change Program  Achievements to date  Work still to do

4 Test Process Improvement m Test Strategy m Test Process S R A D BU I A Unit Testing Integration Testing System Testing Acceptance Testing Stage EntryStage ExecutionStage Exit

5 Test Process Improvement  Fault Database  Process Metrics

6 Test Process Analysis STAGE ENTRY  Some software components delivered late  Incomplete documentation  Test environment availability STAGE EXIT  High percentage of integration faults  Leading to a lack of test coverage  Project priorities steer testing

7 Change Program Software Management Programme Office Development Oracle/Unix Dev. Env. Unit/Int. Testing Timely Documents Release Maintenance Procure CM Tool Software Versioning Configurable Item Control Change Control Testing Automated Testing RQMTS Mgmt Envelope Reg. Test Client Build Testing Release Management Dev. Lifecycle Change Management Programme Management Short TermMid TermLong Term Stream

8 Change Program Models  Incremental  Step Change  Thin Threads

9 Achievements To Date  Development Lifecycle  Requirements process  Test Automation  Unit Testing Tools  Configuration Management  Development Partnership Programme

10 Development Lifecycle Requirements Feasibility Design Build & Unit Test Release Test Implement Requirements Feasibility Design Build & Unit Test Release Test Implement Envelope Test Controlled Delivery Modified for Small items Base Development Model

11 Requirements  Tightened up definition  Testing Specification  Acceptance Criteria  Closely Monitored Process  Drives development activity  Clearly defined targets

12 Test Automation  Generated scripts for key functionality  Run on multiple PC’s controlled through a master  Developed the suite to run unattended and overnight  Successfully execute over 8,000 transactions in a single run

13 Unit Testing Tools  Suite of unit testing tools  Automatic Static Source Code Analysis  Runtime Error Analysis  Automatic Performance Analysis and Optimisation  Automatic Code Coverage Analysis  Automatic Error Detection and Recovery

14 Configuration Management  Identification of CI’s  Branching strategy  Environmental requirements  Stage 1  Secure source CI’s.  Check Out/Check In  3 month time-box Main Release Dev. Client

15 Development Partnership Assisted Development Hosted Test Lab Remote Project Support Hardware Lab Testing Partnership Requirements Feasibility Design Build & Unit Test Release Test Implement Base Development Model

16 Still to Do  Roll-out Unit Testing Tools  CM Phase 2 – Process Change  Introduce Process Workflow with CRM  Increase coverage of automation suite  Leverage automation in other areas  Review testing activities across the whole company

17 The Story So Far  It is hard enough just changing the testing process  You need sponsorship and support to re- engineer the development lifecycle  Change is an inclusive process  Change is continual  It is harder not to change !

18 Where Value was Added  Successes  Test Process Improvement  On-going Change Program  Identified two-speed lifecycle  Tool evaluation for CM & Unit Testing  Broadened the view of quality from QC to QA m Failures  Point solutions only  No quick wins  Pick & Mix approach by development

19 Contact Details Graham Thomas +44 (0)