1. System Testing In a Hurry British Computer Society Specialist Interest Group In Software Testing 6th December 1999 Graham Thomas Wall Street Systems

2. BCS SIGIST06/12/1999 2 Abstract “Hi Graham, we have a project which finished development last Friday and starts system testing today (Monday).” “Unfortunately all of our testers are pre-occupied with Year 2000 projects and we can’t spare them, so we thought that we would ask the development team to carry out the system testing. “ “Can you talk to the team for a couple of hours and tell them everything they need to do ‘system testing in a hurry’?....How long do you need to prepare?....You have a day!”

3. BCS SIGIST06/12/1999 3 Contents l The Presentation l Testing Progress l Feedback from Development l A surprising conclusion

4. BCS SIGIST06/12/1999 4 The Presentation l Planning l Strategy l Test Design l Techniques l Monitoring l Feedback

5. BCS SIGIST06/12/1999 5 Planning l People l Accommodation l Equipment m PC’s, printers, modems l Support; m Development m Operations m Business l Environment provision m CICS Regions m DASD (Disk space) Requirements l How many l Skills l Availability (courses, holidays, leavers, joiners, training) l Need to generate a Plan l Requires a Test Manager role

6. BCS SIGIST06/12/1999 6 Strategy Integration Retro-Fit Regulatory TrialsAccp.System Testing Lifecycle Unit

7. BCS SIGIST06/12/1999 7 Strategy Requirements Analysis Design BuildUnit Integration System Acceptance Testing V Model

8. BCS SIGIST06/12/1999 8 Strategy Requirements Design BuildUnit Integration Acceptance AnalysisSystem Functional Design Spec. Testing V Model

9. BCS SIGIST06/12/1999 9 Test Design Test Methodology Functional Design Spec. Function Accp. Criteria (logical) Condition (physical) Case Script Network Test Data

10. BCS SIGIST06/12/1999 10 Techniques l Boundary Value Analysis l Equivalence Partitioning l Error Guessing Tube Ticket Example Child 0 FreeAdultOAP 51660

11. BCS SIGIST06/12/1999 11 Scripting l Write high level scripts to save time l Make scripts object based e.g. m Select Contract, Print Contract l Include reference to test case for traceability StepActionTest CaseExp. ResultsRef.Act. Results Script Template

12. BCS SIGIST06/12/1999 12 Execution l Common Test Faults m The Test Data m The sequence of instructions (script) m Less likely to be a code problem !!! l Problem Recording m Record the scene of the crime m Don’t waste time trying to diagnose fault m Just record instance of fault via Post-it note

13. BCS SIGIST06/12/1999 13 Monitoring l Don’t get bogged down measuring 6 Test Case generation 6 Script generation 6 Function coverage l Recommended 2 simple ways to monitor m S - Curve m Outstanding Faults

14. BCS SIGIST06/12/1999 14 S - Curve

15. BCS SIGIST06/12/1999 15 Faults Outstanding

16. BCS SIGIST06/12/1999 16 Presentation Feedback l Developers m Not very structured l Management m Relieved to have a structured testing method and limited support l Client Testing Management m Very keen to adopt the proposed method

17. BCS SIGIST06/12/1999 17 Interlude

18. BCS SIGIST06/12/1999 18 Testing Progress l Assigned 1 member of testing team to assist project on a part time basis l Developers found it “a bit boring” generating the test cases l Development Team Leader welcomed a structured testing method l All went very quiet...

19. BCS SIGIST06/12/1999 19 Feedback from Development l The testing support was welcomed l The ‘method’ hung together l The test manager was in control of testing l Actually gave the developers a different perspective on testing l Took the pressure off what could have been a disastrous project

20. BCS SIGIST06/12/1999 20 A Surprising Conclusion l The project was successful m Testing finished on time m Good quality product handed-over m Minimal impact to Y2K program 8 4 l But was it really? m “We now test the way Graham showed us” No early testing feedback No advanced test planning No metrics No repeatability

21. BCS SIGIST06/12/1999 21 Start Testing Early Requirements Analysis Design BuildUnit Integration System Acceptance Testing V Model

22. BCS SIGIST06/12/1999 22 Contact Details Graham Thomas Testing Manager Wall Street Systems Phone: 020 7827 0430 e-mail: graham.thomas@wallstreetsystems.com