System Testing In a Hurry British Computer Society Specialist Interest Group In Software Testing 6th December 1999 Graham Thomas Wall Street Systems

BCS SIGIST06/12/ Abstract “Hi Graham, we have a project which finished development last Friday and starts system testing today (Monday).” “Unfortunately all of our testers are pre-occupied with Year 2000 projects and we can’t spare them, so we thought that we would ask the development team to carry out the system testing. “ “Can you talk to the team for a couple of hours and tell them everything they need to do ‘system testing in a hurry’?....How long do you need to prepare?....You have a day!”

BCS SIGIST06/12/ Contents l The Presentation l Testing Progress l Feedback from Development l A surprising conclusion

BCS SIGIST06/12/ The Presentation l Planning l Strategy l Test Design l Techniques l Monitoring l Feedback

BCS SIGIST06/12/ Planning l People l Accommodation l Equipment m PC’s, printers, modems l Support; m Development m Operations m Business l Environment provision m CICS Regions m DASD (Disk space) Requirements l How many l Skills l Availability (courses, holidays, leavers, joiners, training) l Need to generate a Plan l Requires a Test Manager role

BCS SIGIST06/12/ Strategy Integration Retro-Fit Regulatory TrialsAccp.System Testing Lifecycle Unit

BCS SIGIST06/12/ Strategy Requirements Analysis Design BuildUnit Integration System Acceptance Testing V Model

BCS SIGIST06/12/ Strategy Requirements Design BuildUnit Integration Acceptance AnalysisSystem Functional Design Spec. Testing V Model

BCS SIGIST06/12/ Test Design Test Methodology Functional Design Spec. Function Accp. Criteria (logical) Condition (physical) Case Script Network Test Data

BCS SIGIST06/12/ Techniques l Boundary Value Analysis l Equivalence Partitioning l Error Guessing Tube Ticket Example Child 0 FreeAdultOAP 51660

BCS SIGIST06/12/ Scripting l Write high level scripts to save time l Make scripts object based e.g. m Select Contract, Print Contract l Include reference to test case for traceability StepActionTest CaseExp. ResultsRef.Act. Results Script Template

BCS SIGIST06/12/ Execution l Common Test Faults m The Test Data m The sequence of instructions (script) m Less likely to be a code problem !!! l Problem Recording m Record the scene of the crime m Don’t waste time trying to diagnose fault m Just record instance of fault via Post-it note

BCS SIGIST06/12/ Monitoring l Don’t get bogged down measuring 6 Test Case generation 6 Script generation 6 Function coverage l Recommended 2 simple ways to monitor m S - Curve m Outstanding Faults

BCS SIGIST06/12/ S - Curve

BCS SIGIST06/12/ Faults Outstanding

BCS SIGIST06/12/ Presentation Feedback l Developers m Not very structured l Management m Relieved to have a structured testing method and limited support l Client Testing Management m Very keen to adopt the proposed method

BCS SIGIST06/12/ Interlude

BCS SIGIST06/12/ Testing Progress l Assigned 1 member of testing team to assist project on a part time basis l Developers found it “a bit boring” generating the test cases l Development Team Leader welcomed a structured testing method l All went very quiet...

BCS SIGIST06/12/ Feedback from Development l The testing support was welcomed l The ‘method’ hung together l The test manager was in control of testing l Actually gave the developers a different perspective on testing l Took the pressure off what could have been a disastrous project

BCS SIGIST06/12/ A Surprising Conclusion l The project was successful m Testing finished on time m Good quality product handed-over m Minimal impact to Y2K program 8 4 l But was it really? m “We now test the way Graham showed us” No early testing feedback No advanced test planning No metrics No repeatability

BCS SIGIST06/12/ Start Testing Early Requirements Analysis Design BuildUnit Integration System Acceptance Testing V Model

BCS SIGIST06/12/ Contact Details Graham Thomas Testing Manager Wall Street Systems Phone: