1. 1 March 19, 2008 1 Test Plans William Cohen NCSU CSC 591W March 19, 2008

2. 2 2 Why Have a Test Plan? ● Organize testing ● Show scope of testing ● Provide schedule when things done ● Release criteria (standards of “good enough”) ● Risks and contingencies

3. 3 March 19, 2008 3 IEEE 829-1998 ● Institute of Electrical and Electronic Engineer standard ● Describes documentation for software tests ● Does not describe requirements for testing

4. 4 March 19, 2008 4 IEEE 829 Sections ● Test Plan Identifier (TPI). ● References ● Introduction ● Test Items ● Software Risk Issue ● Features to be Tested ● Features not to be Tested ● Approach ● Item Pass/Fail Criteria ● Entry & Exit Criteria ● Suspension Criteria and Resumption Requirements ● Test Deliverables ● Remaining Test Tasks ● Environmental Needs ● Staffing and Training Needs ● Responsibilities ● Planning Risks and Contingencies ● Approvals

5. 5 March 19, 2008 5 Test Plan Identifier ● Title/Name to uniquely identify test plan ● Test plan may be revised: ● Have version or date information

6. 6 March 19, 2008 6 References ● Pointers to other documents related to the test plan ● Could include: ● Project Plan ● System Requirements specifications ● High Level design document ● Detail design document ● Development and Test process standards ● Methodology ● Low level design

7. 7 March 19, 2008 7 Introduction ● Executive summary ● Describe scope of testing ● Details left to other sections

8. 8 March 19, 2008 8 Test Items ● List of whatis in the scope of testing ● Functional area ● Applications

9. 9 March 19, 2008 9 Software Risk Issue ● Describes the problem areas that might be encountered during testing: ● Dependencies on third party product ● Changes in packaging or tool ● Changes to the software being tested

10. 10 March 19, 2008 10 Features to be Tested ● Describes user view of what is to be tested ● List risk for the feature tested

11. 11 March 19, 2008 11 Features not to be Tested ● What is NOT being testing ● Things that are low risk ● Things not included in the release of the software ● Things not documented as functional in software

12. 12 March 19, 2008 12 Approach ● General test strategy ● What metrics used for testing ● Environment: ● Hardware ● Software ● Testing tools

13. 13 March 19, 2008 13 Item Pass/Fail Criteria ● How is release/do not release decision made? ● What type of issues cause a “stop ship”

14. 14 March 19, 2008 14 Entry & Exit Criteria ● What conditions present to start testing ● What conditions stop testing ● Want to get as much test coverage as possible, so avoid premature exit

15. 15 March 19, 2008 15 Suspension Criteria and Resumption Requirements ● Reasons for suspension: ● Dependent on unavailable system (e.g. Hardware down) ● Defect prevents further testing ● Holidays ● Reasons for resumption: ● System becomes available (hardware repaired) ● Defect fixed ● Holiday ends

16. 16 March 19, 2008 16 Test Deliverables ● How are the test results presented? ● Lists of failures ● List of what portions of the code is tested

17. 17 March 19, 2008 17 Remaining Test Tasks ● Testing may be phased ● Parts of the testing available at different times ● May be better to give early partial results than later full results: ● Results on i386 machine then results on s390 when they become available

18. 18 March 19, 2008 18 Environmental Needs ● What is required to run the tests? ● Hardware ● Software ● Sole user of machine while running? ● Special configuration: ● Ability to trigger power fail on UPS to see that machine shuts down properly ● Ability to reset or power cycle machine

19. 19 March 19, 2008 19 Staffing and Training Needs ● Teach people how to use the application or system

20. 20 March 19, 2008 20 Responsibilities ● Who doing what? ● Make sure coverage of items: ● Scoping risks ● Scoping testing ● Designing the test strategy ● Training people ● Making the ship/no-ship decision

21. 21 March 19, 2008 21 Planning Risks and Contingencies ● How to handle problems ● Adjustment of schedule for test problems ● Adding more resources to meet schedule ● Changing (reducing) scope to fit alloted time ● Allowing more defects

22. 22 March 19, 2008 22 Approvals ● Who approves the process? ● Write the document for the appropriate audience: ● Programmers ● Users ● Testers

23. 23 March 19, 2008 23 Further Reading ● http://en.wikipedia.org/wiki/Test_plan http://en.wikipedia.org/wiki/Test_plan ● http://en.wikipedia.org/wiki/IEEE_829 http://en.wikipedia.org/wiki/IEEE_829 ● http://www.stellman-greene.com/aspm/content/view/39/41/ http://www.stellman-greene.com/aspm/content/view/39/41/