Software Test Plan Why do you need a test plan? –Provides a road map –Provides a feasibility check of: Resources/Cost Schedule Goal What is a test plan? A document that outlines: 1.What is the goal of this testing effort (e.g. “exit criteria”) 2.What are the items to be tested –High level list (e.g. requirements, design, code) –Detailed features (e.g. from req., design & code) 3.What test process and methodologies will be used –For different deliverables and different features (e.g. inspection versus execution of test cases) –What record keeping and measurements will be performed 4.What resources are needed 5.What is the schedule 6.What are the risks and contingencies May be hard for many students to “grasp” because: - “I” am the resource - schedule is given to me (2 nights before due date) - goal is to “hand in the assignment”

1. Goal(s) of Your Testing Goals may be different for each test. –They often need to match the product goal or the project goal –Generic : “Meets customer requirements or satisfaction” is a bit too broad.

1. Goal(s) of Your Testing Goals may be different for each test. –They often need to match the product goal or the project goal –Generic : “Meets customer requirements or satisfaction” is a bit too broad. –A Level Deeper Examples are: Ensure that all the (requirements) features exist in the system Ensure that the developed components and the purchased components integrate as specified Ensure that performance targets (response time, resource capacity, transaction rate, etc) are met. Ensure the system is robust (stress the system beyond boundary) Ensure that the user interface is “clear,” “novel,” or “catchy,” etc. Ensure that the required functionality and features are “high quality.” Ensure that industry standards are met. Ensure that internationalism works (laws and looks) Ensure that the software is migratable Note that this list seem to deal more with “implemented” design and code --- what about req. doc. itself?

2. Test Items – (what artifacts?) Testing the deliverables (Depends on the goal): –Requirements specification –Design specification –Executable code –User guides –Initialization data Testing the individual function/features (Depends on the goal): –Logical application functions (list all of them – usually from requirement spec from the users perspective) –At the detail level for small applications (list all the modules – from design spec. or build list from development perspective) –User interfaces (list all the screens – usually from Req. and Design spec ---- from user perspective) –Navigation and especially the “sequence” (e.g. usability is a goal) Looking at Major Artifacts Looking at Details of the artifacts

What is NOT included in the Test List of items not included in this test Possible Reasons of NOT testing the code: –Low risk items ---- how determined? –Future release items that has finished coding phase (and unfortunately “may be” a problem if included in the code) –Not a high priority item ( from req. spec.)

3. Test Process and Methodologies What testing methodology will be applied: –Non executable – formal inspection, informal reviews, (unit test user guide examples) –Executable – unit test, functional test, system test, regression test, etc. Blackbox testing –Logic – Decision Table testing –Boundary Value Equivalence testing –Stress testing and performance testing –Installation test Whitebox testing –Path testing : Code coverage; Branch coverage; linearly indep. paths, D-U path coverage –Object testing (inheritance, polymorphism, etc.) What data will be gathered and what metrics will be used to gauge testing. –# of test cases and the amount of coverage (paths, code, etc.) –# of Problems found by severity, by area, by source, by time –# of problems fixed, returned with no resolution, in abeyance –# of problems resolved by severity, turn around time, by area, etc. How to gauge if goals are met? –Metrics for validating the goal –Decision process for release Bug seeding approach?

4. Test Resources Based on the amount of testing items and testing process defined so far, estimate the people resources needed The skills of the people Additional training needs # of people The non-people resources needed –Tools Configuration management Automated test tool Test script writing tool –Hardware, network, database –Environment (access to: developers, support personnel, management personnel, etc.)

5. Test Schedule Based on: –Amount of test items –Test process and methodology utilized –Number of estimated test cases –Estimated test resources A schedule containing the following information should be stated: –Tasks and Sequences of Tasks –Assigned persons –Time duration

6. Risks and Contingencies Examples of Risks: –Changes to original requirements or design –Lack of available and qualified personnel –Lack of (or late) delivery from the development organization: Requirements Design Code –Late delivery of tools and other resources State the contingencies if the Risk item occurs –Move schedule –Do less testing –Change the goals and exit criteria

7. Appendix of Actual Test Cases General Test Case Specification Template Developed Test Cases (Modular Level): –Black Box Testing –White Box Testing Developed Test Cases (component/System Level): –Component Level –System Level

You may want to see: IEEE 829 Documents on Testing Test preparation documents: –IEEE 829 –Test plan –IEEE 829 – Test Design Specifications –IEEE 829 – Test Case Specifications –IEEE 829 – Test Procedure Specification –IEEE 829 – Test Item Transmittal Report Test Running documents: –IEEE 829 – Test Log –IEEE 829 – Test Incident Report Test Completion Document: –IEEE 829- Test Summary