1. Test Planning Mike O’Dell (some edits by Vassilis Athitsos)
Based on an earlier presentation by
Mike O’Dell, UTA
test

2. Test Planning … when? Occurs throughout the product life cycle
Requirement completeness
Architectural integrity
Component design precision
Implementation correctness
But all too often we wait until the end to “validate” what we have already done.
CSE 4317

3. System Verification System Definition SRD: System Requirements
ADS: Architecture
Specification
DDS: Detailed
Design Specification
Implementation
MAP:
All Subsystem & Layer
Interfaces &
Interactions
MAP:
All Module
Interfaces &
Interactions
MAP:
All Specified
Requirements
MAP:
All Modules
System Validation Test
Integration Test
Component Test (a.k.a. Function Test)
Unit Test
System Verification
CSE 4317

4. System Verification
Unit Test: verifies that EVERY module (Hardware & Software) specified in the DDS operates as specified.
Component/Function Test: verifies integrity of ALL inter-module interfaces and interactions.
Integration Test: verifies integrity of ALL inter-subsystem/layer interfaces and interactions.
System Verification Test: verifies ALL requirements are met.
CSE 4317

5. System Test Plan Contents
Introduction
can be very short, unless you need more content.
Executive Summary (purpose, scope, etc.)
References
SRD (requirements drive the system-level tests you do)
ADS (architecture drives the subsystem and integration testing)
Etc. (detailed design, standards, sample plans, etc. that are test drivers)
CSE 4317

6. System Test Plan Contents
Test Item(s) – What will be tested
Functions
Modules
Layers
Subsystems
Interfaces between modules, layers subsystems.
NOT customer requirements (this comes under features).
CSE 4317

7. System Test Plan Contents
Test Item(s) – What will be tested for each item.
What are specific risks for that item, that you need to test against?
Correct output for specific inputs? Are some functions overly complex and make this difficult?
Possible memory leaks?
Does not overburden CPU, memory, network?
Mean time between failures?
Other…?
CSE 4317

8. System Test Plan Contents
Limitations of product under test (restrictions, assumptions, caveats, etc.)
This may not apply to your project. Ask yourselves: can I test the real product in realistic comditions? In most of your projects you can.
Other product-level constraints on testing
Risks (in test phase)
Specific risks that may affect testing/test outcome
Impact assessment and management plan as it relates to these risks
CSE 4317

9. System Test Plan Contents
Features to be tested
List of specific features (from the USERS’ viewpoint) that you will test
Features NOT to be tested
List of those features (USERS’ perspective) that cannot/will not be tested
Describe rationale for not testing each item (not in current release, insufficient test capability, low risk feature that is hard to test, not a user-visible feature)
CSE 4317

10. System Test Plan Contents
Approach (Strategy)
Overall test strategy (what, why, and how of your test plan)
Some descriptions can be generic (e.g., for each function we will verify that it gives correct output given specific inputs).
Some descriptions can be specific (e.g., "we will verify that this function can execute within 2 seconds", or "we will verify that communication between these two modules can sustain up to 10 requests per minute", or "this subsystem does not require too much memory").
CSE 4317

11. System Test Plan Contents
Approach (Strategy)
Any special tools/debuggers used for testing?
Number of hardware and software configurations tested/not tested
Plan to deal with defects identified (re-integration, regression)
Identify metrics for overall success (number of bugs found/corrected/open, number of critical features passed, etc.)
Identify any special requirements for testing
CSE 4317

12. System Test Plan Contents
Item Pass/Fail criteria
List specific criteria for each unit/module/feature to be tested (number and severity of defects, any specific tests that indicate system failure, etc.)
Test Deliverables
List what is to be delivered by this test plan (plan document, test cases, error logs, etc.)
Incident/Bug/Problem Reports (IRs)
Test Schedule (should be consistent with your published project plan)
CSE 4317

13. Where to Place Limitations, Constraints, Risks?
Some limitations, constraints, risks may be particular to specific items.
Those are best discussed when you discuss the item to be tested.
Some limitations, constraints, risks may be generic a lot of items or even to the whole product under testing.
Those are best discussed in the overall strategy section.
CSE 4317

14. System Test Plan - Resources
IEEE Std , IEEE Standard for Software Test Documentation
Test Plan Outline for IEEE Format (from Gerrard Consulting)
CSE 4317