1. Department of
Computer Science
Regression Testing

2. Regression testing Testing changed software Regression test suite
Retest all
Selective regression testing
Regression test suite
Types of changes
Defect fixes
Enhancements
Adaptations
Perfective maintenance

3. Retest all Assumption:
Changes could have introduced errors anywhere in the code
Expensive, prohibitive for large systems
Reuse existing test suite
Add new tests as needed
Remove obsolete tests (discrepancies between expected and actual output)

4. Selective regression testing
Impact analysis
Only code impacted by change needs to be retested
Select tests that exercise such code
Add new tests if needed
Remove obsolete tests

5. Regeneration strategy
Often for defect fixes when unsure whether defect is really fixed
Develop new tests
Probe around defect fix
Test affected functionality
Comes in retest all, and selective regression testing flavors

6. Regression testing process
Identify changes
Determine which tests will remain valid: T’
Test modified software with T’
If T’ does not meet test criterion, generate new test cases T’’
Execute modified software with T’’

7. Regression testing process (2)
test whether modifications have broken anything
Determined either by retest all, or selective strategy
T’’:
test whether modified code works
Use scope appropriate criteria

8. Classification of tests in T
Reusable:
does not test software modification
Should produce same result
Need not be rerun
Retestable:
Tests software modification
Must be rerun
Obsolete: no longer applies

9. Practical regression testing
Approach
Quality
Change
Impact
Schedule
Full new
Low
Key Code
Extensive
None
Minimal
High
Isolated
Localized
Very tight
Expanded
Moderate
Full reuse
Series of fixes

10. Approaches Full new test cycle Minimum regression test
Redesign entire test plan
Rebuild whole test suite
Minimum regression test
Reuse test plan as much as possible
Rerun minimum number of retestable test cases
Generate minimum number of new test cases for changes

11. Approaches (2) Expanded scope of regression testing
Reuse part of test plan
Rerun all retestable test cases
Generate new test cases on the full scope of changes
Full reuse of existing test suite
Reuse test plan and test suite

12. Attributes of regression tests
Safe:
All tests that could possibly exhibit different outputs on the modified software
Is 100% inclusive
Precision:
Percentage of tests in a reduced test suite that cannot reveal regression faults, and
Are not selected for the reduced test suite

13. Attributes of regression tests (2)
Efficiency: cost of identifying a reduced regression test suite
Generality: range of application for the selection strategy

14. Regression test patterns
Retest changed code
Class, cluster, subsystem scope
Identify segment coverage by test case
Use version control tool to identify changed segments
Select tests covering changed/deleted segments
Develop new tests for new segments
Safe

15. Retest by profile Assumes operational profile Unsafe
Include critical use cases
Include other use cases by frequency in operational profile
No impact analysis
Better: determine impact on use case

16. Retest risky use cases Suspicious use cases Critical use cases Unsafe
Use cases that depend on components, objects, middleware, resources that are
Unstable, unproven
Have not been shown to work together before
Implement complex business rules
Are complex
Were fault-prone during development
Critical use cases
Unsafe

17. Retest within Firewall
Firewall: set of components whose test cases will be included in a regression test
External interface, contract change
Internal implementation change