1. Regression
Testing

2. Regression Testing Software maintenance is expensive
Development may take 2 to 4 years
Same may have to be maintained for 10 to 15 years

3. Regression Testing Reason for software change Errors during actual use
Additional functionality
Changes in external world
Restructuring work
Change in technologies
6. Obsolete capabilities to be deleted.

4. Regression Testing This retesting is called regression testing.
Development testing
When we modify software, we typically retest it.
This retesting is called regression testing.
Regression testing is the process of retesting the modified parts of the software and ensuring that no new errors have been introduced into previously tested code.

5. Regression Testing Purposes
Increase confidence in the correctness of the modified program.
Locate errors in the modified program.
Preserve the quality & reliability of software.
Ensure the software’s continued operation.
We may perform regression testing during latter stage of software development.

6. Regression Testing Sr.No. Development Testing Regression Testing
We can make use of existing test suites and test plans 1
We create test suites and test plan
We retest modified components or affected by modifications 2
We test all software components 3
Budget give time for testing
No time 4
Once on software product
Many times
Performed under the pressure of release date
Performed in crisis situations, under greater time constraint. 5

7. Regression Testing Regression Test Selection
Objective is to select an appropriate subset of the test suite to be run.
Fragment F
Fragment F’
S1. y=(x-1)*(x+1)
S2. if (y=0)
S3. return (error)
S4. else
S5. return (1/y)
S5. return(1/(y-3))

8. Regression Testing Test Cases Test # Input Execution History t1 x = 1
S1, S2, S3 t2
x = -1 t3
x = 2
S1, S2, S5 t4
x = 0

9. Selective retest techniques
Retest all
Partition existing test suite
Reusable
Retestable
Obsolete
Unmodified code & unmodified specification
Modified code & modified specification
Specify incorrect Input Output relationships due to specification modification.
No longer exercise the components & specifications they were designed.

10. Selective retest techniques classification Minimisation techniques
Regression Testing
First
Specification obsolete test cases
2nd
Coverage obsolete test cases
Selective retest techniques classification
Coverage techniques
Minimisation techniques
Safe techniques

11. Regression Testing Coverage Techniques Minimization Techniques
They locate coverable program components that have been modified, and select test cases that exercise these components.
Minimization Techniques
They work like coverage techniques, except that they select minimal sets of test cases.
Safe Techniques
They select every test case that causes a modified program to produce different output than its original version.

12. Regression Testing Theoretical Foundations P : Program
: modified program
S : specifications for P
: specifications for P’
P(i) : refers to the output of P on input i.

13. Regression Testing (i) : refers to the output of P’ on input i.
S(i) : refers to specified output for P on input i
(i) : refers to specified output for P’ on input i.
T : set of test cases (test suite) created to test P

14. < identifier, input, output >
Regression Testing
A test case is a 3 - tuple,
< identifier, input, output >
< t, i, s(i) >
Selective retest problem
Find a way, making use of T, to gain sufficient confidence that P’ is correct.
Select T, a set of test cases to re-execute on .
2. Test with , to establish the correctness of with respect to .

15. Regression Testing
3. If necessary, create , a set of new functional or structural test cases for .
4. Test with , to establish correctness of with respect to .
5. Create , a new test suite and test history for , from , and .
Step 5 addresses the test suite maintenance problem: the problem of updating and storing test information.

16. Regression Testing Fault revealing test cases
We want to select of T that will establish confidence that was modified correctly.
Objective is to choose test cases from T that detect faults.
A test case t detects a fault in if it causes to fail. Hence t is fault revealing for .
There is no effective procedure which can select test cases of T that are fault revealing for .
We may select a superset

17. Regression Testing Modification revealing test cases
A test case t is modification revealing for P and if and only if it causes the outputs of P and to differ.
Two assumptions
1. P-Correct-for –T : For each test case t in T, when P was tested with t, P halted and produced the correct outputs.
2. Obsolete – Test - Identification
Determine for each test case in T, whether t is obsolete for

18. Regression Testing Modification Traversing test cases
Test case t is obsolete for if and only if t either specifies an input to that, according to , is invalid for , or t specifies an input output relation for
With two assumptions, we may find modification revealing test cases but it is not easy.
Modification Traversing test cases
A test case is modification traversing for P and if and only if :
a. Executes new or modified code in
b formerly executed code that has since been deleted from P.

19. Modification revealing Modification traversing
Regression Testing
obsolete
Test suite T
Fault-revealing
Non-obsolete
Modification revealing
fault revealing
Modification traversing

20. Regression Testing Execution Trace
For a test case t = < n , i, s(i) > , the execution trace for t on P, ET (P(i)), is the sequence of statements < s1, s2, ……. Sn > in P that are executed when P is run on input i, listed in the order in which they are executed.
We assume that in any trace ET(P(i)), the first statement is unique entry statement, and the final statement is a unique exit statement.

21. Regression Testing Regression test selection algorithm
There are two varities of test case prioritization techniques
General test case prioritization
Version specific test case prioritization

22. Regression Testing General test case prioritization
For a given program P and test suite T, we prioritize the test cases in T that will be useful over a succession of subsequent modified version of P without any knowledge of modified version.
Version specific test case prioritization
We prioritize the test cases in T, when P is modified to , with the knowledge of the changes that have been made in P.
Objective (Version Test Case Prioritization)
Execute the modified lines of code with minimum number of test cases.
We consider a hypothetical program of 60 lines of code. There are 10 code coverage based test cases.

23. Regression Testing Execution history is given below Test Case Array T1
1, 2, 20, 30, 40, 50 T2
1, 3, 4, 21, 31, 41, 51 T3
5, 6, 7, 8, 22, 32, ,42, 52 T4
6, 9, 10, 23, 24, 33, 43, 54 T5
5, 9, 11, 12, 13, 14, 15, 20, 29, 37, 38, 39 T6
15, 16, 17, 18, 19, 23, 24, 25, 34, 35, 36 T7
26, 27, 28, 40, 41, 44, 45, 46 T8
46, 47, 48, 49, 50, 53, 55 T9
55, 56, 57, 58, 59
T10
3, 4, 60
Test Case Array

24. Regression Testing Suppose 1,2,5,15,35,45 and 55 Total : 8
are modified.
Simplest is to execute all test cases that have any of the modified lines of code.
Selected test cases :
T1, T2, T3, T5, T6, T7, T8 and T9
Total : 8

25. Regression Testing Test cases Modified lines No. of matches T1 1, 2 25 1 T4 -- T5
5, 15 2 T6
15, 35 2 T7 45 1 T8 55 1 T9 55 1
T10 --

26. Regression Testing
Number of matches found (NFOUND) is stored in an array and elements are stored in descending order.
The test case that has the maximum number of matches is selected by making its candidate value = 1. This test case will be executed first & is shown on next slide.

27. Test cases in descending order
Regression Testing
Test cases
No. of matches (nfound)
Matches found
Candidate T1 2
1, 2 1 T5 2
5, 15 T6 2
15, 35 T2 1 1 T3 1 5 T7 1 45 T8 1 55 T9 1 55
Test cases in descending order

28. Regression Testing
T1 is selected for execution and it covers line nos. 1 and 2
LOCs that are still to be executed:
[ 1, 2, 5, 15, 35, 45, 55 ] – [ 1, 2 ]
= [5, 15, 35, 45, 55]
Again, we check for the number of modified lines of code covered by each test case and sort them in descending order & select the one with maximum number of matches.

29. Regression Testing Test cases No. of matches (nfound) Matches found
Candidate T5 2
5, 15 1 T6 2
15, 35 T3 1 5 T7 1 45 T8 1 55 T9 1 55

30. Regression Testing
T5 is selected for execution and it covers line nos. 5 and 15.
LOCs that are still to be executed:
[ 5, 15, 35, 45, 55 ] – [ 5, 15 ]
= [35, 45, 55]
Again, we check for the number of modified lines of code covered by each test case and sort them in descending order & select the test case with maximum number of matches found.

31. Regression Testing Test cases No. of matches (nfound) Matches found
Candidate T6 1 35 1 T7 1 45 T8 1 55 T9 1 55

32. LOCs still to be executed : [ 45, 55 ]
Regression Testing
T6 is selected and covers line no. 35.
LOCs still to be executed : [ 45, 55 ]
Test cases
No. of matches (nfound)
Matches found
Candidate T7 1 45 1 T8 1 55 T9 1 55

33. LOCs still to be executed : [ 55 ]
Regression Testing
T7 is selected for execution covering line no. 45.
LOCs still to be executed : [ 55 ]
Test cases
No. of matches (nfound)
Matches found
Candidate T8 1 55 1 T9 1 55
T8 is selected for execution
Order : T1, T5, T6, T7, T8

34. Regression Testing Second Case
Out of 10 test cases, we need to run only 5 test cases for 100% code coverage of modified lines of code.
50% saving of test case
Second Case
Lines of code have been modified and also deleted.
Example has 20 lines of code. 5 Test Cases as given on next slide.

35. Regression Testing Execution History Test Cases
1, 5, 7, 15, 20 T2
2, 3, 4, 5, 8, 16, 20 T3
6, 8, 9,10, 11, 12, 13, 14, 17, 18 T4
1, 2, 5, 8, 17, 19 T5
1, 2, 6, 8, 9, 13
Modified lines : 6, 13, 17, 20
Deleted lines : 4, 7, 15

36. Regression Testing Execution History
From the test history information, remove the deleted lines say : 4,7,15.
Test case array will be:
Test Cases
Execution History T1
1, 5, 20 T2
2, 3, 5, 8, 16, 20 T3
6, 8, 9,10, 11, 12, 13, 14, 17, 18 T4
1, 2, 5, 8, 17, 19 T5
1, 2, 6, 8, 9, 13

37. Regression Testing
Now, we want to find redundant test case. This situation has become important due to deletion of few lines.,
If we see execution history carefully. We come to know that T1 & T5 are redundant test cases.

38. Regression Testing Test cases Line numbers Found in test case
Redundant Y/N T1 1 T4 Y 5 T2 Y 20 T2 Y T5 1 T4 Y 2 T2 Y 6 T3 Y 8 T3 Y 9 T3 Y 13 T3 Y

39. The remaining test cases are = T2, T3, T4
Regression Testing
The remaining test cases are = T2, T3, T4
Test case array is:
Test Cases
Execution History T2
2, 3, 5, 8, 16, 20 T3
6, 8, 9, 10, 11, 12, 13, 14, 17, 18 T4
1, 2, 5, 8, 17, 19

40. No. of matches found (nfound)
Regression Testing
Number of matches for modified lines are
Modified lines are : 6, 13, 17, 20
Test cases
Matches found
No. of matches found (nfound) T2 20 1 T3
6, 13, 17 3 T4 17 1

41. Regression Testing
Test cases are sorted on the basis of matches found (descending order)
Test cases
No. of matches found
Line numbers
Candidate T3 3
6, 13, 17 1 T2 1 20 T4 1 17

42. Regression Testing
T3 is selected for execution. Remaining modified line is : 20
Hence T3 and T2 are sufficient to execute modified lines.
T1 & T5 should be deleted from the test suite.
Saving : 60%