1. Software testing techniques Software testing techniques Input Output Analysis Presentation on the seminar Kaunas University of Technology

2. I/O analysis Black-Box Tests Reduction method Determines Inputs and Outputs Search for relations between input and output Goal is to reduse a combinatorial test suite by removing tests that are repetitive from the perspective of the program outputs 2016.01.26Input Output Analysis2

3. Why IO analysis is used? Testing from the black-box perspective generates a large number of tests (e.g. AT&T Definity PBX > 3 million tests) Test reduction done randomly or based on „engineering judgment“ may lead to ineffective or redundant tests 2016.01.26Input Output Analysis3

4. Combinatorial Testing For programs with multiple inputs we have to decide how to test different combinations of input sets. The most thorough approach is to test all combinations - this is quite often impossible due to the large number of tests 2016.01.26Input Output Analysis4

5. Combinatorial „disaster“ If n = 10 and 10 test data values are selected for each variable, the total number of combinatorial test cases is 10 10 X 1 X 2 X 3... X n Program P 2016.01.26Input Output Analysis5

6. Finding I/O Relationships  Information may be gathered in a variety of ways (e.g. from specifications, code)  Manual collection of information difficult and time consuming  Automated collection possible through:  Static Analysis  Dynamic Analysis  Execution-oriented Analysis 2016.01.26Input Output Analysis6

7. I/O analysis techniques  Static techniques analyze a program’s source code  Dynamic techniques analyze a program’s execution trace across a set of test data - dynamic dependencies used to determine IO relationships  Execution-oriented techniques manipulate individual inputs across multiple program executions in a test to determine IO relationships 2016.01.26Input Output Analysis7

8. Example A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z Number of Combinatorial Tests: 2 * 4 * 2 = 16 2016.01.26Input Output Analysis8

9. Relationships A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z 2016.01.26Input Output Analysis9

10. Results 2016.01.26Input Output Analysis10 T(W) (1, N, TDC) (1, N, BDM) (3, N, TDC) (3, N, BDM) T(Z) (1, N, TDC) (1, S, TDC) (1, E, TDC) (1, W, TDC) A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z T(W)  T(Z) (1, N, TDC) (1, N, BDM) (3, N, TDC) (3, N, BDM) (1, S, TDC) (1, E, TDC) (1, W, TDC)

11. Optimization 2016.01.26Input Output Analysis11 We have to create the smallest possible test set which covers all data combinations in the individual output test sets Toptimized (1, N, TDC) (1, S, BDM) (3, E, TDC) (3, W, BDM) T(W) (1, X, TDC) (1, X, BDM) (3, X, TDC) (3, X, BDM) T(Z) (X, N, DC) (X, S, DC) (X, E, DC) (X, W, DC) X – element, which have no influence

12. Summary Input Output analysis is for reducing test suite Saves test team`s time Helps to make better software 2016.01.26Input Output Analysis12