Software testing techniques Software testing techniques Input Output Analysis Presentation on the seminar Kaunas University of Technology

I/O analysis Black-Box Tests Reduction method Determines Inputs and Outputs Search for relations between input and output Goal is to reduse a combinatorial test suite by removing tests that are repetitive from the perspective of the program outputs Input Output Analysis2

Why IO analysis is used? Testing from the black-box perspective generates a large number of tests (e.g. AT&T Definity PBX > 3 million tests) Test reduction done randomly or based on „engineering judgment“ may lead to ineffective or redundant tests Input Output Analysis3

Combinatorial Testing For programs with multiple inputs we have to decide how to test different combinations of input sets. The most thorough approach is to test all combinations - this is quite often impossible due to the large number of tests Input Output Analysis4

Combinatorial „disaster“ If n = 10 and 10 test data values are selected for each variable, the total number of combinatorial test cases is X 1 X 2 X 3... X n Program P Input Output Analysis5

Finding I/O Relationships  Information may be gathered in a variety of ways (e.g. from specifications, code)  Manual collection of information difficult and time consuming  Automated collection possible through:  Static Analysis  Dynamic Analysis  Execution-oriented Analysis Input Output Analysis6

I/O analysis techniques  Static techniques analyze a program’s source code  Dynamic techniques analyze a program’s execution trace across a set of test data - dynamic dependencies used to determine IO relationships  Execution-oriented techniques manipulate individual inputs across multiple program executions in a test to determine IO relationships Input Output Analysis7

Example A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z Number of Combinatorial Tests: 2 * 4 * 2 = Input Output Analysis8

Relationships A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z Input Output Analysis9

Results Input Output Analysis10 T(W) (1, N, TDC) (1, N, BDM) (3, N, TDC) (3, N, BDM) T(Z) (1, N, TDC) (1, S, TDC) (1, E, TDC) (1, W, TDC) A={1, 3} B={N,S,E,W} C={TDC, BDM} W Z T(W)  T(Z) (1, N, TDC) (1, N, BDM) (3, N, TDC) (3, N, BDM) (1, S, TDC) (1, E, TDC) (1, W, TDC)

Optimization Input Output Analysis11 We have to create the smallest possible test set which covers all data combinations in the individual output test sets Toptimized (1, N, TDC) (1, S, BDM) (3, E, TDC) (3, W, BDM) T(W) (1, X, TDC) (1, X, BDM) (3, X, TDC) (3, X, BDM) T(Z) (X, N, DC) (X, S, DC) (X, E, DC) (X, W, DC) X – element, which have no influence

Summary Input Output analysis is for reducing test suite Saves test team`s time Helps to make better software Input Output Analysis12