Dependence Analysis in Reduction of Requirement Based Test Suites Boris Vaysburg Luay Tahat Bogdan Korel Computer Science Department Bell Labs Innovations Computer Science Department Illinois Institute of Technology Lucent Technologies Illinois Institute of Technology Chicago, IL USA Naperville, IL USA Chicago, IL USA

Outline Model-based testing Selective model-based testing Selective test suite reduction using EFSM analysis EFSM dependencies Test Reduction based on EFSM analysis  Static approach  Dynamic approach Conclusions

Model Based Testing Model based testing is used to test software systems based on a system model Test cases are generated from the system model Testing state-based software systems

Model Based Testing Systems can be modeled using:  EFSM: Extended Finite State Machine  SDL: Specification Description Language

Model Based Testing Systems can be modeled using:  EFSM: Extended Finite State Machine  SDL: Specification Description Language

Extended Finite State Machine EFSM EFSM consists of:  States  Transitions The following elements are associated with each transition:  Event  Enabling condition  Action(s)

EFSM Transition State 1 State 2 Event(p)[Condition]/Action

EFSM Model of the ATM System

Model Based Testing Strategies State coverage Transition coverage Path coverage Constrained path coverage: each transition can be traversed at most “n” times in a test

Model-Based Testing These strategies are used to test the whole system Model-based testing strategies may generate large test suites

Selective Model-Based Testing Testers are frequently interested in testing only selected functionality (requirement) In a system model requirements frequently correspond to transitions The problem of selective testing of a requirement can be converted to a problem of testing a transition in the model (selective testing of a transition)

Selective Model-Based Testing

Selective Testing Strategies with respect to a transition State coverage Transition coverage Path coverage Constrained path coverage Each test must contain a transition (requirement) under test

Selective Model-Based Testing The size of a selective model-based test suite is frequently significantly smaller than the size of a complete system test suite However, the size of selective test suites still may be large

Reduction of Selective Test Suites The goal is to develop an approach of reduction of selective test suites EFSM dependence analysis is used to reduce the size of selective test suites

Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

Test Suite Reduction Interactions? Dependencies between EFSM elements.

EFSM dependence analysis Dependencies between transitions (“active” elements of EFSM) Two types of dependencies between transitions in the EFSM are identified:  Data dependence  Control dependence

Data Dependence Data dependence captures the notion that one transition defines a value to a variable and another transition may potentially use this value.

Data Dependence

Control Dependence Control dependence captures the notion that one transition may affect traversal of another transition It is defined based on the concept of post- dominance

Control Dependence

EFSM Dependence Graph Nodes represent transitions Arcs represent control and data dependencies

Static EFSM Dependence Graph T5T6T8T1T4T9T2T3T7 data dependence control dependence

Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

Test suite reduction Interactions: Data dependencies Control dependencies Pattern of interactions: Dependence sub-graph  Static interaction pattern  Dynamic interaction pattern

Test #1: T1, T4, T6, T8, T6, T8, T5, T8, T9 Test #2: T1, T2, T4, T6, T8, T6, T8, T5, T8, T9

Selective test suite reduction Static Interaction Pattern All dependencies that occurred during traversal of the test (path) are mapped to the EFSM dependence sub-graph Dependencies that do not “influence” a transition under test are removed The resulting dependence sub-graph is called Static Interaction Pattern If two tests have the same Static Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

T1 T4 T6 T8 T6 T8 T5 T8 T9 Test #1:

Dependence Sub-graph T5T6T8T1T4T9

Dependence Sub-graph T5 T6T8T1T4T9

Dependence Sub-graph T5 T6T8T1T4T9

Static Interaction Pattern for Test #1 T5 T6T1T4

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Test #2:

Dependence Sub-graph T5T6T8T1T4T9T2

Dependence Sub-graph T5 T6T8T1T4T9T2

Dependence Sub-graph T5 T6T8T1T4T9T2

Static Interaction Pattern for Test #2 T5 T6T1T4 Test #1 and Test #2 have the same Static Interaction Patterns

Test suite reduction Significant test reduction can be achieved using Static Interaction Patterns

Test Suite Size Reduction n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern , , ,300 16

Selective test suite reduction Test reduction using Static Interaction Patterns is relatively “coarse” Static Interaction Patterns are not able to capture intricate interactions Dynamic Interaction Pattern is used

Dynamic EFSM dependencies Dynamic dependencies are dependencies that occur during traversal of the test (path) Dynamic EFSM dependence graph

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependencies for Test #2 Dynamic EFSM Dependence Graph

Selective test suite reduction Dynamic Interaction Pattern All dependencies that occurred during traversal of the test (path) are used to create a dynamic EFSM dependence graph. Dependencies that do not “influence” a transition under test are removed. The resulting dependence sub-graph is called Dynamic Interaction Pattern. If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite.

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic Interaction Pattern for Test #2 If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

Selective Test Suite Size Reductions n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern Size of Reduced Test Suite Using Dynamic Interaction Pattern , , ,

Conclusions Using EFSM dependence analysis to reduce selective test suites Significant reduction may be achieved Fault detection capability of the reduced test suite? The approach is under development We plan to perform an experimental study Develop efficient algorithms of test suite reduction

End of Presentation