1. Dependence Analysis in Reduction of Requirement Based Test Suites Boris Vaysburg Luay Tahat Bogdan Korel Computer Science Department Bell Labs Innovations Computer Science Department Illinois Institute of Technology Lucent Technologies Illinois Institute of Technology Chicago, IL USA Naperville, IL USA Chicago, IL USA

2. Outline Model-based testing Selective model-based testing Selective test suite reduction using EFSM analysis EFSM dependencies Test Reduction based on EFSM analysis  Static approach  Dynamic approach Conclusions

3. Model Based Testing Model based testing is used to test software systems based on a system model Test cases are generated from the system model Testing state-based software systems

4. Model Based Testing Systems can be modeled using:  EFSM: Extended Finite State Machine  SDL: Specification Description Language

5. Model Based Testing Systems can be modeled using:  EFSM: Extended Finite State Machine  SDL: Specification Description Language

6. Extended Finite State Machine EFSM EFSM consists of:  States  Transitions The following elements are associated with each transition:  Event  Enabling condition  Action(s)

7. EFSM Transition State 1 State 2 Event(p)[Condition]/Action

8. EFSM Model of the ATM System

9. Model Based Testing Strategies State coverage Transition coverage Path coverage Constrained path coverage: each transition can be traversed at most “n” times in a test

10. Model-Based Testing These strategies are used to test the whole system Model-based testing strategies may generate large test suites

11. Selective Model-Based Testing Testers are frequently interested in testing only selected functionality (requirement) In a system model requirements frequently correspond to transitions The problem of selective testing of a requirement can be converted to a problem of testing a transition in the model (selective testing of a transition)

12. Selective Model-Based Testing

14. Selective Testing Strategies with respect to a transition State coverage Transition coverage Path coverage Constrained path coverage Each test must contain a transition (requirement) under test

15. Selective Model-Based Testing The size of a selective model-based test suite is frequently significantly smaller than the size of a complete system test suite However, the size of selective test suites still may be large

16. Reduction of Selective Test Suites The goal is to develop an approach of reduction of selective test suites EFSM dependence analysis is used to reduce the size of selective test suites

17. Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

18. Test Suite Reduction Interactions? Dependencies between EFSM elements.

19. EFSM dependence analysis Dependencies between transitions (“active” elements of EFSM) Two types of dependencies between transitions in the EFSM are identified:  Data dependence  Control dependence

20. Data Dependence Data dependence captures the notion that one transition defines a value to a variable and another transition may potentially use this value.

21. Data Dependence

22. Control Dependence Control dependence captures the notion that one transition may affect traversal of another transition It is defined based on the concept of post- dominance

23. Control Dependence

25. EFSM Dependence Graph Nodes represent transitions Arcs represent control and data dependencies

26. Static EFSM Dependence Graph T5T6T8T1T4T9T2T3T7 data dependence control dependence

27. Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

28. Test suite reduction Interactions: Data dependencies Control dependencies Pattern of interactions: Dependence sub-graph  Static interaction pattern  Dynamic interaction pattern

29. Test #1: T1, T4, T6, T8, T6, T8, T5, T8, T9 Test #2: T1, T2, T4, T6, T8, T6, T8, T5, T8, T9

30. Selective test suite reduction Static Interaction Pattern All dependencies that occurred during traversal of the test (path) are mapped to the EFSM dependence sub-graph Dependencies that do not “influence” a transition under test are removed The resulting dependence sub-graph is called Static Interaction Pattern If two tests have the same Static Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

31. T1 T4 T6 T8 T6 T8 T5 T8 T9 Test #1:

32. Dependence Sub-graph T5T6T8T1T4T9

33. Dependence Sub-graph T5 T6T8T1T4T9

34. Dependence Sub-graph T5 T6T8T1T4T9

35. Static Interaction Pattern for Test #1 T5 T6T1T4

36. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Test #2:

37. Dependence Sub-graph T5T6T8T1T4T9T2

38. Dependence Sub-graph T5 T6T8T1T4T9T2

39. Dependence Sub-graph T5 T6T8T1T4T9T2

40. Static Interaction Pattern for Test #2 T5 T6T1T4 Test #1 and Test #2 have the same Static Interaction Patterns

41. Test suite reduction Significant test reduction can be achieved using Static Interaction Patterns

42. Test Suite Size Reduction n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern 218 3 31007 445014 5 1,806 16 6 6,762 16 7 24,300 16

43. Selective test suite reduction Test reduction using Static Interaction Patterns is relatively “coarse” Static Interaction Patterns are not able to capture intricate interactions Dynamic Interaction Pattern is used

44. Dynamic EFSM dependencies Dynamic dependencies are dependencies that occur during traversal of the test (path) Dynamic EFSM dependence graph

45. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependencies for Test #2 Dynamic EFSM Dependence Graph

46. Selective test suite reduction Dynamic Interaction Pattern All dependencies that occurred during traversal of the test (path) are used to create a dynamic EFSM dependence graph. Dependencies that do not “influence” a transition under test are removed. The resulting dependence sub-graph is called Dynamic Interaction Pattern. If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite.

47. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

48. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

49. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence graph for Test #2

50. T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic Interaction Pattern for Test #2 If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

51. Selective Test Suite Size Reductions n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern Size of Reduced Test Suite Using Dynamic Interaction Pattern 2 1833 310077 4 4501415 51,8061631 66,7621663 724,30016127

52. Conclusions Using EFSM dependence analysis to reduce selective test suites Significant reduction may be achieved Fault detection capability of the reduced test suite? The approach is under development We plan to perform an experimental study Develop efficient algorithms of test suite reduction

53. End of Presentation