1. Multi-Paradigm Models as Source for Automatic Test Construction Victor Kuliamin ISP RAS, Moscow

2. Why Multiple Models? Requirement s FunctionalityReliabilityEfficiencyUsability Testing ?

3. Modeling Techniques Operational Can be executed by virtual machine Contract Pre- and postconditions, data integrity constraints History-based Constraints on possible traces Algebraic Equivalence between different execution histories (C)(E)FSM, LTS, PN, CSP, ASM SDL, LOTOS, Lustre, VDM, Murphi, Simulink Z, B, ADL, JML, Eiffel, VDM, RSL Larch-C++ TL, MSC Larch, ML, OBJ

4. Tasks of Testing Software under Test Evaluate Correctness Organize Bundle of Test Inputs Construct Single Test Input Evaluate Testing Quality Test Results Transform Test Inputs and Responses Gather Responses

5. Modeling Techniques Comparison Operational Contract History-based Algebraic Behavior EvaluationCloseness to Requirements Low-level Coverage High-level Coverage Test Sequence Construction Single Input Construction Scalability Concurrency

6. Comparison Results There is no the best technique No one technique is good for everything May be a mix of different approaches can fit more needs?

7. UniTesK Technology Model-based testing technology Developed in 2000 – 2002 in ISP RAS

8. UniTesK Solutions Contract specifications of behavior FSM and LTS testing models

9. Contract Specifications Preconditions and postconditions of interface operations and asynchronous reactions Data integrity constraints Close to requirements Suitable for oracle generation Provide low-level coverage criteria Functional Requirements Contract Specifications

10. FSM and LTS Testing Models Define states and admissible input actions More abstract than original specifications Guarantee some low-level coverage Suitable for test sequence construction Provide high-level coverage criteria Contract Specifications ! Coverage Requirements ? ! ? ! ! ? ? ?

11. Relation between Models states parametersoperation domain 1 2 3 coverage goals

12. Whole Picture I Software under Test Model of Behavior Testing Model Coverage Model Test Oracle Test Sequence Construction

13. Whole Picture II Software under TestModel of BehaviorTesting Model Coverage Model Operation Data Event Operation prepost prepost Event prepost invariants Data model Operation State Calculation Scenario method

14. Tool Demo

15. Set of Integers – Scenario I 0 12 31 2 7 5 States of behavior model 3 5 States of FSM model

16. Mapping Abstract Call to Specific 1 2 3 current state parameters states

17. Set of Integers – Scenario II 0 12 31 2 7 5 States of FSM model = States of behavior model

18. Failure { -2147483648, 2147483647 } Add ( -715827883 ) / false

19. References 1. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI 2003. LNCS, Springer-Verlag, 2003. 2. V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite Architecture. Proc. of FME 2002. LNCS 2391, pp. 77-88, Springer-Verlag, 2002. 3. A. K. Petrenko, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Experiences in using testing tools and technology in real-life applications. Proceedings of SETT’01, India, Pune, 2001 4. I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in Program Testing. "Programmirovanije", 2000, No. 2 (in Russian). Programming and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73 (English version) 5. I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications. Proceedings of World Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp. 608- 621 6. http://www.ispras.ru/groups/rv/rv.html http://www.ispras.ru/groups/rv/rv.html

20. Contact Victor V. Kuliamin E-mail: kuliamin@ispras.ru 109004, B. Kommunisticheskaya, 25 Moscow, Russia Web: http://www.ispras.ru/groups/rv/rv.html Phone: 007-095-9125317 Fax: 007-095-9121524

21. Thank you!