Multi-Paradigm Models as Source for Automatic Test Construction Victor Kuliamin ISP RAS, Moscow

Why Multiple Models? Requirement s FunctionalityReliabilityEfficiencyUsability Testing ?

Modeling Techniques Operational Can be executed by virtual machine Contract Pre- and postconditions, data integrity constraints History-based Constraints on possible traces Algebraic Equivalence between different execution histories (C)(E)FSM, LTS, PN, CSP, ASM SDL, LOTOS, Lustre, VDM, Murphi, Simulink Z, B, ADL, JML, Eiffel, VDM, RSL Larch-C++ TL, MSC Larch, ML, OBJ

Tasks of Testing Software under Test Evaluate Correctness Organize Bundle of Test Inputs Construct Single Test Input Evaluate Testing Quality Test Results Transform Test Inputs and Responses Gather Responses

Modeling Techniques Comparison Operational Contract History-based Algebraic Behavior EvaluationCloseness to Requirements Low-level Coverage High-level Coverage Test Sequence Construction Single Input Construction Scalability Concurrency

Comparison Results There is no the best technique No one technique is good for everything May be a mix of different approaches can fit more needs?

UniTesK Technology Model-based testing technology Developed in 2000 – 2002 in ISP RAS

UniTesK Solutions Contract specifications of behavior FSM and LTS testing models

Contract Specifications Preconditions and postconditions of interface operations and asynchronous reactions Data integrity constraints Close to requirements Suitable for oracle generation Provide low-level coverage criteria Functional Requirements Contract Specifications

FSM and LTS Testing Models Define states and admissible input actions More abstract than original specifications Guarantee some low-level coverage Suitable for test sequence construction Provide high-level coverage criteria Contract Specifications ! Coverage Requirements ? ! ? ! ! ? ? ?

Relation between Models states parametersoperation domain coverage goals

Whole Picture I Software under Test Model of Behavior Testing Model Coverage Model Test Oracle Test Sequence Construction

Whole Picture II Software under TestModel of BehaviorTesting Model Coverage Model Operation Data Event Operation prepost prepost Event prepost invariants Data model Operation State Calculation Scenario method

Tool Demo

Set of Integers – Scenario I States of behavior model 3 5 States of FSM model

Mapping Abstract Call to Specific current state parameters states

Set of Integers – Scenario II States of FSM model = States of behavior model

Failure { , } Add ( ) / false

References 1. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI LNCS, Springer-Verlag, V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite Architecture. Proc. of FME LNCS 2391, pp , Springer-Verlag, A. K. Petrenko, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Experiences in using testing tools and technology in real-life applications. Proceedings of SETT’01, India, Pune, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in Program Testing. "Programmirovanije", 2000, No. 2 (in Russian). Programming and Computer Software, Vol. 26, No. 2, 2000, pp (English version) 5. I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications. Proceedings of World Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp

Contact Victor V. Kuliamin , B. Kommunisticheskaya, 25 Moscow, Russia Web: Phone: Fax:

Thank you!