1. Automated test pattern selection from requirements « Requested » Yves Le Traon Clémentine Nebut

2. Outline Objectives Objectives A contract language for requirements A contract language for requirements Test generation from enhanced use cases Test generation from enhanced use cases Results Results Conclusion Conclusion

3. Objectives Benefiting from the sequential dependencies of the use cases … Benefiting from the sequential dependencies of the use cases … Expressing the sequential dependencies: a contract language for the requirementsExpressing the sequential dependencies: a contract language for the requirements … To generate test objectives/test patterns under the form of sequences of use cases … To generate test objectives/test patterns under the form of sequences of use cases Expressing test covering criteriaExpressing test covering criteria

4. Global method Requirements (use cases and contracts) Execution model (UCTS) Interpreter Simulator Test objective generator Test criterion (functional) Test criterion (robustness) Test patterns and verdicts Test case generation « Mapper » Set of test cases and their verdicts Useful for model-checking or improving requirements

5. A contract language for requirements Based on Catalysis and Meyer’s «design by contract» approach Based on Catalysis and Meyer’s «design by contract» approach Contract = pre and post conditions Contract = pre and post conditions Dealing with parameters of the use case Dealing with parameters of the use case Easy to learn Easy to learn Making easy the expression of constraints and dependencies on use cases Making easy the expression of constraints and dependencies on use cases

6. Use case parameters Actors or Business concepts Actors or Business concepts Used to represent the inputs of the use case Used to represent the inputs of the use case Parameter type: enumerated (enumeration only needed for the simulation) Parameter type: enumerated (enumeration only needed for the simulation)

7. Contracts: logical expression on predicates Predicate = a name + set of typed parameters Predicate = a name + set of typed parameters Ex: created(m:meeting)Ex: created(m:meeting) manager(u:participant,m:meeting) manager(u:participant,m:meeting) Logical expressions: classical boolean operators (and, or, implies, not) + quantifiers (forall, exists) Logical expressions: classical boolean operators (and, or, implies, not) + quantifiers (forall, exists)

8. Contracts: Grammar BOOLEXPR -> DISJONCTION DISJONCTION -> CONJONCTION (or CONJONCTION)* CONJONCTION -> UNARYEXPR (and UNARYEXPR)* UNARYEXPR -> ( BOOLEXPR ) | NEGATION | FORALL | IMPLIES | EXISTS | PREDICATE | DIFF | EQUALITY PREDICATE -> IDENT ( IDENT (, IDENT)* ) | IDENT() EQUALITY -> IDENT = IDENT DIFF -> IDENT /= IDENT NEGATION -> not BOOLEXPR FORALL -> forall ( LISTFORMALPARAMS ) { BOOLEXPR } EXISTS -> exists ( LISTFORMALPARAMS ) { BOOLEXPR } IMPLIES -> { BOOLEXPR } implies { BOOLEXPR }

9. Contracts example:the virtual meeting case study

10. Contracts: example #use case OPEN UC open(u:participant;m:mtg) pre created(m) and moderator(u,m) and not closed(m) and not opened(m) and connected(u) post opened(m) #use case CLOSE UC close(u:participant; m:mtg) pre opened(m) and moderator(u,m) post not opened(m) and closed(m) and forall(v:participant) {not entered(v,m) and not asked(v,m) and not speaker(v,m) } OPEN(u1,m1);CLOSE(u1,m1) is a correct sequence

11. Test generation from enhanced use cases A use case transition system A use case transition system DefinitionDefinition Building algorithmBuilding algorithm Coverage criteria Coverage criteria 4 structural criteria4 structural criteria 1 semantical criterion1 semantical criterion Robustness tests Robustness tests

12. Use Case Transition System (UCTS) : definition Quadruple M=(Q,q 0,A,) Quadruple M=(Q,q 0,A,) Q = non empty set of statesQ = non empty set of states State = set of instantiated predicates State = set of instantiated predicates q 0 = initial stateq 0 = initial state A = alphabet of actionsA = alphabet of actions action= instantiated use case action= instantiated use case   Q x A x Q = transition function  Q x A x Q = transition function

13. UCTS: example 0 ->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), opened(m1)} 1->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), opened(m1), entered(p2, m1)} 2->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), opened(m1), entered(p1, m1)} 3->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), closed(m1)} 4->{moderator(p2, m1), connected(p1), connected(p2),created(m1), manager(p2, m1), opened(m1), entered(p2, m1), entered(p1, m1)} 5->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), opened(m1), entered(p2, m1), asked(p2, m1)} 6->{moderator(p2, m1), connected(p1), connected(p2), created(m1), manager(p2, m1), opened(m1), entered(p1, m1), asked(p1, m1)}

14. Building algorithm algorithmparam algorithm buildUCLTS param initState: STATE ; useCases : SET[ACTION] var var result : UCLTS to_visit : STACK[STATE] currentState : STATE newState : STATE init init result.initialState := initState to_visit.push(initState)body while (to_visit≠) { currentState := to_visit.pop ∀ uc in useCases | currentState ⇒ uc.pre do newState := apply(currentState, uc) if newState ∉ result then result.Q := result.Q ∪ {newState} to_visit.push(newState) fi  result.  :=result.  ∪ {(currentState,uc,newState)} } } }

15. Test criteria All edges criterion All edges criterion All vertices criterion All vertices criterion All instantiated use cases All instantiated use cases All instantiated use cases and all vertices All instantiated use cases and all vertices All precondition terms All precondition terms

16. Implementation of test criteria Breadth first search of the UCTS Breadth first search of the UCTS To obtain short test objectives To obtain short test objectives Prototype: UCTBuilder Prototype: UCTBuilder

17. Robustness tests Generate paths leading to an invalid application of the use case Generate paths leading to an invalid application of the use case Exercize correctly the system, then make a non specified action Exercize correctly the system, then make a non specified action Criterion: similar to all precondition term (all the terms that makes the precondition fail) Criterion: similar to all precondition term (all the terms that makes the precondition fail)

18. Demonstration of the Requested prototype

19. Is system testing efficient ? Dead code 9% Robustness code (w.r.t. env) 18% Nominal code 65% Robustness code (w.r.t.spec) 8%

20. Results Criterion# generated test objectives average size of the tests All edges1384111 All vertices76910 All instantiated UC505 AV-AIUC81910 All precondition terms 155

21. Results Code covered by functional test cases Code covered by robustness test cases % covered code

22. Results # covered satements # test cases # covered satements # test cases

23. Conclusion 2 relevant test criteria 2 relevant test criteria 1 relevant robustness test criterion 1 relevant robustness test criterion Test objectives generated automatically from enhenced use cases Test objectives generated automatically from enhenced use cases Prototype tool Prototype tool

24. Future tool support (in a close future) Integrated to Objecteering Integrated to Objecteering Using OCL expressions Using OCL expressions Map test objectives into test cases using scenarios (with contracts) Map test objectives into test cases using scenarios (with contracts)