1. Jüri Vain Dept. of Computer Science Tallinn University of Technology J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 1

2. 2 Vain, Jüri; Kääramees, Marko; Markvardt, Maili Online testing of nondeterministic systems with reactive planning tester. In: Petre, L.; Sere, K.; Troubitsyna, E. (Eds.). Dependability and Computer Engineering : Concepts for Software-Intensive Systems. Hershey, PA: IGI Global (2012), pages 113-150.

3. Outline  Preliminaries Model-Based Testing Online testing  Reactive Planning Tester (RPT)  Synthesis of the RPT  Performance issues  Test execution environment dTron  Demo J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 3

4. What is testing for?  to check the quality (functionality, reliability, performance, …) of an (software) object -by performing experiments -in a controlled way In avg. 10-20 errors per 1000 LOC 30-50 % of development time and cost in software 4 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

5. What is a Test? Software under Test (SUT) Test Data Output Test Cases Correct result? Oracle 5 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

6. 6 Model- Based Testing (typically)

7. Specifics of testing embedded systems J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 7 – non-determinism – partial observability – RT constraints – dependability  test coverage issues How to address them in testing? Real environment in the loop!

8. Option 1: Offline vs Online testing  Offline testing Open “control” loop Test is not adaptive to outputs or timing of SUT Test planning – result analysis loop is long  Online testing Flexible, test control is based on SUT feedback One test covers usually many test cases J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 8

9. Online testing  Group of test generation and execution algorithms that compute successive stimuli at runtime directed by  the test purpose and  the observed outputs of the SUT J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 9 TesterSUT

10. Online testing  Advantages: The state-space explosion problem is reduced because only a limited part of the state-space needs to be kept track of at any point in time.  Drawbacks: Exhaustive planning is diffcult due to the limitations of the available computational resources at the time of test execution. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 10

11. Option 2: Test scripting vs model-based generation J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 11  Scripting:  does not need SUT modelling+  sensitive to human errors-  inflexible, needs rewriting even with small changes of SUT specs-  hard to achieve test coverage-  Model-based generation:  considerable SUT modelling effort -  correctness of tests is verifiable+  easy to modify and regenerate+  clear characteristics of coverage+

12. Model-Based Testing  Given Model of the SUT specification System Under Test (SUT), The test goal (in terms of spec. model elements)  Find If the SUT conforms to the specification in terms expressed in the test goal. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 NEEDED! - Sufficiently rich modelling formalism, - Supporting tool set. 12

13. Model-Based Testing  We assume formal specs as: UML State Charts Extended Finite State Machines MSC OCL  etc. UPTA -Timed Automata (timing, parallelism, test data) J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 13

14. UPPAAL Tools www.uppaal.com  TIGA: Timed games (reachability and safety)  CORA: Cost Optimal reachability from priced TA  TRON: Testing Real-time Online Uppaal Model- checker: Efficient reachability analysis of network of timed automata (TA) Uppaal- CORA Uppaal- TIGA Uppaal- TRON … 14 www.Uppaal.com

15. Test Gene- rator tool click? x:=0 click? x<2 x>=2 DBLclick! input Online Model-Based Testing fail pass Test execution tool Test execution tool Adaptor Model Test Generator tool output Correctness Relation Selection & optimization Test generated and executed event-by-event (randomly) A.K.A on-the-fly testing Implementation Under Test input output 15 www.Uppaal.com

16. ”Relativized Real-Time i/o conformance” Relation Spec = UppAal Timed Automata Network: Env || IUT Timed Trace: i 1.2½.o 1.3.o 2.19.i 2. 5.i 3 Expected system reactionInput event ordering 16J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Online MBT architecture: UppAal-TRON [www.uppaal.com]

17. Bottleneck of online MBT: planning Spectrum of planning methods  Random walk (RW): select test stimuli in random inefficient - based on random exploration of the state space leads to test cases that are unreasonably long may leave the test purpose unachieved  RW with reinforcement learning (anti-ant) the exploration is guided by some reward function ........  Exploration with exhaustive planning MC provides possibly an optimal witness trace the size of the model is critical in explicit state MC state explosion in "combination lock" or deep loop models 17 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

18. Bottleneck of online MBT: planning Spectrum of planning methods  Random walk (RW): select test stimuli in random inefficient - based on random exploration of the state space leads to test cases that are unreasonably long may leave the test purpose unachieved  RW with reinforcement learning (anti-ant) the exploration is guided by reward function ........  Exploration with exhaustive planning MC provides possibly an optimal witness trace the size of the model is critical in explicit state MC state explosion in "combination lock" or deep loop models ??? 18 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Full space 1 step ahead Zero planning

19. Bottleneck of online MBT: planning Spectrum of planning methods  Random walk (RW): select test stimuli in random inefficient - random exploration of the state space test cases that are unreasonably long may leave the test purpose unachieved  RW with reinforcement learning (anti-ant) the exploration is guided by some reward function ........  Exploration with exhaustive planning MC provides possibly an optimal witness trace the size of the model is critical in explicit state MC state explosion in "combination lock" or deep loop models Planning with adaptive horizon! 19 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Full space 1 step ahead Zero planning

20. Reactive Planning in a Nutshell (1)  Instead of a complete plan, only a set of decision rules is derived  The rules direct the system towards the test goal.  Based on current situation evaluation just one subsequent input is computed at a time.  Planning horizon is adjusable J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 20

21. Reactive Planning: Planning cycle  Identify current state of SUT: Observe the output (or history) of the SUT  Pick the next move: Select one from unsatisfied test (sub-)goals G i  Compute the best strategy for G i : Gain function guides the exploration of the model (choose the transition with the shortest path to the (sub-)goal G i ) J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 21

22. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Reactive Planning Tester in a Nutshell (2) State model of the IUT Test goal Reactive Planning Tester (RPT) model Reachability Analysis Reactive Planning Tester (RPT) implementation SUT Offline phaseOnline phase Stimuli / responses RPT is another state model 22 RPT autonomously generates stimuli to reach the goal

23. Constructing the RPT model J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 23 Model of SUT Test goal Model of RPT Test strategy parameters: -planning horizon - timeouts Test strategy parameters: -planning horizon - timeouts Synthesis of RPT Extraction of the control strcture Constructing gain guards (GG) Constructing gain functions Reduction of GG

24. RPT: Key Assumptions  Testing is guided by the (EFSM) model of the tester and the test goal.  Decision rules of reactive planning are encoded in the guards of the transitions of the tester model.  The SUT model is presented as an output observable nondeterministic EFSM in which all paths are feasible 1.  1 - A. Y. Duale and M. U. Uyar. A method enabling feasible conformance test sequence generation for EFSM models. IEEE Trans. Comput.,53(5):614–627,2004. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 24

25. Example: Nondeterministic FSM s2s2 s3s3 e 6 : i 6 /o 6 e 7 : i 7 /o 7 s1s1 e 0 : i 0 /o 0 e 1 : i 0 /o 1 e 2 : i 2 /o 2 e 3 : i 3 /o 3 e 4 : i 3 /o 4 e 5 : i 5 /o 5 i 0 and i 3 are output observable nondeterministic inputs J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 25

26. Encoding the Test Goal in SUT Model  Trap - a boolean variable assignment attached to the transitions of the IUT model  A trap variable is initially set to false.  The trap update functions are executed (set to true) when the transition is visited. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 26

27. Goal directed testing: The power of traps Several goals can be expressed all/selected transitions transition sequences (traps with reference to other traps) advanced goals using auxiliary variables, consequent transitions, repeated pass, … traps with 1st order predicates data variables Properties not expressible by traps Assertions/invariants – always/never properties The model specifies only the allowed behaviours Åbo Akademi University, Dec 2011

28. Add Test Purpose s2s2 s3s3 e 6 : i 6 /o 6, t 6 =true e 7 : i 7 /o 7, t 7 =true s1s1 e 0 : i 0 /o 0 t 0 =true e 1 : i 0 /o 1, t 1 =true e 2 : i 2 /o 2, t 2 =true e 3 : i 3 /o 3, t 3 =true e 4 : i 3 /o 4, t 4 =true e 5 : i 5 /o 5, t 5 =true bool t 0 = false;... bool t 7 = false; J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 28

29. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Example: Adding the Test Goal s2s2 s3s3 e 6 : i 6 /o 6, t 6 =true e 7 : i 7 /o 7, t 7 =true s1s1 s1s1 e 0 : i 0 /o 0 t 0 =true e 1 : i 0 /o 1, t 1 =true e 2 : i 2 /o 2, t 2 =true e 3 : i 3 /o 3, t 3 =true e 4 : i 3 /o 4, t 4 =true e 5 : i 5 /o 5, t 5 =true Initial values: bool t 0 = false;... bool t 7 = false; Trap variables are initially set to false Trap update functions are executed (set to true) when the transition is visited Test goal is defined by trap variables t i attached to transitions 29

30. Model of the tester  Generated from the SUT model decorated with test purpose  Transition guards encode the rules of online planning  2 types of tester states: active – tester controls the next move passive – SUT controls the next move  2 types of transitions: Observable – source state is a passive state (guard  true), Controllable – source state is an active state (guard  p S /\ p T where p S – guard of the SUT transition; p T – gain guard) The gain guard (defined on trap variables) must ensure that only the outgoing edges with maximum gain are enabled in the given state. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 30

31. Construction of the Tester Sceleton s2s2 s3s3 e 6 : i 6 /o 6, t 6 =true e 7 : i 7 /o 7, t 7 =true s1s1 e 0 : i 0 /o 0 t 0 =true e 1 : i 0 /o 1, t 1 =true e 2 : i 2 /o 2, t 2 =true e 3 : i 3 /o 3, t 3 =true e 4 : i 3 /o 4, t 4 =true e 5 : i 5 /o 5, t 5 =true s1s1 sasa s2s2 sbsb scsc s3s3 sdsd sese sfsf J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 States:Transitions: - active - observable - passive - controllable 31

32. Add IO and Gain Guards s1s1 s4s4 s2s2 s5s5 s6s6 s3s3 s6s6 s9s9 s7s7 e o 0 : o 0 /t 0 =true e o 1 : o 1 /t 1 =true e c 01 : [p c 01 (T)] -/i 0 e o 2 : o 2 /t 2 =true e o 7 : o 7 /t 7 =true e c 2 : [p c 2 (T)] -/i 2 e c 34 : [p c 34 (T)] -/i 3 e o 3 : o 3 /t 3 =true e o 6 : o 6 /t 6 =true e c 6 : [p c 6 (T)] -/i 6 e c 7 : [p c 7 (T)] -/i 7 e c 5 : [p c 5 (T)] -/i 5 e o 5 : o 5 /t 5 =true J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 32

33. Constructing the gain guards (GG): intuition  GG must guarantee that each transition enabled by GG is a prefix of some locally optimal (w.r.t. test purpose) path; tester should terminate after the test goal is reached or all unvisited traps are unreachable from the current state; to have a quantitative measure of the gain of executing any transition e we define a gain function g e that returns a distance weighted sum of unsatisfied traps that are reachable along e. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 33

34. Constructing Gain Guards: intuition J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Observed current state Alternative choices... tr i tr j tr k... Planning cones to be covered for decision making 34

35. Constructing Gain Guards: intuition J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 tr i tr j tr k Alternative continuation choices... g e (tr i,T )... g e (tr j,T ) Gain functions characterize planning options 35 Current state

36. Constructing the gain guards: properties of gain functions  g e = 0, if it is useless to fire the transition e from the current state with the current variable bindings;  g e > 0, if fireing the transition e from the current state with the current variable bindings visits or leads closer to at least one unvisited trap;  g ei > g ej for transitions e i and e j with the same source state, if taking the transition e i leads to unvisited traps with smaller distance than taking the transition e j ;  Having gain function g e with given properties define GG: p T  (g e = max k g ek )  g e > 0 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 36

37. Constructing the Gain Functions: shortest path trees  Reachability problem of trap labelled transitions can be reduced to single-source shortest path problem 1.  Arguments of the gain function g e are then the shortest path tree TR e with root node e V T – vector of trap variables  To construct TR e we create a dual graph G = (V D,E D ) of the tester control graph M T where the vertices V D of G correspond to the transitions of the M T, the edges E D of G represent the pairs of consequtive transitions sharing a state in M T (2-switches) 1 - Fredman & Tarjan 1987O(E + V log V)Fredman & Tarjan 1987 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 37

38. Constructing the Gain Guards: shortest path tree (example) The dual graph of the tester modelThe shortest-paths tree (left) and the reduced shortest-paths tree (right) from the transition e c 01 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 38

39. Constructing the gain guards: gain function (1)  Represent the reduced tree TR(e i, G) as a set of elementary sub-trees each specified by the production i  j{1,..n} j  Rewrite the right-hand sides of the productions as arithmetic terms: (3) t i - trap variable t i lifted to type N, c - constant for rescaling the numerical value of the gain function, d( 0, i ) the distance between vertices 0 and i, where l - the number of hyper-edges on the path between 0 and i w j – weight of j-th hyperedge J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 39

40. Constructing the gain guards: gain function (2)  For each symbol i denoting a leaf vertex in TR(e,G) define a production rule (4)  Apply the production rules (3) and (4) starting from the root symbol 0 of TR(e,G) until all non- terminal symbols i are substituted with the terms that include only terminal symbols t i and d( 0, i ) J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 40

41. Example: Gain Functions J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 41

42. Example: Gain Guards J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 42

43. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Example: Construction of the Tester Structure s2s2 s3s3 e 6 : i 6 /o 6, t 6 =true e 7 : i 7 /o 7, t 7 =true s1s1 e 0 : i 0 /o 0 t 0 =true e 1 : i 0 /o 1, t 1 =true e 2 : i 2 /o 2, t 2 =true e 3 : i 3 /o 3, t 3 =true e 4 : i 3 /o 4, t 4 =true e 5 : i 5 /o 5, t 5 =true s1s1 s4s4 s2s2 s5s5 s6s6 s3s3 s6s6 s9s9 s7s7 e o 0 : o 0 /t 0 =true e o 1 : o 1 /t 1 =true e c 01 : [p c 01 (T)] -/i 0 e o 2 : o 2 /t 2 =true e o 7 : o 7 /t 7 =true e c 2 : [p c 2 (T)] -/i 2 e c 34 : [p c 34 (T)] -/i 3 e o 3 : o 3 /t 3 =true e o 6 : o 6 /t 6 =true e c 6 : [p c 6 (T)] -/i 6 e c 7 : [p c 7 (T)] -/i 7 e c 5 : [p c 5 (T)] -/i 5 e o 5 : o 5 /t 5 =true e o 4 : o 4 /t 4 =true The gain guards guarantee that only the outgoing edges with maximum gain are enabled in the given state 43

44. Complexity of constructing and running the tester  The complexity of the synthesis of the reactive planning tester is determined by the complexity of constructing the gain functions.  For each gain function the cost of finding the TR e by breadth-first- search is O(|V D | + |E D |) [Cormen], where |V D | = |E T | - number of transitions of M T |E D | - number of transition pairs of M T (is bounded by |E S | 2 )  For all controllable transitions of the M T the upper bound of the complexity of the computations of the gain functions is O(|E S | 3 ).  At runtime each choice by the tester takes O(|E S | 2 ) arithmetic operations to evaluate the gain functions J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 44

45. Performance of reactive planning J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Test Goal: All Transitions Test Goal: Selected Transition 45

46. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Experiments: Case Study: Model of the IUT Model of Feeder Box Controller power management (31 states, 73 transitions) 46

47. Test environment of the FBCU J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 47

48. How to plan in large models? Test goal: all transitionsTest goal: single transition J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 48 Average lengths of the test sequences Adjustable planning horizon Dependancy between horizon and test length Horizon saturation point

49. Shaping RPT planning cones (i) J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 tr i tr j tr k Decision point Alternative choices In online-testing decision time must be strictly bounded!... g e (tr i,T )... g e (tr j,T ) Gain functions 49

50. Shaping RPT planning cones (ii) J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 tr i tr j tr k Decision point Alternative choices Decision time strictly bounded! Prune the cone!... g e (tr i,T )... g e (tr j,T ) Gain functions Horizon h Different ways for defining h, and ‘visibility’ depending on depth of tree. Fully visible Partially visible 50

51. Average lengths of test sequences in the experiments J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 Horizon All transitions test coverageSingle transition test coverage Planning with horizon anti-antrandom choiceanti-antrandom choice 018345 ± 531144595 ± 195502199 ± 9914928 ± 4455 118417 ± 400319725 ± 70172156 ± 11546656 ± 5447 25120 ± 16784935 ± 18751276 ± 5312516 ± 2263 34187 ± 9783610 ± 2538746 ± 5031632 ± 1745 42504 ± 8152077 ± 552821 ± 4211617 ± 1442 52261 ± 6121276 ± 426319 ± 233618 ± 512 62288 ± 4911172 ± 387182 ± 116272 ± 188 71374 ± 346762 ± 177139 ± 74147 ± 125 8851 ± 304548 ± 165112 ± 75171 ± 114 9701 ± 240395 ± 8672 ± 25119 ± 129 10406 ± 102329 ± 5773 ± 29146 ± 194 11337 ± 72311 ± 5879 ± 3086 ± 59 12323 ± 61284 ± 3841 ± 1574 ± 51 13326 ± 64298 ± 4434 ± 848 ± 31 14335 ± 64295 ± 4034 ± 940 ± 23 15324 ± 59295 ± 4225 ± 426 ± 5 16332 ± 51291 ± 5223 ± 224 ± 3 17324 ± 59284 ± 3222 ± 221 ± 1 18326 ± 66307 ± 4721 ± 1 19319 ± 55287 ± 2921 ± 1 20319 ± 68305 ± 4321 ± 1 51

52. Average time spent for online planning of the next step J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 52

53. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 53  How to generate test data?

54. Example: RPT synthesis for INRES protocol J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 54 Model of the SUT local component

55. Example: RPT synthesis for INRES protocol Local SUT component model RPT model Timeout! Off! LowICONreq? DR? CC? ? AK? 55J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

56. Online planning constraints for the RPT J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 56

57. How to derive data constraints?  For all transitions t(s i,.) of state s i generate reduced reachability tree RRT i s.t. transition t(s i,.) is a root and the trap labeled transitions the terminal nodes of the RRT i.  Compute data constraint for each path  j of RRT i  use wp-algorithm (starting from trap node) for pairs of neigbbour traps of  j  unfold loops using gfp for termination  for constructing the gain function of  j record (when traversing  j ):  traps remaining on the path  j and  the lengths of inter-trap paths  construct the gain function for full path  j using trap-to-trap distances on that path and the vector of trap variables.  Global data constraint for the path is a conjunction of data constraints pairwise traps of  j 57J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

58. Summary  RPT synthesis technique relies on offline static analysis of the SUT model and test goals.  Efficiency of planning: Number of rules that have to be evaluated at each step is relatively small (i.e., = the number of outgoing transitions of a current state) The execution of decision rules is significantly faster than looking through all potential alternatives at runtime. Scalability supported by RRT pruning technique Provides test sequences that are lengthwise close to optimal. J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 58

59. DTRON: Distributed RPT execution (http://dijkstra.cs.ttu.ee/~aivo/dtron/usage.html) 59 SPREAD J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

60. || LTC communicating timed automata (TA) 60 TA model of local SUT component (extended with online planning constraints) TA model of the local test goal (property) MsMs MgMg || TA model of local SUT component (extended with online planning constraints) TA model of the local test goal (property) MsMs MgMg “TCI update” signals “Test coverage item (TCI) reached” signal TCI synchro LTC J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

61. 61 Tester components communicate over channels J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

62. DTRON: distibuted, heterogeneous components 62 SPREAD Selenium TestCast Z3 J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012

63. DTRON: dynamic test reconfiguration 63 SPREAD Selenium TestCast Selenium Model (Tester k )Model (Tester 1 )Model (Test Plan Supervisor) Tester component updates

64. DTRON: Visualization of test runs 64

65. Questions? Thank You! J.Vain “...Reactive Planning Testing...”Abo, Feb 3, 2012 65