1. The State of Hybrid Model-Based Testing Michiel van Osch 21-06-2005

2. 22 Index Introduction Testing a Water Tank Controller Hybrid Model-Based Testing –Conformance Relation –Test-case generation Work in Progress Questions

3. 33 Model-Based Testing Discrete Model Test-tool Discrete System pass !“red” ?“stop traffic”

4. 44 Hybrid Systems ABS “brake” Hydrolic pressure “1”, “2”, “1” Coffee with Milk and Sugar Room Temperature Desired Temperature: 25 o

5. 55 Hybrid Model-based Testing Hybrid-Model Test-tool Hybrid System pass Lens temperature Lens temperature !”open Cooling Valve” !”open Cooling Valve”

6. 66 Challenges Theory: –Develop Conformance Concepts –Develop Test-case Generation Algorithms –Prove Correctness and Applicability of Theory Practice: –Implement Tooling –Exactly Model Relevant Aspects –Connect Models, Tools and Systems

7. 77 Index Introduction Testing a Water Tank Controller Hybrid Model-Based Testing –Conformance Relation –Test-case generation Work in Progress Questions

8. 88 The water Tank Controller Tank Controller Discrete input: Set point Discrete output: communication Continuous input: Temperature (from sensor) Continuous output: Heat (from Heater) Water Tank Tank Controller Temperature Sensor Heater set point messages

9. 99 Specification (1) Hybrid I/O automaton Tank Controller Discrete input: Set_setpoint(sp) Discrete output: Print(“warning”) Continuous input: S T Continuous output: H

10. 10 Specification (2) Heater OFFHeater ON H H STST STST inputoutputinputoutput

11. 11 SpecificationTest-toolImplementation Test Setup Tank Controller Temperature Sensor Heater Set_Setpoint(sp)Set-point STST Temp. Sens. Print(..)Message HeatH

12. 12 Specification Test-tool Implementation Testing... Tank Controller Temperature Sensor Heater STST H Heat Temp. Sens. SpecificationImplementation

13. 13 Testing… Test-tool Set_setpoint(SP)Set-point Specification Implementation Tank Controller Temperature Sensor Heater STST H Heat Temp. Sens. SpecificationImplementation

14. 14 STST H Heat Temp. Sens. Testing… Test-tool Specification Implementation Tank Controller Temperature Sensor Heater SpecificationImplementation

15. 15 Testing Test-tool Print(“warning”)“error” Specification Implementation Tank Controller Temperature Sensor Heater STST H Heat Temp. Sens. SpecificationImplementation fail

16. 16 Testing… Test-tool Specification Implementation Tank Controller Temperature Sensor Heater fail STST H Heat Temp. Sens. SpecificationImplementation

17. 17 Index Introduction Testing a Water Tank Controller Hybrid Model-Based Testing –Conformance Relation –Test-case generation Work in Progress Questions

18. 18 Conformance A hybrid implementation is input-output conform a hybrid specification if for all observable traces of the specification: The set of possible output actions and trajectories, after the execution of a trace on the implementation, is a subset of the set of possible output actions or trajectories, after the execution of a trace on the specification.

19. 19 Some Decisions Defined on Hybrid I/O Automata –Input Action Enabled –Input Trajectory Enabled –Non-zeno State Based: possible output actions and trajectories in a certain state Input and output trajectories are treated together There is no quiescence observation (as of yet)

20. 20 Conformance relation We Formally Define: A hybrid trace  =  0 a 1  1 a 2 … as a sequence of discrete actions and trajectories of continuous variables (alternating) A after  : the set of states in which A can be after executing trace  out(q) : the set of output actions or trajectories possible in state q Out(Q) : the set of output actions or trajectories possible in the set of states Q Then implementation HIOA i is hioco specification HIOA s if: i hioco s if   traces(s) : out(i after  )  out(s after  )

21. 21 Examples !a !b !a !b !c Correct! implementationspecification implementation qq specification Correct!

22. 22 Examples !a !b !a !b Incorrect! implementationspecification !c implementation q specification Incorrect! q

23. 23 Test-Case generation & Execution Input actions: Select and stimulate Output actions: Observe and validate Trajectories: 1.Select a time up to which to apply input 2.Select an input trajectory to use for stimulation 3.Stimulate and observe output variables : –Until the end of the trajectory, without observing a mismatch between observed and specified output trajectories, in this case continue –Until an output action is observed, in this case validate the output action and if correct, then continue –Until a mismatch between observed and specified output variables occurs As long as verdict “fail” is given, the verdict “pass” can be given

24. 24 Test-case generation fail pass fail ?Set_setpoint(SP) !Print(“warning”) !Print(“error”) {SP > S T }

25. 25 Index Introduction Testing a Water Tank Controller Hybrid Model-Based Testing –Conformance Relation –Test-case generation Work in Progress Questions

26. 26 Work in Progress (1) Does a continuous variable always have a value? State Changes During a Trajectory “1”,”2” Coffee temperature When no coffee comes out of the machine, temperature Is not continuously 0! While applying an input trajectory and observing the output, States change qq’

27. 27 Work in Progress (2) Quiescence –Is quiescence a property on the discrete actions of the system or also on continuous behavior of the system? Notation What does this mean?

28. 28 Current & Future Work Further Develop these Concepts (Formalization, Theorems, Proofs) Make Concepts Workable in Practice –Sampling of continuous behavior –Selecting Realistic Input Trajectories –Deal with Uncountable Many Possible Input and Output Trajectories and States –Deal with Inaccuracy of Observed Output Implement Proof of Concept

29. 29 Questions