1. System Design Research Laboratory Model-based Testing and Monitoring for Hybrid Embedded Systems Li Tan Jesung Kim Oleg Sokolsky Insup Lee University of Pennsylvania November 2004

2. System Design Research Laboratory Outline 1. Motivations and overview 2. Model-based testing. 1. Generating model-based testers. 3. Model-based monitoring 1. Synthesizing model-based monitors from logic specifications 2. Instrumenting models 4. The case study on SONY AIBO dog 1. Design-level validation 2. “on-board” validation 5. Conclusion

3. System Design Research Laboratory The Goal Goal: apply the state-of-the-art runtime verification technique to model-based embedded system design and validation domain. Requirements: Incorporating the rigid specification of the system properties using temporal logics. Using the existing techniques in model-based design to reduce the cost of migration. Supporting both design-level and implementation-level validations. The procedure can be fully automatically by a set of software tools.

4. System Design Research Laboratory Road Map V.Implementation Level Validation Code Generator Self-testing and Self-monitoring Code Model Simulator IV. Design Level Validation Self-testing and Self -monitoring Model Coverage Criteria/ environment constrains Tester model I. Create Model-based Tester System model Instrumented model P2C instrumentator mPEDL event definition II. Instrument the Model mMEDL specification M2C Synthesizer Monitor model III. Synthesize Model-based Monitor Five steps towards model-based self-testing and self-monitoring embedded systems

5. System Design Research Laboratory Embedded System Modeling with Hybrid Automata A hybrid automaton A ={S, V, T, G, W, D, I, s 0 } extends a EFSM {S, V, T, G, W, s 0 } with continuous behaviors 1. S is the set of modes 2. V is the set of variables 3. T=V £ V is the set of transitions 4. G assigns each t 2 T a guard, a predicate over V. 5. W assigns each t 2 T an assignment for V µ V. 6. D assigns each s 2 S a set of differential equations on V. 7. I assigns each s 2 S an invariant, a predicate over V. 8. s 0 2 S is the initial mode.

6. System Design Research Laboratory An example: Dog Head Controller. x  45? x  -45 |  |  10? |  | >10? x   : The visibility of the ball x: The angle of the head  : The angle of the ball Hybrid automata can be composed concurrently. Hybrid automata can be composed hierarchically. 1. A mode can be a collection of submodes.

7. System Design Research Laboratory Model-based Tester A model-based tester emulates an environment that supplies a test case Environment /hardware constrains Environ. Model + Simulator System Model No Test Trace Yes Tester Model Determination Coverage Checker Coverag e Criteria Coverag e Criteria

8. System Design Research Laboratory Runtime verification MEDL specification Program Instrumentation Java Programs Runtime verification (monitoring) is to check the execution of a software program against its logic specification Java Programs +Filter Monitoring script Monitor MEDL compiler Computer events

9. System Design Research Laboratory Model-based Runtime Verification MEDL specification Model Instrumentation Monitor Synthesizer Simulator System Automaton ObserverMonitoring Automaton Instrumented Model

10. System Design Research Laboratory mMEDL: expressing your properties 1. MEDL (Meta Events Definition Language) is a linear interval temporal logic for specifying safety properties. 1. MEDL is initially introduced for monitoring Java programs in MaC (Monitor and Checking) System [KKL01] 2. Syntax 1. Defined on conditions, events, and expressions. C := [E, E) | : C | C && C | C||C | Q  Q E := e | start(C) | end(C) | E || E | E&&E | E when C Q := time(E) | c |Q } Q Where e is primitive event, c is a constant,  2 {>, <, =}, and } 2 {*,/,+,-}.

11. System Design Research Laboratory Semantics by example Informal Specification: if the dog loses the ball 50 seconds after the ball becomes visible, an alarm should be raised. mMEDL script Begin import event isVisible, isInvisible, track,lost; condition visible= [isVisible, isInvisible); event becameTruelost= lost when visible; alarm lostTrack=start (time(becameTruelost)-time(isVisible)>50); End isVisible, isInvisible, track, and lost are primitive events isVisible (isInvisible) occurs when  >10 becomes true (false). Lost (track) occurs when |  -x| > 10 become true (false). An alarm is an event which indicates the violation of the safety requirement..

12. System Design Research Laboratory Synthesizing model-based monitor 0: import events e 1,e 2, e 3 1: condition C=[e 1, e 2 ) 2: alarm E= e 1 when C || P  Ç: NewEvent T 0 : P=0\and NewEvent? T:=t, P:=1 NewEvent ´ : ( V e 1 >T Ç V e> >T Ç V e 3 >T) “engine” automaton V C : current value of C  C : last time when C changes V E : last time when E occurs Violation is detected if V E >0 Sequence of Primitive Events P2P2 T 0 : P=2 Æ V C =1 Æ V e 1 =t?V E :=t, P=0 T d : P=2 Æ ~( V C =1 Æ V e 1 =t)? P=0 P1P1 T 0 : P=1 Æ Up?V C :=1,  C :=t, P:=2 T 1 : P=1 Æ Down?V C :=0,  C :=t, P:=2 Up ´ (V C =0 Æ V e 1 =t) Down ´ (V C =1 Æ V e2 =t) T d : P=1 Æ : Up Æ : Down?P:=2 Synthesis is modularized

13. System Design Research Laboratory mPEDL: define primitive events Primitive events report the status change in the system model. Primitive events are formally encoded in mPEDL (model-based Primitive Event Definition Language) Events are defined as the changes on some predicate. MonScr Dog export event isVisible, isInvisible, lost, track; monobj int dog.vision; monobj int dog.ball_pan_delta; event isVisible= start (dog.vision>10); event isInvisible= end (dog.vision>10); event lost = start (dog.ball_pan_delta>10); event track = end (dog.ball_pan_delta>10); End

14. System Design Research Laboratory Model Instrumentation x  45? x  -45 |  |  10? |  | >10? 1. Model Augmentation A observer automaton is concurrently composed with the model. The structure of the model will not be changed. |  -x| > 10?V lost =t |  -x|  10?V track =t |  | > 10?V isVisible =t |  |  10?V isInvisible =t Observer

15. System Design Research Laboratory Design-level validation The composition of instrumented model, tester, and monitor forms a self-validating Charon model Tester supplies test trace during simulation Model-based tester emulates the movement of a “virtual” ball by changing input variables  and  The occurrence of an event is indicated by the changes on the event variables during simulation (a) Alarm Detection

16. System Design Research Laboratory Design-level validation (b) Primitive Events Emitted by Instrumented Model (c) The simulation trace of monitor

17. System Design Research Laboratory Implemtation-level validation Model-based TesterModel-based Monitor System Model Modular Code Generation Monitor Generated Code Tester Link as needed

18. System Design Research Laboratory Conclusion We proposed a framework for testing and monitoring model- based embedded systems. The property being verified is encoded in a temporal logic. It doesn’t require the changes on existing model-based design environment. The framework is for both design-level and implementation-level validations. It produces a self-testing and self-monitoring code for embedded system A set of tools are developed to automate the process.

19. System Design Research Laboratory Thank you!

20. System Design Research Laboratory Computation overhead The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step) On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. The controller program is a real-time system The slowdown is tolerated because the extra cost of tester and monitor can be observed within each integration step.

21. System Design Research Laboratory Implementation-level validation The alarm is used for calling the external functions to perform visible actions. 1. “play” function is called when an alarm lostTrack is detected The space overhead of tester and monitor.

22. System Design Research Laboratory MEDL semantics Interpreting MEDL on runs of hybrid automata, 1. A condition C maps a time period with true, false, or ?. If C is, 1. [E 1, E 2 ). C is true from event E 1 to E 2 (not included) 2. C 1 &&C 2. C is true when C 1 and C 2 are both true. 3. C 1 ||C 2. C is true when either of C 1 and C 2 is true. 4. : C 1. C is the dual of C 1. 2. An event E maps a time instance with true or false. If E is, 1. start(C). E occurs at the time C becomes true. 2. end(C). E occurs at the time C becomes not true. 3. E 1 ||E 2. E occurs when either E 1 or E 2 occurs. 4. E 1 &&E 2. E occurs when both E 1 and E 2 occur. 5. E 1 when C. E occurs when E 1 occurs and C is true 3. Q is an expression. If Q is, 1. time (E). Q’s value is the latest time E occurs. 2. Q 1 } Q 2. Q’s value is Q 1 } Q 2..

23. System Design Research Laboratory Related works and future directions 1. Related works: “platform-specific” approach v.s. “general purpose” approaches. 1. NASA Ames: Java PathExplorer 2. UPenn: MaC tools 3. Works on synthesizing finite model/test- oracle from the formal specification [DilRam96,GiaHav01] 2. Ongoing and future researches 1. Optimizing monitor-synthesizing algorithm. 2. Approximating general linear temporal property as monitoring automaton. 3. Testing and monitoring the code from conventional sources.

24. System Design Research Laboratory Computation overhead The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step) On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. The controller program is a real-time system The slowdown is tolerated because the extra cost of tester and monitor can be observed within each integration step.

25. System Design Research Laboratory Itemized space overhead The space overhead of tester and monitor.