1. MaCS: Monitoring, Checking and Steering O. Sokolsky, S. Kannan, I. Lee, U. Sammapun, J. Shin, M. Viswanathan CIS, Penn M. Kim SECUi.com, Korea

2. Outline Motivation Runtime Formal Analysis Our MaCS Prototype Examples Future Work

3. Motivation Two traditional approaches to certify correctness of systems Testing cannot guarantee correctness of application completely and lacks of formality. Formal verification lacks scalability and does not apply to implementation directly gap between models and implementations We need a new approach - run-time formal analysis

4. Run-time Formal Analysis Run-time formal analysis ensures the run-time compliance of an execution of a system with its formal requirement. The analysis validates properties on the current execution of application. The analysis can detect incorrect execution of applications predict error and steer computation collect statistics of actual execution

5. Run-time Formal Analysis using MaCS To detect incorrect execution, we need Monitor: monitors changes in an application by instrumenting monitor code into an application. This will create an abstract view of the application. Checker: uses monitor information and requirement spec to detect violation To steer, we need The violation information from Checker and the Feed-back to the application

6. System Spec System Spec Requirement Spec Requirement Spec Formal verification Design System Implementation System Implementation Monitoring Data Monitoring Data Implementation Monitor + Checker Monitor + Checker System Filter Communication Run-time Check MaCS Methodology

7. Issues in Run-time Formal Analysis Side effect of instrumentation to a target system An expressive formal language describing correctness criteria A proper granularity of monitoring Automatic v.s. Manual instrumentation Synchronous v.s. Asynchronous monitoring

8. Outline Motivation Runtime Formal Analysis Our MaCS Prototype Examples Future Work

9. MaCS languages PEDL: abstraction Run-time state: control locations object state local variables Abstract state: events conditions auxiliary variables MEDL PEDL SADL MEDL: abstract transformation SADL: feedback

10. Events and Conditions Information about the system comes in two different forms: Conditions, which are true, false, or undefined for a finite duration of time Events, which are either present or absent at some instant of time

11. Examples of Events and Conditions C3 C3. condition safeVolts = ( -10 < IP.volts ) && ( IP.volts < 10 ); IP.volts = 6IP.volts = 15 E1E2 IP.init() E1. event startProg = startM( IP.init() ); E2. event voltsChange= update( IP.volts);

12. Logic for Events and Conditions Let c be a primitive condition Let e be a primitive event Define Condition (C) and Event (E) as: ::= c | [, ) | ! | && | || | => ::= e | start( ) | end( ) | && | || | when

13. Conditions ::= [, ) ! && || => c| | | | | Primitive condition T from E1 until E2 T if C is F T if C1 and C2 are T T if C1 or C2 are T T if C2 is T whenever C1 is T

14. Events ::= start( ) && || when e| | | | | Primitive event An instant when C becomes T An instant when C becomes F An instant when both events happen An instant when either events happen An instant when E happens and C holds T end( )

15. The language that creates an abstract view of the application It maps the low-level state information of the system to high-level events/conditions used in describing the requirements. Primitive Event Definition Language (PEDL)

16. x=0,y=0 x=1,y=0 x=2,y=0 x=2,y=1 Program Execution Abstract View Monitor Sees x=3,y=1 x < 2 x =2 x> 2 x=3,y=2 Information Extraction

17. Primitive Event Definition Language (PEDL) Provides primitives to refer to values of variables and to certain points in the execution of the program. condition safeVolts = ( -10 < IP.volts ) && ( IP.volts < 10 ); event startProg = startM( IP.init() ); event voltsChange= update( IP.volts); Provides primitive “time” to refer to time when events happen condition slowDoor = ( time(doorClose) - time(doorOpen) ) > 3000;

18. Meta Event Definition Language (MEDL) Expresses requirements using the events and conditions, sent by a monitor. Defines Auxiliary Variables, which may be used to store history. RaiseGateEvent -> { num_train_pass++; } Defines more events and conditions using auxiliary variables and events/conditions sent by a monitor. event safeStart = startProg when (safeVolts)

19. Meta Event Definition Language (MEDL) (Cont. ) Using auxiliary variables, events and conditions, we describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. safeprop safeIP = safeTrack && safeVolts; alarm violation = start (!safeIP); Specifies updates to variables and steering invocations in response to events violation -> { controller = 0; invoke change2SafetyController(); }

20. Property checking A MEDL specification can also be seen as an automaton running on a stream of events provided by the monitor and the auxiliary variable aux. variables

21. Steering Action Definition Language (SADL) What: Identifies object used in steering // the target of steering is the object dm of type // DecisionModule located in the class IP DecisionModule IP:dm; How: Defines steering actions // setSC() method of dm is invoked steering action change2SC = { call (IP:dm).setSC(); } Where: Specifies steering conditions locations in the code where the actions can be executed before read DecisionModule:volts;

22. Steering process system checker violation action invoked action executed detection steering condition satisfied action invocation received event received

23. MaCS Toolset Program (.class) (Java byte code) Monitoring Script (PEDL) Requirements (MEDL) PEDLCompiler MEDLCompiler Instrumented Code Filter Generator (JTREK) Instrumentation Information Compiled PEDL Compiled MEDL Event Recognizer Checker Steering Script (SADL) SADLCompiler Instrumentation Information Injector class (Java byte code)

24. Filter (or an instrumented class) Initializes the connection between application and monitor. For each monitored object: for each variable update or start/end method, send events/conditions to the monitor.

25. Filter (or an instrumented class) IP.init() Instrumented class IP.volts = 6 IP.volts = 15 send to monitor

26. Implementation of monitor EventReconizer class: Listener (i.e. server) accepts events and conditions, specified in PEDL, from filter. Notify (i.e. client) sends events and conditions to runtime checker

27. Implementation of Monitor IP.init() Instrumented class Monitor IP.volts = 6 IP.volts = 15 startProg safeVolts = T voltsChange safeVolts = F send to checker

28. Implementation of checker RuntimeChecker class: Listener (i.e. server) accepts events and conditions, specified in MEDL, from monitor. Check for any violation If violation occurs, check if any action needs to be invoked by Steering Notify (i.e. client) sends action to be invoked to steerer.

29. Implementation of Checker IP.init() Instrumented class Monitor IP.volts = 6 IP.volts = 15 startProg safeVolts = T voltsChange safeVolts = F startProg alarm violation = voltsChange when !safeVolts ALARM OK Checker send to steering component

30. Implementation of steering Injector class: Action bodies as methods Listener thread accepts action invocations and sets invocation flags Calls to action bodies at fixed locations defined in the steering script, added by instrumentation

31. Implementation of Checker IP.init() Instrumented class Monitor IP.volts = 6 IP.volts = 15 startProg safeVolts = T voltsChange safeVolts = F startProg alarm violation = voltsChange when !safeVolts ALARM OK CheckerInjector Action change2SC IP.setSC()

32. Implementation Issues How close should the application and the monitor be? Tight coupled Pro: Decrease Steering Latency Con: Compete for resources

33. Implementation Issues Conservativeness How to ensure that the application does not suffer a significant performance degradation as long as steering is not invoked?

34. Implementation Issues Effectiveness How to ensure that the steering action cannot be ignored by the application? How to ensure that after the action is invoked, the application will not “undo” the action?

35. Inverted Pendulum (courtesy L. Sha) m l  x g M ufuf Example

36. Simplex Architecture (courtesy L. Sha) Safety Experimental Decision Module Physical System usus ueue xu SC EX x0x0 Equilibrium state Experimental controllers provide improved performance but uncertain stability properties Can be dynamically added or replaced Safety controller has the largest stability region

37. Experimental Controller Experimental Controller Inverted Pendulum in MaCS Device Drivers angle, track volts Decision Module Experimental Controller Switching logic Safety Controller JNIJNI monitor steer

38. Case study results It works! Without disturbance, there is slightly more jitter partly due to instrumentation but mostly due to Java Switch to the safety controller takes 0.32s (16 control cycles) bad values compensated by the safety controller in general may not be good enough

39. Future Work Synchronous vs. asynchronous steering currently: the application and may produce erroneous outputs when checking is slow May be unacceptable in critical situations alternative: before producing critical output, pause and wait for checker feedback IP case study suggests it is possible in some cases Need predictable checker turnaround Currently work in progress

40. Future Work Static analysis to enhance steering currently, user is responsible for specifying steering locations – error prone control flow analysis can identify locations by code- independent criteria e.g. “steering location is visited once in each main loop iteration” or even “if action changes variable x, do not place before the system writes x” What are the right criteria? Study other uses of steering