Presentation is loading. Please wait.

Presentation is loading. Please wait.

Run Time Monitoring of Reactive Systems Mikhail Auguston Naval Postgraduate School Mark Trakhtenbrot Holon Academic Institute of Technology.

Similar presentations


Presentation on theme: "Run Time Monitoring of Reactive Systems Mikhail Auguston Naval Postgraduate School Mark Trakhtenbrot Holon Academic Institute of Technology."— Presentation transcript:

1 Run Time Monitoring of Reactive Systems Mikhail Auguston Naval Postgraduate School maugusto@nps.edu Mark Trakhtenbrot Holon Academic Institute of Technology ilmarktr@yahoo.com

2 Outline Problems in development of reactive systems Model-based development and analysis Our approach:  TL-based assertion language  Automatic generation of monitors for run-time verification Outline of translation scheme Further steps

3 Why development of reactive systems is difficult? Main problem: complex behavior  Intricate event-driven interaction with the environment  Concurrency  Timing factors Typically:  Critical applications (embedded RT controllers, …)  Must fulfill tough requirements (safety, timeliness, … )

4 Challenges  Capture requirements to express the expected behavior  Check that system meets the requirements  ideal: full verification  more realistic: testing * how to ensure a proper coverage? * how we know the test result is / isn’t correct? can this task be automated?

5 So, what to do? Apply formal methods! Precise spec of behavior:  Variants of TL [Pnueli, …] (future/past modalities; timing)  Patterns for typical properties [Avrunin, …] (response, precedence, existence, absence,…) Model-based development:  formal model of system (behavior, structure, …)  statecharts to capture behavior; a renown standard  executable model; a basis for automated tools  model-level analysis (closer to problem domain)

6 Expressing system properties Properties expressed using a special language, e.g. Temporal Logic Examples: ALWAYS (cross_empty or barrier_closed) ALWAYS (request  EVENTUALLY allocate_resource) Meaning: the property holds in any computation More complicated: property holds on part of execution paths (need quantifiers on path) Time-related properties: ALWAYS (p  EVENTUALLY (5) q) Q should happen within 5 time units after P.

7 Expressing system properties May be difficult to use without a strong logic background Combination of English with C-like expressions: Event A follows event B after less than 10 time units Patterns for typical behavioral properties E.g. Precedence patterns (S precedes P): Before R EVENTULLY P  (not P Until (S or R)) After Q Between Q and R After Q until R Pattern defines scope in which the property should hold

8 Statechart for Early Warning System

9 Model-based analysis with Statemate Simulation: check, record and replay test scenarios Verification: based on predefined assertion templates Code synthesis: executable C, for host or target OS Watchdog charts: - not part of the system model, but co-executed alongside it - used to drive and monitor system execution (in simulation or generated code run)

10 Model-based analysis Simulation: - Analysis of quality attributes (e.g. stability with respect to changes of the data) - Analysis of events occurring simultaneously (to resolve scheduling and priorities issues in the system) - "What if" checks (alternative decisions, "impossible" scenarios) - Check rarely occurring situations - Check behavior of the system under extreme values. - Influence of various uncontrolled and unpredictable factors - Time-related analysis (performance, very fast / very slow processes)

11 Model-based analysis Verification: Generic properties (e.g. existence of racing, or non- determinism) Model-specific properties, e.g. reachability of a state in which certain condition is fulfilled (formulated in terms of model elements) If violation found: a script is generated, driving simulation to the problematic state

12 Monitor charts Monitor (watchdog) chart: - Direct access (visibility) to all model elements - Support for box black-box and white-box testing Sample assertion: Processing of a request must be accomplished within 5 seconds, and before receiving of the next request Violation: entering into ERROR state WaitProcessError Request Processing Done Request or Timeout(5) /Violation Notification

13 Suggested approach: Statemate-based run-time verification Goal: support effective testing (finding errors rather than validation / showing correctness) Monitoring of code generated from system model Main features:  Assertion language to specify: - allowed and forbidden behavior - actions to be taken (collect and report diagnostic data)  Automatic creation of monitor charts from assertions Used Statemate capabilities:  Co-translation of model and monitor charts to C  Co-execution of generated code components

14 Advantages of the approach Flexible assertion language vs. predefined patterns Smooth transition: assertions  executable monitors Overcome limitations of simulation and verification

15  Analyze models reflecting design decisions (tasks/events mapping,…), in realistic target environment  Real time analysis - rather than SIM time SIM synch time (clock-driven systems): all reactions assume same duration SIM asynch time (event-driven systems): all reactions assume zero duration  No "finite state" limitation; may connect to real input sources  No limitations on model size

16 Assertion language  Boolean expressions: system state properties in(S) and (x > 5) ; entered(S)  started(A) ; ch(d)  Regular expressions: describe sequences of states [SELECT (Open | Read | Write | Close)] SATISFY Open (Read | Write )* Close  Temporal formulas: order properties ALWAYS (request  EVENTUALLY response) ALWAYS (execute  SOMETIME_WAS do_set_up)  Timed temporal formulas: real-time restrictions ALWAYS (request  EVENTUALLY (10) response)  Actions: to trace interesting events (e.g. property violation) reports, profiling info (based on objects’ attributes)

17 Example - assertion for EWS ALWAYS (OUT_OF_RANGE  EVENTUALLY (15) (RESET or started(PRINT_ALARM))) ON_FAIL [printf(“Violation after occurrence of OUT_OF_RANGE at time %f “, OUT_OF_RANGE.occur_time)]

18 More on temporal properties Manna & Pnueli Classification (P – past formula) :  Safety: ALWAYS (P), Guarantee: EVENTUALLY (P)  Obligation: Boolean combination of safety & guarantee  Response: ALWAYS (EVENTUALLY(P))  Persistence: EVENTUALLY (ALWAYS(P))  Reactivity: Bool. combination of response & persistence  Any TL formula is equivalent to a Reactivity formula; others used for convenience & flexibility Limitations:  nesting depth <= 2 for unrestricted future operators  unrestricted operator not embedded in a restricted one

19 Sample monitor construction ALWAYS (OUT_OF_RANGE  EVENTUALLY (15) (RESET or started(PRINT_ALARM))) / ERROR_MESSAGE

20 Translation of restricted formula ALWAYS (N) P

21 Translation of unrestricted formula ALWAYS (P)

22 Translation of unrestricted formula EVENTUALLY (P)

23 Translation of unrestricted formula ALWAYS (EVENTUALLY (P) )

24 Semantic issues  Monitoring: deals with finite runs; semantics of TL – based on infinite runs  [Eisner, et.al.]: reasoning on truncated executions  Assume the run finishes 4 sec after the last p ALWAYS (p  EVENTUALLY(10) q) ; false if no q after the last p ALWAYS (p  ALWAYS(10) q) ; true if always q after the last p  In general: user responsible to provide proper actions

25 Further steps Implementation:  done - experiments with manually created monitors  next step – actual implementation A more friendly assertion language  e.g. combine English with formulas Optimized translation for certain types of assertions Apply to other design paradigms using statecharts  e.g. in OO: monitor systems with dynamically created objects (problematic for model checking)


Download ppt "Run Time Monitoring of Reactive Systems Mikhail Auguston Naval Postgraduate School Mark Trakhtenbrot Holon Academic Institute of Technology."

Similar presentations


Ads by Google