1. 5/27/03MDES 20031 Supporting Model-Based Validation at Run-time Insup Lee and Oleg Sokolsky Department of Computer and Information Science University of Pennsylvania

2. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 2 Software Development Process Requirements capture and analysis –Informal to formal –Consistency and completeness –Assumptions and interfaces between system components –Application-specific properties Design specifications and analysis –Formal modeling notations –Analysis techniques (simulation, model checking, equivalence checking, testing, etc.) –Abstractions Implementation –Manual/automatic code generation (refinement) –Validation Testing Model extraction and verification Run-time monitoring and checking Requirements Design specification Implementation

3. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 3 Model-based validation at run-time Run-time verification is to assure the compliance of an execution of a system with its formal requirements. This is achieved by monitoring the current execution and checking properties. The monitoring and checking (MaC) system can –detect incorrect execution of an application –predict error and steer computation –collect statistics of actual execution (e.g., QoS validation)

4. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 4 System Spec System Spec Requirements Spec Requirements Spec Formal verification Design System Implementation System Implementation Monitoring Refinement Monitoring Refinement Implementation Event Recognizer + Checker Event Recognizer + Checker System Filter Communication Run-time Check MaC Methodology

5. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 5 The MaC Languages Meta Event Definition Language(MEDL) –Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. –Target program implementation independent. Primitive Event Definition Language (PEDL) –Maps the low-level state information of the system to high-level events. –Provides primitives to refer to values of variables and to certain points in the execution of the program. –PEDL is defined so that events can be recognized in time linear to the size of the PEDL specification –Depends on target program implementation Run-time state: control locations object state local variables Abstract state: events conditions auxiliary variables MEDL PEDL

6. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 6 Design of the MaC Languages Must be able to reason about both time instants and information that holds for a duration of time in a program execution. –Events and conditions are a natural division, which is also found in other formalisms such as SCR. –Conditions, which are true or false for a finite duration of time (e.g., is variable x >5?) –Events, which are either present or absent at some instant of time (e.g., is control right now at the end of method f?). Need temporal operators combining events and conditions in order to reason about traces. start(position==100)end(position==100) 1:00:101:00:301:00:15 raiseGate Time position == 100

7. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 7 Logical Foundation Conditions interpreted over 3 values: true, false and undefined. [.,.) pairs a couple of events to define an interval. start and end define the events corresponding to the instant when conditions change their value.

8. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 8 Meta Event Definition Language (MEDL) ReqSpec /* Import section */ import event ; import condition ; /*Auxiliary variable */ var int ; /*Event and condition */ event =...; condition =...; /*Property and violation */ property =...; alarm =...; /*Auxiliary variable update*/ -> { :=... ; } End Expresses requirements using the events and conditions, sent by event recognizer. Expresses the subset of safety properties. Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. –property safeRRC = IC -> GD; –alarm violation = start (!safeRRC); Auxilliary variables may be used to store history. –endIC-> { num_train_pass’ = num_train_pass + 1; }

9. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 9 Property checking A MEDL specification can be seen as an automaton with auxiliary store running on a stream of events provided by the event recognizer aux. variables

10. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 10 PEDL for Java The language maps the low-level state information of the system to high-level events and conditions used in describing the requirements. Provides primitives to refer to –primitive variables –beginnings/endings of methods Primitive conditions are constructed from –boolean-valued expressions over the monitored variables Ex: condition IC = (position == 100); Primitive events are constructed from –update(x) –startM(f)/endM(f) Ex: event raiseGate= startM(Gate.gu()); MonScr /* Export section */ export event ; export condition ; /* Overhead reduction */ [timestamp;] [valueabstract;] [deltaabstract;] [multithread;] /* Monitored entities */ monobj ; monmeth ; /* Event and condition*/ event =...; condition =...; End

11. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 11 Java-MaC Prototype Requirement Specification Program (Java source code) Program (Java byte code) Monitoring Script (PEDL) Requirements (MEDL) PEDLCompiler MEDLCompiler Instrumented Target program Filter (Compiled MEDL) Run-time Checker (Compiled PEDL) Event Recognizer Filter Generator (JTREK) Instrumentation Information Static Phase Dynamic Phase

12. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 12 PEDL for Java (cont.) Events can have two attributes - time and value time(e) gives the time of the last occurrence of event e –used for expressing temporal properties value(e,i) gives the i th value in the tuple of values of e –value of update(var) : a tuple containing the current value of var –value of startM(f) : a tuple containing parameters of the method f –value of endM(f) : a tuple containing parameters and a return value of the method f

13. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 13 Specifications for Stock Clients MonScr StockClient export event startPgm, periodStart, conFail, queryResend, oldDataUsed; monmeth void Client.main(String[]); monmeth void Client.run(); monmeth void Client.failConnection(ConnectTry); monmeth Object Client.retryGetData(int); monmeth Object Client.processOldData(); event startPgm = startM(Client.main(String[])); event periodStart = startM(Client.run()); event conFail = startM(Client.failConnection(ConnectTry)); event queryResend = startM(Client.retryGetData(int)); event oldDataUsed = startM(Client.processOldData()); End ReqSpec StockClient import event startPgm, periodStart, conFail, queryResend, oldDataUsed; var long periodTime; var long lastPeriodStart; var int numRetried; var int numConFail; alarm violatedPeriod = end((perioidTime’ >= 900) && (periodTime’ <= 1100)); alarm wrongFT = oldDataUsed when ( (numRetries’ < 4)|| (numConFail’ < 3)); startPgm -> {periodTime’ = 1000; lastPeriodStart’ = time(startPgm) -1000; numRetries’ = 0; numConFail’ = 0;} periodStart ->{ numREtries’ = 0; numConFail’ = 0; periodTime’ =time(periodStart)-lastPeriodStart; lastPeriodStart’ = time(periodStart);}... End

14. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 14 Extensions for Embedded Systems MaC based on hybrid system models Distributed MaC Steering-based adaptation Model-based profiling

15. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 15 Monitoring of hybrid systems Embedded systems often contain continuously evolving as well as discrete components Monitoring of continuous entities introduces additional challenges –Monitoring of discrete systems is event-driven, which is not enough for continuous changes –“Continuous monitoring”, in practice, involves sampling –Sampling rates for each continuous entity can be obtained from a hybrid model of the embedded system Deriving properties of embedded systems: –Use a hybrid model as an oracle, driven by the observations

16. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 16 Distributed MaC Run-time monitoring and checking of distributed systems –Local monitors and checkers –How to decompose a global property into locally checkable properties, i.e., Given the set of nodes, N 1,…, N m and a global property P g, find locally checkable properties, P 1,…, P m such that if N i detects the violation of P i, then P g is violated. –Tradeoff between amount and frequency of communication and timely detection of property violation

17. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 17 Steering-based Adaptation

18. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 18 Steering-based adaptation Steering provides feedback from the monitor to the system When can a system be effectively steered? –the system is designed for run-time adjustments e.g., Simplex architecture –the system naturally offers a degree of tolerance e.g., routing protocols: flush buffers when performance decreases Need timing guarantee over feedback loop –Real-time Java

19. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 19 Model-based Profiling Collect statistics during monitored executions Over multiple runs, such statistics will provide dependability and performance measures Possible uses: –Collect coverage Successful runs serve as tests Enhance “synthetic” test suites –Infer dynamic interfaces (e.g. interface automata) Record input and output patterns of successful executions Helps in reusing the component in different contexts

20. SDRL & RTG University of Pennsylvania 5/27/03 MDES 2003 20 Summary The MaC architecture provides a lightweight formal methodology for assuring of the correct execution of a target program at run-time –Rigorous analysis, Flexible, Automatic Identified several possible extensions to MaC Reduction of monitoring overhead Port the MaC architecture to platforms other than Java http://www.cis.upenn.edu/~rtg/mac

21. 5/27/03MDES 200321 Q & A