Run-time Verification of Software Systems Oleg Sokolsky Department of Computer and Information Science University of Pennsylvania January 21, 2003

Acknowledgements Collaboration: Professors: Students: Support: Insup Lee Sampath Kannan Students: Mahesh Viswanathan Moonjoo Kim Usa Sammapun Support: ONR, NSF, ARO 9/18/2018

Outline Introduction Why I work in formal methods? Overview What is run-time verification, and why do we care? Main body A framework for run-time verification and its implementation Conclusions Beyond run-time verification 9/18/2018

What are formal methods? Behavioral properties, e.g. temporal logic Models provide a higher-level view of the system “Check before you build” “Correct by construction” Debug and fix errors 9/18/2018

Advantages of formal methods Modeling as means of reuse and lifecycle support Analyze before you build Early detection of errors Correct by construction Implementation by refinement preserves properties of the model Analysis provides counterexamples Debug and fix errors easier 9/18/2018

Do formal methods deliver? Modeling is tedious and hard to get right State explosion Seldom goes all the way Cryptic and large 9/18/2018

Formal methods drawbacks “Garbage in, garbage out” Large detailed models are almost as hard to craft as code, and have less tool support Analysis is as good as the model Only small models can be exhaustively analyzed Rather than proving correctness, formal methods provide sophisticated debugging support Only simple models can be refined into code If implementation is not fully automatic, what have we learned from analyzing the model? 9/18/2018

Is it hopeless? Of course not! What do we need? Better modeling languages More domain-specific, more declarative, better tool support Model-driven implementation techniques Model-driven code generation rather than refinement Model-driven test generation Novel applications of formal methods Run-time verification 9/18/2018

Outline Introduction Why I work in formal methods? Overview What is run-time verification, and why do we care? Main body A framework for run-time verification and its implementation Conclusions Beyond run-time verification 9/18/2018

Motivation Problem: No matter what we do, the system can misbehave at run time We check models, not implementations We cannot always check the model exhaustively Solution: Validate system executions at run-time 9/18/2018

Advantages and disadvantages + Check implementation, not model Avoid state explosion - Only one execution is checked Assess coverage metrics? And what if it breaks? Provide feedback for recovery? 9/18/2018

What is run-time verification? Check the same properties Simpler checking algorithm Extract only relevant information Low-level execution data vs. high-level requirements 9/18/2018

Monitoring behavioral properties Formulas in a temporal logic Always evaluated over a finite execution trace Safety properties “something bad does not happen” Raise alarm when the bad happens Liveness properties Requires non-traditional interpretation Ultimately, properties determine what observations are relevant 9/18/2018

Checking a property of a trace Satisfaction relation t: Simple algorithm, linear in the trace length At each step, trace becomes longer t’: Furthermore, traces are too big to store Need a different approach 9/18/2018

Incremental checking of a trace In fact, we do not need to check the whole trace over and over again Keep a checker state values of all subformulas Upon each observation, update checker state 9/18/2018

Requirements vs. observations System requirements are high-level and independent of an implementation Run-time observations are low-level and implementation-specific Software: variable assignments, function calls, exceptions, etc. Network: send, receive, route packets, update routing tables, etc. Need an abstraction layer to match the two 9/18/2018

Trace extraction Too much information is just too much! Trace is a sequence of observations A temporal projection of execution Observation is a projection of system state Keep only relevant state components Too little information is a problem, too Did you miss anything important? Can you see everything you need? Software state is observable Networks/hardware may have unobservable state Can you see well enough? 9/18/2018

Outline Introduction Why I work in formal methods? Overview What is run-time verification, and why do we care? Main body A framework for run-time verification and its implementation Conclusions Beyond run-time verification 9/18/2018

MaC: Monitoring and Checking Designed at UPenn Components: Architecture for run-time verification Languages for monitoring properties and trace abstraction Steering in response to alarms Prototype implementation for Java programs Implementation of checking algorithms Automatic instrumentation 9/18/2018

The MaC framework Requirement Program Spec Human Monitoring Script low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

MaC languages PEDL Abstract state: Run-time state: SADL MEDL control locations object state local variables Abstract state: events conditions auxiliary variables SADL MEDL PEDL: Primitive Event Definition Language abstraction MEDL: Meta Event Definition Language abstract transformation SADL: Steering Action Definition Language feedback 9/18/2018

Properties: events and conditions Natural distinction for monitoring properties: instantaneous vs. durational Instantaneity depends on time granularity Motivation for the distinction: Cannot monitor every time instance If you saw something in an observation, is it still there while you are not looking? Yes – it is a condition No – it is an event 9/18/2018

Example: hundred years’ war The war is a condition Battles are events Battle durations notwithstanding Events change the state of conditions end(War)=FallOfBordeaux FinalDefeat = [FallOfParis,FallOfBordeaux) 9/18/2018

Property specification Program Requirement Spec Human Monitoring Script low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

Meta Event Definition Language Express requirements using the events and conditions, gathered from an execution define events and conditions using incoming primitive events and checker state variables Describe the safety requirements properties (conditions that must always be true) alarms (events that must never be raised) Independent of the monitored system 9/18/2018

MEDL: events and conditions Grammar: Obvious tautologies: c = [start(c),end(c)) start([e1,e2))=e1 end([e1,e2))=e2 9/18/2018

Semantics of events and conditions 2-sorted past temporal logic: Interpreted over timed traces Semantic function defines The value of a condition at a given time instance When an event occurs Two-valued or three-valued interpretation 9/18/2018

MEDL: auxiliary variables Checker state, in addition to values of events and conditions, contains auxiliary variables Used to define additional events and conditions: event 5th_e = e when (e_count==5) Updates triggered by events: e -> { e_count += 1 } Traces are interpreted over both system state variables and checker state variables 9/18/2018

Trace extraction and abstraction Program Requirement Spec Human Monitoring Script low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

Primitive Event Definition Language Abstracts low-level run-time observations into high-level events each primitive event or condition used in MEDL must have a PEDL definition Semantically, a subset of MEDL Single-state definitions By necessity, PEDL is system dependent 9/18/2018

PEDL for Java Monitored objects Monitored methods Class variables and local variables of methods Updates of monitored objects define primitive events Expressions over monitored objects define primitive conditions Monitored methods Calls and returns define primitive events “inside a call to a monitored method” is a primitive condition 9/18/2018

Instrumentation Process Variable names are available inside classfile Bytecode instructions to be instrumented: local variables : <T>store, <T>store_<n> and iinc global variables : putfield and putstatic exec points : beginning and return points of the method Instrumentation algorithm inserts code for monitoring updates target addresses of jump instructions updates the size of operand stack 9/18/2018

Filter Probes inserted into the target system Thread for handling update buffers Updates reports are synchronized for correct event ordering. We need a guard against the race condition due to multi threads in order to ensure the correct order of event reports. Without synchronization, the order of reports can be incorrect. We use a lock for synchronization mechanism. 9/18/2018

Event recognition Requirement Program Spec Human Monitoring Script low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

Event recognition Event Recognizer evaluates a PEDL script on each message from the filter store the last seen value of each monitored variable Specified by PEDL condition foo = x + y ≤ 5 x y foo undefined 1 undefined 2 true send event to checker 4 true 2 false send event to checker 9/18/2018

MEDL evaluation Requirement Program Spec Human Monitoring Script low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

Property checking A MEDL specification can be seen as an automaton with auxiliary store running on a stream of events provided by the event recognizer aux. variables 9/18/2018

Efficient evaluation of MEDL Checker state: Value of each condition and auxiliary variable Timestamp of most recent occurrence of each event Representation: forest of dependencies between events and conditions Lazy bottom-up evaluation Based on statically assigned height 9/18/2018

Steering Requirement Program Spec Human Monitoring Script System Event low level description high level Input System Filter Automatic Instrumentation Event Recognizer Automatic Translation Run-time Checker Automatic Translation The inputs to the MaC framework are program and informal requirement specification. Based on these two input, human user writes down the monitoring script. The monitoring script consists of both low-level description and high-level description. Based on the low level description and program, the program is instrumented automatically. Instrumented program has a filter as a component. Also, based on the low level description, event recognizer is automatically generated. Similarly, based on the high level description, run-time checker is generated. At run-time, filter extracts low level information like values of variables and sends it to the event recognizer. The event recognizer detects high level events and sends these high level events to the run-time checker. Run-time checker detects violation of properties based on these high level events. As you see, we separate the low level activity and high level activity both in descriptions and run-time components of the MaC framework. This separation has three benefits. First, whenever the target program is changed, we have to change only the low level description, not the high level description. The high level description of properties in the MaC framework is reusable. Second, the MaC framework causes less overhead to the target system because most of components are running on different machines. Third, the run-time components are modularized so that easy to implement the MaC freamework. I concentrate on left half of the MaC framework. Mahesh Viswanathan concentrates on the right half of the framework and theoretical foundation for run-time formal analysis. As you see, the monitoring script is the core of the MaC framework. Let us see the MaC language. Static Phase low-level observations high-level events Run-time Phase steering 9/18/2018

Steering Steering is a way to provide feedback to the system that a violation has occurred Requires exact knowledge of the system functionality System should be ready to receive feedback User specifies when it is safe to steer and what is the appropriate action Not a recovery mechanism, but a vehicle to invoke recovery mechanism 9/18/2018

Steering process system checker action executed detection action invocation received steering condition satisfied violation action executed system event received action invoked detection checker 9/18/2018

Steering Action Definition Language Identifies objects used in steering Defines steering actions Specifies steering conditions locations in the code where the actions can be executed steering action change2SC = { call (IP:dm).setSC() } before read DecisionModule:volts Invocation in response to events (in MEDL) SEviolation -> { invoke change2SC } 9/18/2018

Implementation of steering Injector class: Action bodies as methods Listener thread accepts action invocations and sets invocation flags Calls to action bodies at fixed locations defined in the steering script, added by instrumentation Injector class Checker execution invoke Invocation flags test steering conditioni satisfied i n Action bodies steering conditioni satisfied i call n 9/18/2018

MaC Prototype with steering Steering Script (SADL) Program (Java byte code) Monitoring Script (PEDL) Requirements (MEDL) SADL Compiler PEDL Compiler MEDL Compiler Instrumentation Information Injector class (Java byte code) Instrumentation Information Filter Generator (JTREK) Compiled PEDL Compiled MEDL Instrumented Code Event Recognizer Checker 9/18/2018

Case studies I Particle formations based on artificial physics Balance attraction and repulsion Susceptible to disturbances Monitoring Check the progress of pattern formation Restart pattern formation if disturbance is observed Suspend and then restore repulsion Statistically improves quality of formations 9/18/2018

Case studies II MaC-based Simplex architecture Controller for an inverted pendulum with hot-swapping of experimental controllers Enhanced performance, uncertain stability Use checker to compute safety envelope Use steering to switch between controllers 9/18/2018

Inverted Pendulum in MaC Experimental Controller Experimental Controller Experimental Controller Safety Controller angle, track monitor Device Drivers J N I Decision Module Switching logic volts steer 9/18/2018

Outline Introduction Why I work in formal methods? Overview What is run-time verification, and why do we care? Main body A framework for run-time verification and its implementation Conclusions Beyond run-time verification 9/18/2018

Summary Run-time verification is a recent (since 1999) direction in formal methods Bridge the gap between model and implementation Avoid state explosion in checking algorithms Annual workshop on run-time verification 4th workshop is a satellite to TACAS ’04 9/18/2018

Summary MaC is a framework for run-time verification Architecture Monitoring languages Checking algorithms Applications for checking and steering Java programs Network simulations QoS requirements (on-going) 9/18/2018

Related work Other run-time verification approaches Java Path Explorer, Time Rover Incremental checking algorithms, collection of statistics, … Model-based diagnostics State estimation Event definition languages Monitoring tools Automatic instrumentation Post-mortem analysis 9/18/2018

Future directions Integration of static and run-time analysis Suppose we did partial model checking? Synchronous checking Faster detection of problems More efficient extraction of data Distributed MaC Collection from distributed platforms Decentralized checking Monitoring coverage estimation How much confidence have we gained? 9/18/2018

Other research directions Modeling approaches Hierarchical hybrid systems Uniform resource modeling Code generation from hybrid models Feasibility constraints on models Model-based test generation Use model checkers to generate the “right” counterexamples Modeling of genetic pathways by hybrid systems 9/18/2018