Path-aware Time-triggered Runtime Verification Samaneh Navabpour 1, Borzoo Bonakdarpour 2, Sebastian Fischmeister 1 1 Department of Electrical and Computer Engineering 2 School of Computer Science University of Waterloo

Runtime Verification 2 Observation Report Steering Runtime Verification Framework

Event-triggered Monitoring Jittery overhead Bursts of invocations of the observer 3 1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, Sampling-based Runtime Verification, , FM11

Alternative Monitoring Approach Goals for the monitoring approach: Predictable monitoring Bounded overhead at each intervention 4

Problem of Time-triggered Monitoring 5 L1 L2L3L4L5L6 L7 L8L9L10 L11 L12L13L14L15L16 L17 m Program Execution Monitor L18 L19 Sample from monitor Critical instruction Sampling period = 2 m …… Achieving sound state reconstruction 1 1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, Sampling-based Runtime Verification, , FM11

Longest Sampling Period (LSP) 6 C16C A D5D5 C27C27 1 C38C38 C49C B 1 10 B Longest Sampling Period (LSP) 1 : is the minimum shortest path between two critical nodes 1. fib(int n) { 2. int i, Fnew, Fold, temp,ans; 3. Fnew = 1; Fold = 0; 4. i = 2; 5. while( i <= n ) { 6. temp = Fnew; 7.* Fnew = Fnew + Fold; 8.* Fold = temp; 9. i++; } 10.* ans = Fnew; 11. return ans;} 1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, Sampling-based Runtime Verification, , FM11

Cause of Redundant Sampling Using complete CFG to calculate LSP 7 C16C A D5D5 C27C27 1 C38C38 C49C B 1 10 B LSP path1 = 1LSP path2 = 5 LSP = 1 optimal LSP 6 samples Path 2 LSP path2 1 samples 84% reduction in samples path1 Not optimal path2

Path-aware Time-triggered Monitoring 1.Predict execution path 2.Calculate LSP using only predicted path 8 C16C A D5D5 C27C27 1 C38C38 C49C B 1 10 B path2 LSP = 5

Path Prediction Function 1.Predict execution path 9 Path prediction function Implement path prediction function using symbolic execution Execution path 1Path constraint 1 Execution path 2Path constraint 2 ……. Execution path nPath constraint n 1.Symbolize inputs. 2. Create table. 3. Check path constraints Environment

Adaptive Path-aware Time-triggered Monitoring Hypothetical execution path: 10 C16C A D5D5 C27C27 1 C38C38 C49C B 1 10 B LSP = 1LSP path3 = 1 region1 LSP= 5 region2 LSP= 1 18 samples 7 samples path 3 = region1 region2

LSP Regions An LSP region is a set of subpaths of an execution path: the same paLSP each subpath is maximal Regionalization objectives: 1.Reducing the number of LSP regions 2.Reducing the number of samples 3.Maintaining the absolute jitter of paLSP 11

Regionalization Algorithm 12 A W1W1 W2W2 W3W3 B C D E F A B C D E F A B C D E F A B C D E F A B C D E F

General Regionalization Can have different regions for different subpaths: 13 A BC D Path 1 A B F Path E LSP = 5 LSP = 10 LSP = 1

Tool Chain 14

Assumptions Limited to programs handled by KLEE Program is sequential Program runs on a single processor 15

Handling KLEE Limitations Concretization: Extract the instruction where concretization happens Find the node containing the instruction in CFG Append following sub-CFG to executed path 16 … … … …

Handling KLEE Limitations (cont) Incomplete paths: Extract the last executed instruction Find the node containing the last executed instruction in CFG Append following sub-CFG to executed path 17 … … … …

Reducing Table Size 18

Reducing Table Size KLEE patch: extracting unique paths: Table Compressor Remove entries that do not improve LSP. 19 Path with loop sequence Consecutive occurrences of Reduce consecutive occurrences to

Tool Chain (cont) Table Compressor: Implication Reduction: 20 A C1C1 B1B1 E1E1 E2E2 B2B2 B3B3 Z C2C2 C3C3 D1D1 D2D2 D3D3 D4D4 D5D5 D6D6 D7D7 D8D paLSP = 1 paLSPPCLSP Regions 1

Experimental Settings We use programs from SNU benchmark We run the program and monitor on MCB1700 board with RTX OS Time-triggered monitoring modes: Fixed-LSP Path-aware LSP Adaptive path-aware LSP ( ) History B. Bonakdarpour, S. Navabpour, and S. Fischmeister, Sampling-based Runtime Verification, , FM11

Experimental Settings (cont) Metrics for evaluation: 1.The values of the fixed LSP, paLSP, and adaptive paLSP 2.The number of redundant samples taken at run time by the monitor 3.The execution time of the monitored program. This value projects the amount of monitoring overhead 22

Values of paLSP and Adaptive paLSP 23 paLSP increases sampling period 2.4 times Adaptive paLSP increases sampling period 3.3 times

Redundant Samples of paLSP and Adaptive paLSP 24 paLSP decreases redundant samples by 44.8% Adaptive paLSP decreases redundant samples by 64%

Monitoring Overhead of paLSP and Adaptive paLSP 25 paLSP reduces monitoring overhead by 34% Adaptive paLSP reduces monitoring overhead by 51% 78% reduction in redundant samples Overhead of adaptive paLSP more than paLSP

Monitoring Overhead of paLSP and Adaptive paLSP with History 26 66% of paLSP+history has less overhead than event-triggered 75% adaptive paLSP+history has less overhead than event- triggered

Summary Sampling period must be devised based on execution path of the program (paLSP). Redundant samples can be further reduced when sampling period changes dynamically at runtime (adaptive paLSP). By merging history and paLSP or adaptive paLSP, we achieve a monitor suitable for time sensitive systems. Predictable monitoring Bounded overhead Imposes less overhead than event-triggered 27

Thank you Questions? 28

Future Work Creating a time-triggered monitor for component-based models Combining SMCO with time-triggered monitoring Using symbolic execution to combine event-triggered with time-triggered monitoring 29

Reduce over provisioning 30 Reduced over provisioning 1 1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, Sampling-based Runtime Verification, , FM11

Reducing Redundant Samples Using History Use auxiliary memory to avoid redundant samples LSP = 6 31 L1 L2L3L4L5L6 L7 L8L9L10 L11 L12L13L14L15L16 L17 Program Execution Monitor L18 L19 Longest Sampling Period (LSP) = 2 ….. x = 2 x = 10 x = 5 … … 51015

Regionalization Algorithm 32 A W1W1 W2W2 W3W3 B C D E F A B C D E F A B C D E F A B C D E F

33 W1W1 W2W2 W3W3 Regionalization Algorithm (cont) A B C D E F A B C D E F A BC D E F A BC D E F

34 W1W1 W2W2 W3W3 Regionalization Algorithm (cont) A B C D E F A B C D E F A B C D E F A B C D E F