Monte Carlo Model Checking Radu Grosu SUNY at Stony Brook Joint work with Scott A. Smolka

Model Checking ? Is system S a model of formula φ?

Model Checking S is a nondeterministic/concurrent system.  is a temporal logic formula. –in our case Linear Temporal Logic (LTL).

LTL Model Checking Every LTL formula  can be translated to a Büchi automaton B  such that L(  ) = L(B  ) Automata-theoretic approach: S |=  iff L ( B S )  L ( B  ) iff L ( B S  B  )   Checking non-emptiness is equivalent to finding a reachable accepting cycle (lasso).

recurrence diameter Lassos Computation tree (CT) Explore all lassos in the CT DDFS,SCC: time efficient DFS: memory efficient Checking Non-Emptiness LTL

Randomized Algorithms Huge impact on CS: (distributed) algorithms, complexity theory, cryptography, etc. Takes of next step algorithm may depend on random choice (coin flip). Benefits of randomization include simplicity, efficiency, and symmetry breaking.

Randomized Algorithms Monte Carlo: may produce incorrect result but with bounded error probability. –Example: Election’s result prediction Las Vegas: always gives correct result but running time is a random variable. –Example: Randomized Quick Sort

recurrence diameter Explore N( ,  ) independent lassos in the CT Error margin  and confidence ratio  Monte Carlo Approach LTL … flip a k-sided coin Lassos Computation tree (CT)

Lassos Probability Space Sample Space: lassos in B S  B  Bernoulli random variable Z : –Outcome = 1 if randomly chosen lasso accepting –Outcome = 0 otherwise p Z = ∑ p i Z i (expectation of an accepting lasso) where p i is lasso prob. (uniform random walk)

Example: Lassos Probability Space ½ ¼⅛ ⅛ q Z = 7/8 p Z = 1/8

Geometric Random Variable Value of geometric RV X with parameter p z : No. of independent lassos until success. Probability mass function: p(N) = P[X = N] = q z N-1 p z Cumulative Distribution Function: F(N) = P[X  N] = ∑ i  N p(i) = 1 - q z N

How Many Lassos? Requiring P[X  N] = 1- δ yields : N = ln (δ) / ln (1- p z ) Lower bound on number of trials N needed to achieve success with confidence ratio δ.

What If p z Unknown? Requiring p z  ε yields : M = ln (δ) / ln (1- ε)  N = ln (δ) / ln (1- p z ) and therefore P[X  M]  1- δ Lower bound on number of trials M needed to achieve success with confidence ratio δ and error margin ε.

Statistical Hypothesis Testing Null hypothesis H 0 : p z  ε Alternative hypothesis H 1 : p z < ε If no success after N trials, then reject H 0 Type I error: α = P[ X > M | H 0 ] < δ Since: P[ X  M | H 0 ]  1- δ

Monte Carlo Model Checking (MC 2 ) input: B=(Σ,Q,Q 0,δ,F), ε, δ N = ln (δ) / ln (1- ε) for (i = 1; i  N; i++) if (RL(B) == 1) return (1, error-trace ); return (0, “reject H 0 with α = Pr[ X>N | H 0 ] < δ”); where RL(B) performs a uniform random walk through B to obtain a random lasso.

Correctness of MC 2 Theorem: Given a Büchi automaton B, error margin ε, and confidence ratio δ, if MC 2 rejects H 0, then its type I error has probability α = P[ X > M | H 0 ] < δ

Complexity of MC 2 Theorem: Given a Büchi automaton B having diameter D, error margin ε, and confidence ratio δ, MC 2 runs in time O(N∙D) and uses space O(D), where N = ln(δ) / ln(1- ε) Cf. DDFS which runs in O(2 |S|+|φ| ) time for B = B S  B .

Implementation Implemented DDFS and MC 2 in jMocha model checker for synchronous systems specified using Reactive Modules. Performance and scalability of MC 2 compares very favorably to DDFS.

(Deadlock freedom) DPh: Symmetric Unfair Version

(Starvation freedom) DPh: Symmetric Unfair Version

DPh: Asymmetric Fair Version (Deadlock freedom) δ = ε = 1.8*10 -3 N = 1278

DPh: Asymmetric Fair Version (Starvation freedom) δ = ε = 1.8*10 -3 N = 1278

Related Work Random walk testing: –Heimdahl et al: Lurch debugger. Random walks to sample system state space: –Mihail & Papadimitriou (and others) Monte Carlo Model Checking of Markov Chains: –Herault et al: LTL-RP, bonded MC, zero/one ET –Younes et al: Time-Bounded CSL, sequential analysis –Sen et al: Time-Bounded CSL, zero/one ET Probabilistic Model Checking of Markov Chains: – ETMCC, PRISM, PIOAtool, and others.

Conclusions MC 2 is first randomized, Monte Carlo algorithm for the classical problem of temporal-logic model checking. Future Work: Use BDDs to improve run time. Also, take samples in parallel! Open Problem: Branching-Time Temporal Logic (e.g. CTL, modal mu-calculus).

Talk Outline 1.Model Checking 2.Randomized Algorithms 3.LTL Model Checking 4.Probability Theory Primer 5.Monte Carlo Model Checking 6.Implementation & Results 7.Conclusions & Open Problem

Model Checking S is a nondeterministic/concurrent system.  is a temporal logic formula. –in our case Linear Temporal Logic (LTL). Basic idea: intelligently explore S ’s state space in attempt to establish S |= .

Linear Temporal Logic LTL formula: made up inductively of atomic propositions p, boolean connectives , ,  temporal modalities X (neXt) and U (Until). Safety: “nothing bad ever happens” E.g. G(  (pc 1 =cs  pc 2 =cs)) where G is a derived modality (Globally). Liveness: “something good eventually happens” E.g. G( req  F serviced ) where F is a derived modality (Finally).

Emptiness Checking Checking non-emptiness is equivalent to finding an accepting cycle reachable from initial state (lasso). Double Depth-First Search (DDFS) algorithm can be used to search for such cycles, and this can be done on-the-fly! s1s1 s2s2 s3s3 sksk s k-2 s k-1 s k+1 s k+2 s k+3 snsn DFS 2 DFS 1

Bernoulli Random Variable (coin flip) Value of Bernoulli RV Z: Z = 1 (success) & Z = 0 (failure) Probability mass function: p(1) = Pr[Z=1] = p z p(0) = Pr[Z=0] = 1- p z = q z Expectation: E[Z] = p z

Statistical Hypothesis Testing Example: Given a fair and a biased coin. –Null hypothesis H 0 - fair coin selected. –Alternative hypothesis H 1 - biased coin selected. Hypothesis testing: Perform N trials. –If number of heads is LOW, reject H 0. –Else fail to reject H 0.

Statistical Hypothesis Testing H 0 is TrueH 0 is False reject H 0 Type I error w/prob. α Correct to reject H 0 fail to reject H 0 Correct to fail to reject H 0 Type II error w/prob. β

Random Lasso (RL) Algorithm

Randomized Algorithms Huge impact on CS: (distributed) algorithms, complexity theory, cryptography, etc. Takes of next step algorithm may depend on random choice (coin flip). Benefits of randomization include simplicity, efficiency, and symmetry breaking.

Randomized Algorithms Monte Carlo: may produce incorrect result but with bounded error probability. –Example: Rabin’s primality testing Las Vegas: always gives correct result but running time is a random variable. –Example: Randomized Quick Sort

Lassos Probability Space L 1 = 11 L 2 = 1244 L 3 = 1231 L 4 = Pr[L 1 ]= ½ Pr[L 2 ]= ¼ Pr[L 3 ]= ⅛ Pr[L 4 ]= ⅛ q Z = L 1 + L 2 = ¾ p Z = L 3 + L 4 = ¼

Alternative Sampling Strategies 01 n n-1 Multilasso sampling: ignores backedges that do not lead to an accepting lasso. Pr[L n ]= O(2 -n ) Probabilistic systems: there is a natural way to assign a probability to a RL. Input partitioning: partition input into classes that trigger the same behavior (guards).