SS 2017 Software Verification Probabilistic modelling – DTMC / MDP Prof. Dr. Holger Schlingloff 1,2 Dr. Esteban Pavese 1 (1) Institut für Informatik der Humboldt Universität (2) Fraunhofer Institut für offene Kommunikationssysteme FOKUS

Model checking so far...

Model checking so far...

Model checking so far...

Model checking so far... Consider the following situation N processes run and communicate in a ring topology; i.e. each one sees two neighbours (left/right) The processes need to agree on a leader (a distinct process) Can only communicate left or right, and processes are anonymous (i.e., cannot send a message with pid)

Leader election Could you model this in SPIN or NuSMV or UPPAAL, or ...? We can’t. (At least not a deterministic version) Prove it!

Leader election We need to break the symmetry Idea – each process has a ”coin” Toss the coin, send result to process on the right, and receive from the left If tossed heads and received tails, keep participating but already decide not to be leader Receive a counter from left, add 1 to counter if already decided not to be leader, send counter to right If received a counter of N-1, become leader

Model checking so far... The techniques we’ve seen so far are somewhat lacking ...especially when we attempt to perform more realistic modelling ... or when the process we are modelling relies on randomizing its decisions ... or if we want to verify non-functional properties of the system

Probabilistic information The leader election is a nice example Many verification questions we can ask does it actually work? does it terminate? if it terminates, does it pick a unique leader? what is the chance it does not terminate? if 0, how long does it take to terminate? what is the chance it terminates in T time?

Probabilistic automata Of course, neither Finite Automata or Timed Automata suffice to answer these questions We need a new formalism (again!) – that of Probabilistic Automata “Probabilistic Automata” is a very general blanket term In this lesson we will focus on Discrete Time Markov Chains

Discrete Time Markov Chain (DTMC) State-based definition of a DTMC Outside of sw verification a random variable sequence definition seems more common A DTMC is a tuple with

No actions? DTMC transitions are purely probabilistic. Having different actions would imply some level of non-determinism So we can safely drop actions from the model

Example – Knuth/Yao dice or, how to play dice with coins

Knuth/Yao dice Is it correct? What is the chance of ending up on 2 pips? and 6 pips?

Probability basics

Probability basics Given an experiment the set Ω denotes the set of possible outcomes For 1 coin toss, Ω = {Heads,Tails} For 2 tosses, Ω = {HH,HT,TH,TT} An event E is any subset of Ω For 2 tosses, “at least 1 heads” is event = {HH,HT,TH} infinite tosses an event could be “4 heads in a row” A probability measure is a function μ: E → [0,1] such that μ(∅)= 0 μ(Ω)=1 For any pair of disjoint events e1 and e2, μ(e1) + μ(e2) = μ(e1∪e2)

Probability basics

Knuth/Yao dice Is it correct? What is the chance of ending up on 2 pips? and 6 pips?

Knuth/Yao dice Is it correct? do all have 1/6 chance? What is the chance of ending up on 2 pips? 1/6 (?) and 6 pips? 1/6...but how to measure?

Measuring over DTMCs Ω = (possibly infinite) set of (possibly infinite) paths Events: (possibly infinite) sets of (possibly infinite) paths Things get strange quite fast How is the probability of a path calculated? What about infinite paths? What about infinite sets of infinite paths?

Cylinder sets The cylinder (also called cone) of a finite path on a DTMC is the set of infinite paths that stems from it The measure of the cylinder is then easily calculated from the probability of the finite path

Knuth/Yao dice What is the chance of 2 pips? Infinite cones! (0,1,4,2p) + (0,1,3,1,4,2p) + ... + (0,1,(3,1)*,4,2p) + ... Infinite cones! How to solve this?

Knuth/Yao dice 2 pips state = 1, all other pip states 0 Each intermediate state measures the chance to get to 2p Define a (recurrent) equation system

Knuth/Yao dice

Another example...

Another example... It is customary to define the DTMC by its stochastic adjacency matrix

Some properties to check What is the probability of failing the first time? Measure the cones of the paths where the first send results in failure

Some properties to check What is the chance of eventually succeeding, without any failure in between?

Some notes for next class A large set of properties over DTMCs can be expressed as reachability properties In particular, both “safety” and “liveness” properties can be expressed this way Moreover, the probability of reaching a given state can be measured from the set of disjoint cones that characterises this reachability The set may be infinite but it is countably infinite

Some notes for next class Next class: the logic PCTL Its relationship with CTL and LTL Model checking procedure for PCTL Maybe you can already figure it out? Implementation details to increase feasibility of the model checking procedure DTMC extensions