Model Checking for Probabilistic Timed Systems Jeremy Sproston Università di Torino VOSS Dagstuhl seminar 9th December 2002

The problem Model checking probabilistic timed systems –In probabilistic systems: Probabilistic choice between alternatives Example: electronic coin flipping in randomized algorithms –In timed systems: Timing parameters are critical for the correct functioning of the system Example: the system must meet a certain deadline –In probabilistic timed systems: Coexistence of probabilistic choice and timing

The focus Probabilistic versions/extensions of timed automata (Alur and Dill 1994) Timed automaton = finite-state graph + clocks + clock constraints Clocks are an appropriate device for modelling time-dependent behaviour –A clock is a real-valued variable which increases at the same rate as real time Clocks can be reset when system transitions occur Therefore, clocks can measure the exact amount of time elapsed since a particular transition

Timed automata Finite-state graph + clocks + clock constraints (examples: x  3, x-y>5) Example: light switch off x2x2 x3x3 on {x:=0}

Timed CTL CTL: a request will always follow a response  ⃞ (request -> (  ⃟ response)) TCTL: timed CTL –Alur, Courcoubetis and Dill (1993) –Henzinger et al. (1994) –A request will always follow a response within 5 milliseconds  ⃞ (request -> (  ⃟  5 response)) –Use ⊨ T for the satisfaction relation of TCTL

Timed automata: semantics Problem: underlying semantic model is –infinite-state: (node space) x R (number of clocks) –infinitely branching: for example Model checking classically assumes a finite state space Off, x=3.5 Off, x=3.7 ……

Model checking for timed automata Reduce to a finite state space: clock equivalence Partitioning bounded by the maximal constant used in the timed automaton or the TCTL formula Clock equivalent states satisfy the same clock constraints now and in the future x y

Model checking for timed automata Region equivalent states have the same –node –clock equivalence class Construct finite-state region graph (transition system) –States: region equivalence classes –Transitions: Time transitions Discrete transitions E.g. crossing an edge with {x:=0}

Model checking for timed automata Let: –TA be a timed automaton, –  T be a TCTL formula, –RG(TA,  T ) be the region graph of TA,  T TA ⊨ T  T if and only if RG(TA,  T ) ⊨  –where ⊨ and  are “untimed” versions of ⊨ T and  T Key result of Alur, Courcoubetis and Dill (1993)

Real-time probabilistic processes Alur, Courcoubetis and Dill (1991:ICALP, 1991:Real-Time) Similar to Generalized Semi-Markov Processes (Whitt (1980), Glynn (1989)) A fully probabilistic model

Real-time probabilistic processes Finite-state graph + clocks + clock scheduling function + probabilistic branching over edges + probabilistic clock resetting Example: light switch off {x} x,y on {y} y y:=Uniform(1,30) x:=3

Timed CTL revisited Interpreting “branching-time” logic over fully probabilistic systems s ⊨  means “the probability that the computations starting in s satisfy  is > 0” s ⊨  means “the probability that the computations starting in s satisfy  is =1” Alur, Courcoubetis and Dill (1991:ICALP) interpret TCTL (branching-time) over real-time probabilistic processes

Timed CTL revisited For example:  ⃞ (request -> (  ⃟  5 response)) With probability 1, a request is followed by a response within 5 milliseconds Use R-TCTL to denote the logic, and ⊨ R for its satisfaction relation

Real-time probabilistic processes: semantics Real-time probabilistic processes use clocks, so are infinite-state Markov processes Clocks are set to negative values drawn from continuous probability distributions When at least one clock reaches 0, a transition is triggered

Model checking for real-time probabilistic processes Again, reduce to a finite state space using (a version of) clock equivalence The set of clocks to reach 0 first is the same for all clock equivalent states x y

Model checking for real-time probabilistic processes Construct finite-state region graph (transition system) –States: region equivalence classes –Transitions: Time transitions Discrete transitions E.g. crossing an edge triggered by y; reset y within (1,2)

Model checking for real-time probabilistic processes Let: –RTPP be a real-time probabilistic process –  R be a R-TCTL formula, –RG(RTPP,  R ) be the region graph of RTPP,  R RTPP ⊨ R  R if and only if RG(RTPP,  R ) ⊨  –where ⊨ and  are “untimed” versions of ⊨ R and  R Key result of Alur, Courcoubetis and Dill (1991:ICALP)

Probabilistic timed automata Introduced by Jensen (1995), Kwiatkowska et al. (2002) Finite-state graph + clocks + clock constraints + probabilistic branching over edges Example: light switch off x2x2 x3x3 on {x:=0}

Probabilistic timed CTL PCTL (Probabilistic CTL): Hansson and Jonsson (1994), Bianco and de Alfaro (1995) –The system will fail with probability < 0.01 P <0.01 [ ⃟ failure] PTCTL (timed PCTL): Kwiatkowska et al. (2002) The system will fail within 5 hours with probability < 0.01 P <0.01 [ ⃟  5 failure] Use ⊨ P to denote the satisfaction relation of PTCTL

Model checking probabilistic timed automata Probabilistic timed automaton semantics: –Infinite-state, infinite-branching Markov decision process Again, reduce to a finite state space using clock equivalence x y

Model checking probabilistic timed automata Construct finite-state region graph (Markov decision process) –States: region equivalence classes –Transitions: Time transitions are as standard Discrete transitions: for example on {x:=0} fail y<3x<7 on fail

Model checking probabilistic timed automata Construct finite-state region graph (Markov decision process) –States: region equivalence classes –Transitions: Time transitions are as standard Discrete transitions: for example on {x:=0} fail y<3x<7 on fail {y:=0} on

Model checking probabilistic timed automata Let: –PTA be a probabilistic timed automaton, –  P be a PTCTL formula, –RG(PTA,  P ) be the region graph of PTA,  P PTA ⊨ P  P if and only if RG(PTA,  P ) ⊨  –where ⊨ and  are “untimed” versions of ⊨ P and  p Key result of Kwiatkowska et al. (2002)

Continuous probabilistic timed automata Introduced by Kwiatkowska et al. (2000) Finite-state graph + clocks + clock constraints + probabilistic branching over edges + probabilistic clock resetting Example: light switch x2x off1on off2 y y  30 x,y x  3 ∧ y  30 y  30 y=30 y:=Uniform(0,29) x:=0

Model checking continuous probabilistic timed automata Continuous probabilistic timed automata semantics –Infinite-state, infinitely branching probabilistic-nondeterministic system with continuous probability distributions Again, reduce to a finite state space using clock equivalence

Model checking continuous probabilistic timed automata Problems with clock equivalence: an example by Alur Clock x is reset within (0,1) in node A; clock y is arbitrary Some time elapses in node A Then we move to node B; clock y is reset within (0,1) 3 cases: (1) x y Probability of (2) is 0, but we do not know the probabilities of (1) and (3) (clock equivalence abstracts from the duration of the time transition in node A) x x=1 y x<1 y=1 A B

Model checking continuous probabilistic timed automata A partial solution: change the granularity of the time scale –For example, from granularity of 1 to granularity of 0.5 –Say we know that x  (0,0.5) –Say that y is then set within (0.5,1) –We know that y>x

Model checking continuous probabilistic timed automata Given a time granularity, construct a finite- state region graph (Markov decision process) –States: region equivalence classes –Transitions: Time transitions are standard Handling of probabilistic branching over edges is straightforward But how do we deal with resetting clocks according to continuous probability distributions?

Model checking continuous probabilistic timed automata Representing continuously distributed clock resets in the region graph: –Integrating over time-unit intervals gives the probability of a clock being set within an interval E.g. with a time granularity of 1, we integrate over intervals such as (0,1), (1,2), … E.g. with a time granularity of 0.5, we integrate over intervals such as (0,0.5), (0.5, 1), … –But the relationship between the ordering on the fractional parts of the newly set clocks and the clocks which keep their old values is not obtainable –The probabilistic choice regarding this relationship is replaced with a nondeterministic choice

Model checking continuous probabilistic timed automata Let: –CPTA be a probabilistic timed automaton, –  P be a PTCTL formula, –n  1 be the chosen time granularity, –RG(CPTA,  P, n) be the region graph of CPTA,  P, n CPTA ⊨ P  P if RG(CPTA,  P, n) ⊨  –where ⊨ and  are “untimed” versions of ⊨ P and  p Key result of Kwiatkowska et al. (2000)

Model checking continuous probabilistic timed automata Replacing probabilistic choice with nondeterministic choice introduces the possibility of an error in the computed probabilities But we know that the maximum probability that CPTA satisfies a path formula is bounded from above by the maximum probability that the RG(CPTA,  P, n) satisfies the path formula (similar with minimum) For example: CPTA ⊨ P P <0.01 [ ⃟ failure] if RG(CPTA,  P, n) ⊨ P <0.01 [ ⃟ failure]

Conclusions: model checking timed automata Achieved success in the form of the development of tools such as UPPAAL (Uppsala/Aalborg) and KRONOS (Grenoble) Use of zone-based algorithms –Manipulate sets of clock equivalence classes

Conclusions: model checking real- time probabilistic processes Activity died off after Alur, Courcoubetis and Dill’s 1991 papers Interest renewed by the development of process algebras with generally distributed delays (Bravetti et al., D’Argenio et al) Model checking of Semi-Markov Chains: Infante-Lopez et al. (2001)

Conclusions: model checking probabilistic timed automata Model checking using PRISM (Kwiatkowska, Norman and Parker (2002)) and: –Region graphs –Discrete-time semantics (given restrictions on clock constraints to x  c and x  c) Based on discrete-time semantics for timed automata developed by Henzinger et al. (1992), Asarin et al. (1998), Bozga et al. (1999) Case studies: FireWire (Kwiatkowska et al. (2002:FAC)), IEEE (Kwiatkowska et al. (2002:PAPM-PROBMIV))

Conclusions: model checking probabilistic timed automata Zone-based algorithms for probabilistic timed automata: –Must carefully distinguish zones which have different probabilities Kwiatkowska et al. (2001:CONCUR, 2002:TCS) –Case study: FireWire Kwiatkowska et al. (2002:FAC), Daws et al. (2002)

Conclusions: model checking continuous probabilistic timed automata Increasing the time granularity blows up the state space Exists a need to concentrate on restricted subclasses