1. Probabilistic Verification of Discrete Event Systems Håkan L. S. Younes

2. The Problem Given a model of a discrete event system, check if certain properties hold The model is a stochastic process (GSMP) Properties are expressed using a logic formalism (CSL)

3. Probabilistic Verification Verification of probabilistic properties “The probability of reaching a failure state within 60 minutes is less than 0.1” Probabilistic verification of properties “The probability of property P holding is at least 0.95”

4. Discrete Event System (DES) Event-driven system Discrete state changes at the occurrence of events Examples: Manufacturing systems Queueing systems Communication protocols

5. Why Probabilistic Verification? The dynamics of a DES is too complex for symbolic methods Use simulation to generate sample paths Use acceptance sampling to verify probabilistic properties

6. Stochastic Processes A stochastic process consists of a set of transitions (or events) a set of states S1S1 S2S2 S3S3 S4S4

7. Markov Processes The Markov assumption There is enough information in the current state to determine the future behavior S1S1 S2S2 S3S3 S4S4 1.0 0.4 0.6 1.0 0.2 0.8

8. Holding Times The holding time is the time spent in a state before an event occurs Holding times are positive random variables Can be discrete or continuous

9. Holding times are governed by exponential distributions 0.4 0.6 1.0 0.20.8 S1S1 S2S2 S4S4 S3S3 4 3 2 1 State Time Continuous-time Markov Chain (CTMC)

10. Semi-Markov Process Holding times are governed by arbitrary (positive) distributions 0.4 0.6 1.0 0.20.8 S2S2 S3S3 S1S1 S4S4

11. Generalized Semi-Markov Process (GSMP) Holding times can depend on the history 4 3 2 1 State Time 0.7 0.6 1.0 E2E2 E3E3 S2S2 S3S3 E1E1 E2E2 S1S1 E1E1 S4S4 0.4 1.0 0.5 0.3 E1E1 E2E2 E3E3 3.5 5.0 - - 1.5 2.0 3.5 5.0 - - - - - 1.5 2.0

12. Properties Qualitative “P will eventually hold on all future execution paths” Quantitative “P will hold before time t with probability at least  on future execution paths”

13. Problem Space GSMP CTMC Model QualitativeQuantitative [ACD91] [ASSB96],[BKH99] My Work Properties [EMSS89]* *Discrete time

14. Continuous Stochastic Logic (CSL) State formulas: a, ¬ ,  1   2, Pr  (  ) Truth value is determined in a single state Path formulas: X ,  1 U  t  2 Truth value is determined over an execution path

15. Execution Paths Current state + current clock settings = internal state The internal state contains enough information to determine the future behavior A sequence of internal states is an execution path S 0,C 0 S 1,C 1 S 2,C 2 t0t0 t2t2 t1t1

16. CSL Semantics (State Formulas) Atomic proposition: a Negation: ¬  Holds iff  does not hold in current state Conjunction:  1   2 Holds iff both  1 and  2 hold in current state

17. CSL Semantics (More State Formulas) Probabilistic statement: Pr  (  ) Holds iff  is true over at most a  proportion of execution paths starting in the current state

18. CSL Semantics (Path Formulas) Next state: X  Holds iff  holds in the next state along the current execution path Until:  1 U  t  2 Holds iff  2 becomes true in some state along the current execution path before time t, and  1 is true in all prior states

19. More on Until Consider the formula a U  17 b a,¬ba,¬ba,¬ba,¬ba,¬ba,¬bbb 23411 a,¬ba,¬b¬a,¬b¬a,¬ba,¬ba,¬bbb 234 a,¬ba,¬ba,¬ba,¬ba,¬ba,¬ba,¬ba,¬bb 234

20. Verifying Probabilistic Statements Verify Pr  (  ) Generate sample execution paths using discrete event simulation Verify  over each sample path If  is true, then we have a positive sample If  is false, then we have a negative sample Based on the proportion of positive samples, determine if Pr  (  ) holds

21. Sequential Hypothesis Testing True, false, or another sample? Hypothesis: Pr  (  )

22. Error Bounds Probability of false negative:  We say that Pr  (  ) is false when it is true Probability of false positive:  We say that Pr  (  ) is true when it is false

23. False negatives False positives Indifference Region  –  +  Indifference region False negatives False positives Actual probability of  holding Probability of accepting Pr  (  ) as true  1 –  

24. Accept Reject Continue sampling Accept Reject Continue sampling Graphical Representation of Statistical Test We can find an acceptance line and a rejection line given , , , and  Number of samples Number of positive samples

25. Verification of Nested Probabilistic Statements Suppose , in Pr  (  ), contains probabilistic statements True, false, or another sample?

26. Indirect Sampling Want samples from random variable X Can only get samples from Y such that Pr[Y=1|X=1]  1 –  ’ Pr[Y=0|X=1]   ’ Pr[Y=1|X=0]   ’ Pr[Y=0|X=0]  1 –  ’

27. Modified Test find an acceptance line and a rejection line given , , , ,  ’, and  ’: Accept Reject Continue sampling Number of samples Number of positive samples

28. Verification of Compound State Formulas To verify ¬  with error bounds  and  Verify  with error bounds  and  To verify  1   2  …   n with error bounds  and  Verify  1 though  n with error bounds  /n and  /n

29. Sequential Verification of Conjunction To verify  1   2  …   n with error bounds  and  1. Verify each  i with error bounds  and  ’ 2. Return false as soon as any  i is verified to be false 3. If all  i are verified to be true, verify each  i again with error bounds  and  /n 4. Return true iff all  i are verified to be true

30. Verification of Path Formulas To verify X  with error bounds  and  Verify  with error bounds  and  in the next state To verify  1 U  t  2 with error bounds  and  Convert to conjunction  1 U  t  2 holds if  2 holds in the first state, or if  2 holds in the second state and  1 holds in all prior state, …

31. More on Verifying Until Given  1 U  t  2, let n be the index of the first state more than t time units away from the current state Conjunction of n conjuncts c 1 through c n, each of size i Simplifies if  1 or  2, or both, do not contain any probabilistic statements

32. Example Verify Pr  0.05 (true U  200 dead) in S 1 S 1 : S 2 S 3 S 4 S 1 : Safe, at rest S 2 : Under attack, at rest S 3 : Under attack, jumping S 4 : Safe, jumping E 1 : Stork attacking E 2 : Frog killed E 3 : Start jumping E 4 : Stork retreats E 5 : Stop jumping E1E1 S1S1 E2E2 E4E4 S3S3 E2E2 E3E3 S2S2 E5E5 S4S4 26.3 - - - - 0.0 - 124.5 44.3 - - 26.3 26.3 - - - - 0.0 - 80.2 - 95.2 - 70.6 - 124.5 44.3 - - 26.3 - - - - - 150.8 - 80.2 - 95.2 - 70.6 89.9 - - - - 0.0 - 155.8 48.3 - - 89.9 89.9 - - - - 0.0 - 107.5 - 61.0 - 138.2 - 155.8 48.3 - - 89.9 - - - - 36.2 199.2 - 107.5 - 61.0 - 138.2 59.4 - - - - 235.4 - - - - 36.2 199.2 E 1 : E 2 : E 3 : E 4 : E 5 : t: 20604080100 2 4 6

33. Summary Algorithm for probabilistic verification of discrete event systems Sample execution paths generated using discrete event simulation Probabilistic properties verified using acceptance sampling Algorithm can be used in an anytime manner

34. Future Work Apply to hybrid dynamic systems Develop heuristics for formula ordering and parameter selection Use verification to aid policy generation for real-time stochastic domains