1. MPRI 3 Dec 2007Catuscia Palamidessi 1 Why Probability and Nondeterminism? Concurrency Theory Nondeterminism –Scheduling within parallel composition –Unknown behavior of the environment –Underspecification Probability –Environment may be stochastic –Processes may flip coins

2. MPRI 3 Dec 2007Catuscia Palamidessi 2 Automata A = (Q, q 0, E, H, D) Transition relation D  Q  (E  H)  Q Internal (hidden) actions External actions: E  H =  Initial state: q 0  Q States

3. MPRI 3 Dec 2007Catuscia Palamidessi 3 Example: Automata A = (Q, q 0, E, H, D) coffee q0q0 q2q2 q1q1 q4q4 q3q3 q5q5 d n n n choc ch Execution: q 0 n q 1 n q 2 ch q 3 coffee q 5 Trace: n n coffee

4. MPRI 3 Dec 2007Catuscia Palamidessi 4 Probabilistic Automata PA = (Q, q 0, E, H, D) Transition relation D  Q  (E  H)  Disc(Q) Internal (hidden) actions External actions: E  H =  Initial state: q 0  Q States

5. MPRI 3 Dec 2007Catuscia Palamidessi 5 Example: Probabilistic Automata q0q0 q1q1 q2q2 q3q3 q4q4 q5q5 fair unfair flip 1/2 2/3 1/3 beep

6. MPRI 3 Dec 2007Catuscia Palamidessi 6 Example: Probabilistic Automata q0q0 qhqh qtqt qpqp flip 1/2 2/3 1/3 beep

7. MPRI 3 Dec 2007Catuscia Palamidessi 7 Example: Probabilistic Automata q0q0 q1q1 q2q2 q3q3 q4q4 q5q5 fair unfair flip 1/2 2/3 1/3 beep What is the probability of beeping?

8. MPRI 3 Dec 2007Catuscia Palamidessi 8 Example: Probabilistic Executions q0q0 q1q1 q3q3 q4q4 q5q5 1/2 q0q0 q2q2 q3q3 q4q4 q5q5 unfair flip 2/3 1/3 beep  (beep) = 2/3  (beep) = 1/2 fair beep flip 1/2 2/3

9. MPRI 3 Dec 2007Catuscia Palamidessi 9 Example: Probabilistic Executions q0q0 q1q1 q2q2 q3q3 q4q4 q3q3 q4q4 q5q5 q5q5 fair unfair flip beep 1/2 2/3 1/3 1/2 1/4 2/6 7/12

10. MPRI 3 Dec 2007Catuscia Palamidessi 10 Sample set –Set of objects  Sigma-field (  -field) –Subset F of 2  satisfying Inclusion of  Closure under complement Closure under countable union Closure under countable intersection Measure on ( , F ) –Function  from F to   For each countable collection { X i } I of pairwise disjoint sets of F,  (  I X i ) =  I  ( X i ) (Sub-)probability measure –Measure  such that  (  ) = 1 (  (  )  1) Sigma-field generated by C  2  –Smallest  -field that includes C Measure Theory Example: set of executions Study probabilities of sets of executions which sets can I measure?

11. MPRI 3 Dec 2007Catuscia Palamidessi 11 Measure Theory Why not F = 2  ? Flip a fair coin infinitely many times  = { h,t }   (  ) = 0 for each   ( first coin h ) = 1/2 Theorem: there is no probability measure on 2  such that  (  ) = 0 for each 

12. MPRI 3 Dec 2007Catuscia Palamidessi 12 Theorem A measure on cones extends uniquely to a measure on the  -field generated by cones q0q0 q1q1 q2q2 q3q3 q4q4 q3q3 q4q4 q5q5 q5q5 fair unfair flip beep 1/2 2/3 1/3 1/2 Cones and Measures Cone of  –Set of executions with prefix  –Represent event “  occurs” Measure of a cone –Product edges of  CC 

13. MPRI 3 Dec 2007Catuscia Palamidessi 13 Examples of Events Eventually action a occurs –Union of cones where action a occurs once Action a occurs at least n times –Union of cones where action a occurs n times Action a occurs at most n times –Complement of action a occurs at least n+1 times Action a occurs exactly n times –Intersection of previous two events Action a occurs infinitely many times –Intersection of action a occurs at least n times for all n Execution  occurs and nothing is scheduled after –Set consisting of  only –C  intersected complement of cones that extend 

14. MPRI 3 Dec 2007Catuscia Palamidessi 14 Schedulers - Resolution of nondeterminism Scheduler Function  : exec*(A)  Q x (E  H) x Disc(Q)  if  (  ) = (q,a, ) then q = lstate(  ) Probabilistic execution generated by  from state r Measure   r (C s ) = 0 if r  s   r   r (C r ) = 1    r (C  aq ) =   r (C  )  q ) if  (  ) = (q,a, )

15. MPRI 3 Dec 2007Catuscia Palamidessi 15 Probabilistic CCS P :: = 0 | P|P | .P | P + P | (  ) P | X | let X = P in X | P  p P Probabilistic processes .P  (P) Prefix  P 1  p P 2  p  1 + (1-p)  2  P  Nondeterministic process  P + Q  

16. MPRI 3 Dec 2007Catuscia Palamidessi 16 Probabilistic CCS Communication P  Interleaving  P 1  P 2   P|Q  |Q  P 1  (P 2 )P 2  (P2)(P2)  (P 2 | P 2 ) âa Hiding ( a) P   P   ( a)   Recursion   P[ let X = P in X / X ]   X   a, â  let X = P in

17. MPRI 3 Dec 2007Catuscia Palamidessi 17 Bisimulation Relations We have the following objectives They should extend the corresponding relations in the non probabilistic case Keep definitions simple Where are the key differences?

18. MPRI 3 Dec 2007Catuscia Palamidessi 18 Strong Bisimulation on Automata Strong bisimulation between A 1 and A 2 Relation R  Q x Q, Q=Q 1  Q 2, such that q0q0 q1q1 q3q3 q2q2 q4q4 s0s0 s1s1 s3s3 aa b a bb s s q q a a R R  q, s, a, q  s +

19. MPRI 3 Dec 2007Catuscia Palamidessi 19 Strong Bisimulation on Probabilistic Automata Strong bisimulation between A 1 and A 2 Relation R  Q x Q, Q=Q 1  Q 2, such that q0q0 q1q1 q3q3 q2q2 q4q4 s0s0 s1s1 s3s3 a b a bb 11  C  Q/ R.  ( C ) =  ( C ) s  q  a a R R  q, s, a,    1 1   R  R  + [LS89]

20. MPRI 3 Dec 2007Catuscia Palamidessi 20 Probabilistic Bisimulations These two Probabilistic Automata are not bisimilar Yet they satisfy the same formulas of a logic PCTL –The logic observes probability bounds on reachability properties Bisimilar if we match transitions with convex combinations of transitions q1q1 q2q2 q3q3 q2q2 q3q3 q2q2 q3q3.2.8.3.7.4.6 s1s1 s2s2 s3s3 s2s2 s3s3.2.8.4.6 aaaaa bcbcbcbcbc ~ ~p~p

21. MPRI 3 Dec 2007Catuscia Palamidessi 21 Weak Bisimulation on Automata Weak bisimulation between A 1 and A 2 Relation R  Q x Q, Q=Q 1  Q 2, such that q0q0 q1q1 q3q3 q2q2 q4q4 s1s1 s3s3   b bb s s q q a a R R  q, s, a, q  s + s  s   : trace(  )=a, fstate(  )=s, lstate(  )=s a

22. MPRI 3 Dec 2007Catuscia Palamidessi 22 Weak bisimulation on Probabilistic Automata q0q0 q1q1 q3q3 q2q2 q4q4 s1s1 s3s3  b bb Weak bisimulation between A 1 and A 2 Relation R  Q x Q, Q=Q 1  Q 2, such that s  q  a a R R  q, s, a,    111  C  Q/ R.  ( C ) =  ( C )   R  R  + [LS89]

23. MPRI 3 Dec 2007Catuscia Palamidessi 23 Weak Transition There is a probabilistic execution  such that –  ( exec* ) = 1 – trace (  ) =  ( a ) – fstate (  ) =  ( q ) – lstate (  ) =  q  a (it is finite) (its trace is a) (it starts from q ) (it leads to  ) q  s  iff    : trace(  )=a, fstate(  )=q, lstate(  )=s a

24. MPRI 3 Dec 2007Catuscia Palamidessi 24 Exercises Prove that the probabilistic CCS is an extension of CCS (to define what this means is part of the exercise) Prove that probabilistic bisimulation is an extension of bisimulation Write the Lehmann-Rabin algorithm in probabilistic CCS (without using guarded choice)