1. The Algorithmics of Probabilistic Automata Weak Bisimulation Andrea Turrini Saarland University based on a joint work with Holger Hermanns

2. Andrea Turrini Saarland University Probabilistic Automata (PA) s u hfhf tftf huhu n tutu flip retry coin 4/16 8/16 12/16 beep u beep f flash f flash u sms coin f vibrate sms

3. Andrea Turrini Saarland University Probabilistic Automata (PA) PA = (Q, q 0, E, H, D) Transition relation D   Q  (E   )  (Q) Internal (hidden) actions External actions: E  H =  Initial state: q 0  Q States   H H Disc

4. Andrea Turrini Saarland University Weak bisimulation p1-p Weak bisimulation between A 1 and A 2 Equiv. Rel. R  Q  Q Q = Q 1  Q 2, such that + bbb s0s0 s1s1 s2s2 t1t1 t3t3 s3s3 s4s4 r m   s, t, a,    s t RR a a  C    Q/R .   (C) =   (C)   R   [JL91]   m r

5. Andrea Turrini Saarland University Usual decision procedure Bisimulation(A 1, A 2 ) W := {Q 1  Q 2 } (C,  ) := FindSplit(W) while C   do W := Refine(W, , C) (C,  ) := FindSplit(W) return W + FindSplit(W) for each (s, , )  D 1  D 2 for each t  [s] W if there does not exist such that t and W return ([s] W,  ) return ( ,  )  r rm r m N N N N = max (| D 1  D 2 |, | Q 1  Q 2 |) ++ +

6. Andrea Turrini Saarland University Weak transition {beep: 5/16, flash: 7/16, vibrate: 4/16}{beep: 2/16, flash: 2/16, vibrate: 12/16} s u hfhf tftf huhu n tutu flip retry coin 4/16 8/16 12/16 beep u beep f flash f flash u coin f vibrate sms

7. Andrea Turrini Saarland University How many weak transitions? C 00 : bbbbb C 01 : bbbbt C 02 : bbbtb C 30 : ttttb C 31 : ttttt    

8. Andrea Turrini Saarland University Weak transition as Flow {beep: 5/16, flash: 7/16, vibrate: 4/16} s u hfhf tftf huhu n tutu flip retry coin 4/16 8/16 12/16 beep u beep f flash f flash u coin f vibrate sms

9. Andrea Turrini Saarland University Weak transition vs. Flow Problem s u hfhf tftf huhu n tutu flip retry coin 4/16 8/16 12/16 beep u beep f flash f flash u coin f vibrate {beep: 0/16, flash: 16/16, vibrate: 0/16} 16/16 sms

10. Andrea Turrini Saarland University Flow as Linear Programming Problem f 0 = 1 f 9 + f 12 = 5/16 f 10 + f 11 = 7/16 f 13 = 4/16 s u hfhf tftf huhu n tutu 8/16 beep u vibrate beep f flash f flash u f 8/16 12/164/16 0 ≤ f i ≤ c i (<  ) f 0 + f 8 = f 1 + f 4 f 1 = f 2 + f 3 f 2 = f 9 f 3 = f 10 f 4 = f 5 + f 6 + f 7 f 5 = f 8 + f 11 f 6 = f 12 f 7 = f 13 f1f1 f4f4 f0f0 f2f2 f3f3 f5f5 f6f6 f7f7 f8f8 f9f9 f 10 f 11 f 12 f 13 min  f i under constraints {beep: 5/16, flash: 7/16, vibrate: 4/16} f 2 = 8/16(f 2 + f 3 ) f 3 = 8/16(f 2 + f 3 ) f 5 = 12/16(f 5 + f 6 ) f 6 = 4/16(f 5 + f 6 ) beep flash vibrate Complexity: #variables  O(|Q|  |D|) #constraints  O(|Q|  |D|) Polynomial! s 1 8/16 4/16 1/16 3/16 0 ✓

11. Andrea Turrini Saarland University Usual decision procedure Bisimulation(A 1, A 2 ) W := {Q 1  Q 2 } (C,  ) := FindSplit(W) while C   do W := Refine(W, , C) (C,  ) := FindSplit(W) return W + FindSplit(W) for each (s, ,  )  D 1  D 2 for each t  [s] W if there does not exist  such that t  and  W  return ([s] W,  ) return ( ,  )  N N N N = max (| D 1  D 2 |, | Q 1  Q 2 |) ++ + Polynomial Polynomial!

12. Andrea Turrini Saarland University From bisimulation to minimal automaton A = (Q, q 0, E, H, D) QA = ( W, [q 0 ] W, E, H, D W ) mA = ( W, [q 0 ] W, E, H, D W ) Quotienting Transition reduction zeroconf-nt: |Q| = 670, |D| = 827 zeroconf-nt: |W| = 41, |D W | = 55 zeroconf-nt: |W| = 41, |D W | = 52

13. Andrea Turrini Saarland University Adding costs to PA c h0h0 h1h1 h n-1 hnhn p 1-p trtr … p trtr p trtr ds 11 r2r2 r2r2 r2r2 r2r2 Overall cost: 1 + nr 2 /p + 1 p 1-p trtr C(csh 0 ) = 1 x 1 C(h 0 t r h 1 ) = r 2 x p C(h 0 t r h 0 t r h 1 ) = (r 2 + r 2 ) x (1-p) x p C(h 0 t r h 0 t r h 0 t r h 1 ) = (r 2 + r 2 + r 2 ) x (1-p) x (1-p) x p C((h 0 t r ) i h 1 ) = (i + 1) x r 2 x (1-p) i x p r 2 / p

14. Andrea Turrini Saarland University Adding costs to PA c h0h0 h1h1 h n-1 hnhn p 1-p trtr … p trtr p trtr ds 11 r2r2 r2r2 r2r2 r2r2 Minimum cost equals optimal LP solution p 1-p trtr c h0h0 h1h1 h n-1 hnhn p 1-p … p p 1 1 r2r2 r2r2 r2r2 r2r2 p r2r2 r2r2 r2r2 r2r2 c 1 1 1 1 (1-p) / p 1 1 1 1 0 0 Polynomial!

15. Andrea Turrini Saarland University Nondeterminism and Cost PA 2 + 89/p  cost  2 + 100/p c h0h0 h1h1 h3h3 h4h4 p 1-p t5t5 h2h2 p t5t5 p t5t5 ds 11 5252 5252 5252 5252 p t5t5 q1q1 q3q3 q4q4 p t4t4 q2q2 p t4t4 p t4t4 4242 4242 4242 q5q5 p t4t4 4242 o o 0 0 o 0 o 0

16. Andrea Turrini Saarland University Comparing Cost PA  2 + 89/p 2 + 80/p w k0k0 k1k1 k3k3 k4k4 p 1-p t4t4 k2k2 p t4t4 p t4t4 ds 11 4242 4242 4242 4242 p t4t4 k5k5 p t4t4 4242 c h0h0 h1h1 h3h3 h4h4 p t5t5 h2h2 p t5t5 p t5t5 ds 11 5252 5252 5252 5252 p t5t5 q1q1 q3q3 q4q4 p t4t4 q2q2 p t4t4 p t4t4 4242 4242 4242 q5q5 p t4t4 4242 o o 0 0 o 0 o 0

17. Andrea Turrini Saarland University Minor Cost Weak Bisimulation

18. Andrea Turrini Saarland University Comparing Cost PA  2 + 89/p 2 + 80/p w k0k0 k1k1 k3k3 k4k4 p 1-p t4t4 k2k2 p t4t4 p t4t4 ds 11 4242 4242 4242 4242 p t4t4 k5k5 p t4t4 4242 c h0h0 h1h1 h3h3 h4h4 p t5t5 h2h2 p t5t5 p t5t5 ds 11 5252 5252 5252 5252 p t5t5 q1q1 q3q3 q4q4 p t4t4 q2q2 p t4t4 p t4t4 4242 4242 4242 q5q5 p t4t4 4242 o o 0 0 o 0 o 0

19. Andrea Turrini Saarland University Conclusion Weak transitions as LP problems Polynomial decision procedure for PA weak probabilistic bisimulation Minimal automaton Cost decorated PA Prototypical implementation Extensions of LP approach – Specific weak transitions, hyper-transitions, restricted transitions, equivalence matching, … – MDP multi-objective reachability – Interval multi-objective reachability – Multi-objective reachability in interval models Base for other models: Markov Automata

20. Andrea Turrini Saarland University Decision procedure in practice

21. Andrea Turrini Saarland University Usual decision procedure Bisimulation(A 1, A 2 ) W := {Q 1  Q 2 } (C,  ) := FindSplit(W) while C   do W := Refine(W, , C) (C,  ) := FindSplit(W) return W + FindSplit(W) for each (s, , )  D 1  D 2 for each t  [s] W if there does not exist such that t and W return ([s] W,  ) return ( ,  )  r rm r m N N N N = max (| D 1  D 2 |, | Q 1  Q 2 |) ++ +

22. Andrea Turrini Saarland University How many weak transitions? s u hfhf tftf huhu n tutu flip retry coin 4/16 8/16 12/16 beep u beep f flash f flash u coin f vibrate sms

23. Andrea Turrini Saarland University How many weak transitions? C 00 : bbbbb C 01 : bbbbt C 02 : bbbtb C 30 : ttttb C 31 : ttttt    

24. Andrea Turrini Saarland University Usual decision procedure Bisimulation(A 1, A 2 ) W := {Q 1  Q 2 } (C,  ) := FindSplit(W) while C   do W := Refine(W, , C) (C,  ) := FindSplit(W) return W + FindSplit(W) for each (s, ,  )  D 1  D 2 for each t  [s] W if there does not exist  such that t  and  W  return ([s] W,  ) return ( ,  )  N N N N = max (| D 1  D 2 |, | Q 1  Q 2 |) ++ + Polynomial Polynomial!

25. Andrea Turrini Saarland University From bisimulation to minimal automaton A = (Q, q 0, E, H, D) QA = ( W, [q 0 ] W, E, H, D W ) mA = ( W, [q 0 ] W, E, H, D W ) Quotienting Transition reduction zeroconf-nt: |Q| = 670, |D| = 827 zeroconf-nt: |W| = 41, |D W | = 55 zeroconf-nt: |W| = 41, |D W | = 52

26. Andrea Turrini Saarland University Weak transition as SMT problem f 0 = 1 f 9 + f 12 = 5/16 f 10 + f 11 = 7/16 f 13 = 4/16 f 2 = 8/16(f 2 + f 3 ) f 3 = 8/16(f 2 + f 3 ) f 5 = 12/16(f 5 + f 6 ) f 6 = 4/16(f 5 + f 6 ) 0 ≤ f i ≤ c i (<  ) f 0 + f 8 = f 1 + f 4 f 1 = f 2 + f 3 f 2 = f 9 f 3 = f 10 f 4 = f 5 + f 6 + f 7 f 5 = f 8 + f 11 f 6 = f 12 f 7 = f 13 min  f i under constraints assert declare-fun f i () Real