1. From Graph Models to Game Models Tom Henzinger EPFL

2. Graph Models of Systems vertices = states edges = transitions paths = behaviors

3. graph Extended Graph Models MULTIPLE ACTORS: game graph LIVENESS: -automaton PROBABILITIES: Markov decision process stochastic game regular game

4. Graphs vs. Games a ba a b a

5. Games model Open Systems Two players: environment / controller / input vs. system / plant / output Multiple players: processes / components / agents Stochastic players: nature / randomized algorithms

6. -synthesis [Church, Rabin, Ramadge/Wonham, Pnueli/Rosner] -receptiveness [Dill, Abadi/Lamport] -scheduling [Sifakis et al.] -reasoning about system components [Kupferman/Vardi et al.] -early error detection [deAlfaro/H/Mang] -model-based testing [Gurevich et al.] -interface compatibility [deAlfaro/H] -program repair [Bloem et al.] -etc. Applications of Graph Games

7. Example P1: init x := 0 loop choice | x := x+1 mod 2 | x := 0 end choice end loop S1: ( x = y ) P2: init y := 0 loop choice | y := x | y := x+1 mod 2 end choice end loop S2: ( y = 0 )

8. Graph Questions 8 ( x = y ) 9 ( x = y ) CTL

9. Graph Questions 8 ( x = y ) 9 ( x = y ) 00 1011 01 X CTL

10. Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) ATL [Alur/H/Kupferman]

11. Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) 00 10 01 11 ATL [Alur/H/Kupferman]

12. Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) 00 10 01 11 ATL [Alur/H/Kupferman] X

13. Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) 00 10 01 11 ATL [Alur/H/Kupferman] X

14. Nonzero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) 00 10 01 11 Secure equilibra [Chatterjee/H/Jurdzinski]

15. Nonzero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) 00 10 01 11 Secure equilibra [Chatterjee/H/Jurdzinski]

16. Winning Conditions Qualitative: -regular (safety; Buchi; parity) Quantitative: max; lim sup; lim avg

17. Quantitative Game Questions hhP1ii lim sup hhP1ii lim avg 4 2 2 0 2 0 0 4 3

18. Quantitative Game Questions hhP1ii lim sup = 3 hhP1ii lim avg 4 2 2 0 2 0 0 4 3

19. Quantitative Game Questions hhP1ii lim sup = 3 hhP1ii lim avg = 1 4 2 2 0 2 0 0 4 3

20. Many Open Problems Buchi (lim sup) games in subquadratic time ? Parity (lim avg) games in polynomial time ??

21. Solving Games by Value Iteration Generalization of the -calculus: computing fixpoints of transfer functions (pre; post). Generalization of dynamic programming: iterative optimization. q Region R: Q ! V q R(q)

22. Solving Games by Value Iteration Generalization of the -calculus: computing fixpoints of transfer functions (pre; post). Generalization of dynamic programming: iterative optimization. q Region R: Q ! V q R(q) R(q) := pre(R(q))

23. Q states transition labels : Q Q transition function = [ Q ! {0,1} ] regions with V = B 9 pre: q 9 pre(R) iff ( ) (q, ) R 8 pre: q 8 pre(R) iff ( ) (q, ) R Graph

24. acb 9 c =( X) ( c Ç 9pre(X) )

25. acb Graph 9 c =( X) ( c Ç 9pre(X) )

26. acb Graph 9 c =( X) ( c Ç 9pre(X) )

27. acb Graph 9 c =( X) ( c Ç 9pre(X) ) 8 c=( X) ( c Ç 8pre(X) )

28. Q 1, Q 2 states( Q = Q 1 [ Q 2 ) transition labels : Q Q transition function = [ Q ! {0,1} ] regions with V = B 1pre: q 1pre(R) iff q 2 Q 1 Æ ( ) (q, ) R or q 2 Q 2 Æ ( 8 2 ) (q, ) 2 R 2pre: q 2pre(R) iff q 2 Q 1 Æ ( 8 ) (q, ) R or q 2 Q 2 Æ ( 9 2 ) (q, ) 2 R Turn-based Game

29. c ab

30. c ab hh1ii c =( X) ( c Ç 1pre(X) )

31. c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) )

32. c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

33. c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

34. c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

35. Q 1, Q 2 states( Q = Q 1 [ Q 2 ) transition labels : Q N £ Q transition function = [ Q ! N ] regions with V = N 1pre: 1pre(R)(q) = (max ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 1 (min 2 ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 2 2pre: 2pre(R)(q) = (min ) max( 1 (q, ), R( (q, )) ) if q 2 Q 1 (max 2 ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 2 Quantitative Game

36. c ab 0 1 2 5 3

37. c ab hh1ii 0 =( X) max( 0, 1pre(X) ) 0 1 2 5 3 0 0 0

38. c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) ) 0 1 2 5 3 1 0 0

39. c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) ) 0 1 2 5 3 1 2 0

40. c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) ) 0 1 2 5 3 2 2 0

41. Q states 1, 2 moves of both players : Q 1 2 Q transition function = [ Q ! {0,1} ] regions with V = B 1pre: q 1pre(R) iff ( 1 1 ) ( 2 2 ) (q, 1, 2 ) R 2pre: q 2pre(R) iff ( 2 2 ) ( 1 1 ) (q, 1, 2 ) R Concurrent Game

42. acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1

43. acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 hh2ii c=( X) ( c Ç 2pre(X) )

44. acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 Concurrent Game hh2ii c=( X) ( c Ç 2pre(X) )

45. acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 Concurrent Game hh2ii c=( X) ( c Ç 2pre(X) ) Pr(1): 0.5 Pr(2): 0.5

46. Q states 1, 2 moves of both players : Q 1 2 Dist(Q) probabilistic transition function = [ Q ! [0,1] ] regions with V = [0,1] 1pre: 1pre(R)(q) = (sup 1 1 ) (inf 2 2 ) R( (q, 1, 2 )) 2pre: 2pre(R)(q) = (sup 2 2 ) (inf 1 1 ) R( (q, 1, 2 )) Stochastic Game [deAlfaro/Majumdar]

47. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game

48. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0 10

49. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0 11

50. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0.8 11

51. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0.96 11

52. acb 1 1 2 2 Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: 0.8 1 1 2 2 Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) limit 1 11

53. Solving Games by Value Iteration Safety: Buchi: Parity: … Many open questions: How do different evaluation orders compare? How fast do these algorithms converge? When are they optimal?

54. Q control locations transition labels Sprogram statements : Q S £ Q transition function Ppredicates = [ Q ! 2 P ] regions with V = 2 P 9 pre: p 9 pre(R)(q) iff ( ) ( wp[ (q, )] R( 2 (q, )) ) p ) Predicate Abstraction for Programs

55. Graph-based (finite-carrier) systems: Q = B m = boolean formulas [e.g. BDDs] pre = ( 9 x 2 B ) Timed and hybrid systems: Q = B m £ R n = formulas of ( Q, ·,+) [e.g. polyhedral sets] pre = ( 9 x 2 Q ) Beyond Graphs as Finite Carrier Sets

56. Summary Model checking is a very special (boolean) case of graph-based optimization problems. It can be generalized to solve much more general questions that involve multiple players, quantitative resources, probabilistic transitions, and continuous state spaces. The theory and practice of this is still wide open …