From Graph Models to Game Models Tom Henzinger EPFL

Graph Models of Systems vertices = states edges = transitions paths = behaviors

graph Extended Graph Models MULTIPLE ACTORS: game graph LIVENESS: -automaton PROBABILITIES: Markov decision process stochastic game regular game

Graphs vs. Games a ba a b a

Games model Open Systems Two players: environment / controller / input vs. system / plant / output Multiple players: processes / components / agents Stochastic players: nature / randomized algorithms

-synthesis [Church, Rabin, Ramadge/Wonham, Pnueli/Rosner] -receptiveness [Dill, Abadi/Lamport] -scheduling [Sifakis et al.] -reasoning about system components [Kupferman/Vardi et al.] -early error detection [deAlfaro/H/Mang] -model-based testing [Gurevich et al.] -interface compatibility [deAlfaro/H] -program repair [Bloem et al.] -etc. Applications of Graph Games

Example P1: init x := 0 loop choice | x := x+1 mod 2 | x := 0 end choice end loop S1: ( x = y ) P2: init y := 0 loop choice | y := x | y := x+1 mod 2 end choice end loop S2: ( y = 0 )

Graph Questions 8 ( x = y ) 9 ( x = y ) CTL

Graph Questions 8 ( x = y ) 9 ( x = y ) X CTL

Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) ATL [Alur/H/Kupferman]

Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) ATL [Alur/H/Kupferman]

Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) ATL [Alur/H/Kupferman] X

Zero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) ATL [Alur/H/Kupferman] X

Nonzero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) Secure equilibra [Chatterjee/H/Jurdzinski]

Nonzero-Sum Game Questions hhP1ii ( x = y ) hhP2ii ( y = 0 ) Secure equilibra [Chatterjee/H/Jurdzinski]

Winning Conditions Qualitative: -regular (safety; Buchi; parity) Quantitative: max; lim sup; lim avg

Quantitative Game Questions hhP1ii lim sup hhP1ii lim avg

Quantitative Game Questions hhP1ii lim sup = 3 hhP1ii lim avg

Quantitative Game Questions hhP1ii lim sup = 3 hhP1ii lim avg =

Many Open Problems Buchi (lim sup) games in subquadratic time ? Parity (lim avg) games in polynomial time ??

Solving Games by Value Iteration Generalization of the -calculus: computing fixpoints of transfer functions (pre; post). Generalization of dynamic programming: iterative optimization. q Region R: Q ! V q R(q)

Solving Games by Value Iteration Generalization of the -calculus: computing fixpoints of transfer functions (pre; post). Generalization of dynamic programming: iterative optimization. q Region R: Q ! V q R(q) R(q) := pre(R(q))

Q states transition labels : Q Q transition function = [ Q ! {0,1} ] regions with V = B 9 pre: q 9 pre(R) iff ( ) (q, ) R 8 pre: q 8 pre(R) iff ( ) (q, ) R Graph

acb 9 c =( X) ( c Ç 9pre(X) )

acb Graph 9 c =( X) ( c Ç 9pre(X) )

acb Graph 9 c =( X) ( c Ç 9pre(X) )

acb Graph 9 c =( X) ( c Ç 9pre(X) ) 8 c=( X) ( c Ç 8pre(X) )

Q 1, Q 2 states( Q = Q 1 [ Q 2 ) transition labels : Q Q transition function = [ Q ! {0,1} ] regions with V = B 1pre: q 1pre(R) iff q 2 Q 1 Æ ( ) (q, ) R or q 2 Q 2 Æ ( 8 2 ) (q, ) 2 R 2pre: q 2pre(R) iff q 2 Q 1 Æ ( 8 ) (q, ) R or q 2 Q 2 Æ ( 9 2 ) (q, ) 2 R Turn-based Game

c ab

c ab hh1ii c =( X) ( c Ç 1pre(X) )

c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) )

c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

c Turn-based Game ab hh1ii c =( X) ( c Ç 1pre(X) ) hh2ii c=( X) ( c Ç 2pre(X) )

Q 1, Q 2 states( Q = Q 1 [ Q 2 ) transition labels : Q N £ Q transition function = [ Q ! N ] regions with V = N 1pre: 1pre(R)(q) = (max ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 1 (min 2 ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 2 2pre: 2pre(R)(q) = (min ) max( 1 (q, ), R( (q, )) ) if q 2 Q 1 (max 2 ) max( 1 (q, ), R( 2 (q, )) ) if q 2 Q 2 Quantitative Game

c ab

c ab hh1ii 0 =( X) max( 0, 1pre(X) )

c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) )

c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) )

c Quantitative Game ab hh1ii 0 =( X) max( 0, 1pre(X) )

Q states 1, 2 moves of both players : Q 1 2 Q transition function = [ Q ! {0,1} ] regions with V = B 1pre: q 1pre(R) iff ( 1 1 ) ( 2 2 ) (q, 1, 2 ) R 2pre: q 2pre(R) iff ( 2 2 ) ( 1 1 ) (q, 1, 2 ) R Concurrent Game

acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1

acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 hh2ii c=( X) ( c Ç 2pre(X) )

acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 Concurrent Game hh2ii c=( X) ( c Ç 2pre(X) )

acb 1,11,21,11,2 2,12,22,12,2 1,11,22,21,11,22,2 2,12,1 Concurrent Game hh2ii c=( X) ( c Ç 2pre(X) ) Pr(1): 0.5 Pr(2): 0.5

Q states 1, 2 moves of both players : Q 1 2 Dist(Q) probabilistic transition function = [ Q ! [0,1] ] regions with V = [0,1] 1pre: 1pre(R)(q) = (sup 1 1 ) (inf 2 2 ) R( (q, 1, 2 )) 2pre: 2pre(R)(q) = (sup 2 2 ) (inf 1 1 ) R( (q, 1, 2 )) Stochastic Game [deAlfaro/Majumdar]

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0 10

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) 0 11

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) )

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) )

acb Pl.1 Pl.2 a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: Pl.1 Pl.2 a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Stochastic Game hh1ii c =( X) max( c, 1pre(X) ) limit 1 11

Solving Games by Value Iteration Safety: Buchi: Parity: … Many open questions: How do different evaluation orders compare? How fast do these algorithms converge? When are they optimal?

Q control locations transition labels Sprogram statements : Q S £ Q transition function Ppredicates = [ Q ! 2 P ] regions with V = 2 P 9 pre: p 9 pre(R)(q) iff ( ) ( wp[ (q, )] R( 2 (q, )) ) p ) Predicate Abstraction for Programs

Graph-based (finite-carrier) systems: Q = B m = boolean formulas [e.g. BDDs] pre = ( 9 x 2 B ) Timed and hybrid systems: Q = B m £ R n = formulas of ( Q, ·,+) [e.g. polyhedral sets] pre = ( 9 x 2 Q ) Beyond Graphs as Finite Carrier Sets

Summary Model checking is a very special (boolean) case of graph-based optimization problems. It can be generalized to solve much more general questions that involve multiple players, quantitative resources, probabilistic transitions, and continuous state spaces. The theory and practice of this is still wide open …