1. Energy Parity Games Laurent Doyen LSV, ENS Cachan & CNRS Krishnendu Chatterjee IST Austria

2. Specificatio n Properties Model “What” Specificatio n Properties Model “What” Verification Synthesis System Implementatio n Algorithm “How” System Implementatio n Algorithm “How” Analysis - Synthesis Analysis

3. Specificatio n Properties Model “What” Specificatio n Properties Model “What” Verification Synthesis System Implementatio n Algorithm “How” System Implementatio n Algorithm “How” Analysis - Synthesis Analysis Reactive Systems (e.g. servers, hardware,…) Interact with environment Non-terminating Finite data (or data abstractions) Control-oriented

4. Two-player games on graphs Games for synthesis - Reactive system synthesis = finding a winning strategy in a two-player game - ω-regular spec : safety, reactivity, … - quantitative spec : resource constraints Game played on a finite graph - Infinite number of rounds - Player’s moves determine successor state - Outcome = infinite path in the graph

5. Two-player games

6. Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite

7. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

8. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

9. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

10. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

11. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

12. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

13. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

14. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

15. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

16. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Play:

17. Two-player games Player 1 (good guy) Player 2 (bad guy) Turn-based Infinite Strategies = recipe to extend the play prefix Player 1: Player 2: outcome of two strategies is a play

18. Two-player games on graphs Parity games Energy games QualitativeQuantitative ω-regular specifications (reactivity, liveness,…) Resource-constrained specifications

19. Two-player games on graphs Parity games Energy games Energy parity games QualitativeQuantitative Mixed qualitative-quantitative ω-regular specifications (reactivity, liveness,…) Resource-constrained specifications

20. Energy games

21. Energy game: Positive and negative weights

22. Energy games Play: Energy game: Positive and negative weights Energy level: 3, 3, 4, 4, 3, 2, 1,… (sum of weights)

23. Energy level: 3, 3, 4, 4, 3, 2, 1,… Energy games Energy game: Positive and negative weights 3 Initial credit A play is winning if the energy level is always nonnegative. Play: “Never exhaust the resource (memory, battery, …)”

24. Energy games Player 1 (good guy) Initial credit problem: Decide if there exist an initial credit c 0 and a strategy of player 1 to maintain the energy level always nonnegative.

25. Energy games Player 1 (good guy) Initial credit problem: Decide if there exist an initial credit c 0 and a strategy of player 1 to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

26. Energy games A memoryless strategy is winning if all cycles are nonnegative when is fixed. Initial credit problem: Decide if there exist an initial credit c 0 and a strategy of player 1 to maintain the energy level always nonnegative.

27. Energy games Initial credit problem: Decide if there exist an initial credit c 0 and a strategy of player 1 to maintain the energy level always nonnegative. A memoryless strategy is winning if all cycles are nonnegative when is fixed. See [CdAHS03, BFLMS08]

28. Energy games Initial credit problem: Decide if there exist an initial credit c 0 and a strategy of player 1 to maintain the energy level always nonnegative. c 0 =0 c 0 =1 c 0 =2 c 0 =3 A memoryless strategy is winning if all cycles are nonnegative when is fixed. Minimum initial credit can be computed in

29. Parity games

30. Parity game: integer priority on states

31. Play: Parity games Parity game: integer priority on states A play is winning if the least priority visited infinitely often is even. “Canonical representation of ω-regular specifications ” (e.g. all requests are eventually granted – G[r -> Fg]) 5, 5, 5, 1, 0, 2, 4, 5, 5, 3…

32. Parity games Player 1 (good guy) Decision problem: Decide if there exists a winning strategy of player 1 for parity condition.

33. Player 1 (good guy) For parity games, memoryless strategies suffice. Parity games Decision problem: Decide if there exists a winning strategy of player 1 for parity condition.

34. A memoryless strategy is winning if all cycles are even when is fixed. Parity games Decision problem: Decide if there exists a winning strategy of player 1 for parity condition.

35. A memoryless strategy is winning if all cycles are even when is fixed. Parity games Decision problem: See [EJ91]

36. Summary StrategyAlgorithmic Player 1Player 2complexity Energy gamesmemoryless NP  coNP Parity gamesmemoryless NP  coNP Energy games - “never exhaust the resource” Parity game – “always eventually do something useful”

37. Summary StrategyAlgorithmic Player 1Player 2complexity Energy gamesmemoryless NP  coNP Parity gamesmemoryless NP  coNP Energy parity games ??? Energy games - “never exhaust the resource” Parity game – “always eventually do something useful”

38. Energy parity games

39. “never exhaust the resource” Energy parity games: and “always eventually do something useful”

40. Energy parity games “never exhaust the resource” Energy parity games: and “always eventually do something useful” Energy games – a story of cycles Parity games – a story of cycles Energy parity games - ? Decide the existence of a finite initial credit sufficient to win. Decision problem:

41. A story of cycles A good cycle ?

42. A story of cycles A good cycle ? A bad cycle ? - energy is (strictly) negative - or, least priority is odd

43. A story of cycles A good cycle ? A bad cycle ? - energy is (strictly) negative - or, least priority is odd Player 1 looses energy parity game iff the opponent can force a bad cycle. The opponent can force bad cycles without memory.

44. A story of cycles In energy parity games, memoryless strategies are sufficient for Player 2. Proof

45. A story of cycles Proof Preliminary fact: under optimal strategy, energy in q is always greater than on first visit to q. In energy parity games, memoryless strategies are sufficient for Player 2.

46. A story of cycles Proof Assume player 1 looses with initial credit c r, then show that player 1 looses also against one of the “memoryless strategies in q”: In energy parity games, memoryless strategies are sufficient for Player 2. clcl crcr ≤ crcr

47. A story of cycles Proof Fix winning strategy of Player 2, all outcomes are loosing for Player 1: In energy parity games, memoryless strategies are sufficient for Player 2. crcr

48. Then also in since A story of cycles Proof Fix winning strategy of Player 2, all outcomes are loosing for Player 1: In energy parity games, memoryless strategies are sufficient for Player 2. Energy < 0 ? crcr

49. A story of cycles Least ∞-visited priority is odd ? Then also in or Proof Fix winning strategy of Player 2, all outcomes are loosing for Player 1: In energy parity games, memoryless strategies are sufficient for Player 2. crcr

50. Complexity StrategyAlgorithmic Player 1Player 2complexity Energy gamesmemoryless NP  coNP Parity gamesmemoryless NP  coNP Energy parity games memoryless NP  coNP

51. A story of cycles A good cycle ? A one-player energy Büchi game

52. A story of cycles A good cycle ? good for parity good for energy

53. A story of cycles Winning strategy: 1. reach and repeat positive cycle to increase energy; 2. visit priority 0; 3. goto 1;

54. A story of cycles Winning strategy: 1. reach and repeat O(2nW) times the positive cycle; 2. visit priority 0; 3. goto 1; requires exponential memory !

55. Complexity StrategyAlgorithmic Player 1Player 2complexity Energy gamesmemoryless NP  coNP Parity gamesmemoryless NP  coNP Energy parity games exponentialmemoryless NP  coNP

56. Note Energy-winning and parity-winning ≠ EnergyParity-winning

57. Structure of strategies

58. 00 0 1 (optimal) energy-winning ≥0 & min.pr. 1 parity-winning <0 & min.pr. 2

59. Structure of strategies 00 0 1 (optimal) energy-winning ≥0 & min.pr. 1 parity-winning <0 & min.pr. 2 Bad

60. Structure of strategies 01 1 2 energy-winning ≥0 & least priority is even if =0 parity-winning <0 & min.pr. 2

61. Good-for-energy strategies A good-for-energy strategy is a winning strategy in the following cycle-forming game: Cycle-forming game: play the game until a cycle is formed Player 1 wins if energy of the cycle is positive, or energy is 0 and least priority is even. If Player wins in an energy parity game, then a memoryless good-for-energy strategy exists. (not iff)

62. Winning strategy = alternate good-for-energy strategy and parity-winning strategy ? Good-for-energy strategies good-for-energy and parity-winning ≠ EnergyParity-winning Good-for-energy and parity-winning strategies are necessary to win…

63. 1.Play good-for-energy. Either energy stabilizes and then least priority is even, or energy (strictly) increases. 2.When energy is high enough (+2nW): 2a. If (and while) game is in Q \ Attr(0), play a winning strategy in subgame defined by Q \ Attr(0). 2b. Whenever game is in Attr(0), reach 0 and start over. If least priority is 0 Structure of strategies 0 Attr 1 (0) Win

64. If least priority is 1 Structure of strategies 1 Attr 2 (1) Win

65. 1.Game can be partitionned into winning regions and their attractor. 2.Winning strategy combines subgame winning stragies and reachability strategies. If least priority is 1 Structure of strategies 1 Attr 2 (1) W1W1

66. 1.Game can be partitionned into winning regions and their attractor. 2.Winning strategy combines subgame winning stragies and reachability strategies. If least priority is 1 Structure of strategies 1 Attr 2 (1) W1W1 W2W2 W3W3 Attr 1 (W 1 ) Attr 1 (W 2 ) Attr 1 (W 3 ) Corollary: memoryless strategies are sufficient in coBüchi energy games.

67. An NP solution Assume NP-algorithm for d-1 priorities NP-algorithm for d priorities: - guess the winning set and good-for-energy strategy - compute 0-attractor, and solve subgame in NP 0 Attr 1 (0) C d (n) ≤ p(n) + C d-1 (n) least priority 0

68. An NP solution Assume NP-algorithm for d-1 priorities NP-algorithm for d priorities: - guess the winning set and partition - compute 1-attractor, and solve subgames in NP 1 Attr 2 (1) W1W1 W2W2 W3W3 Attr 1 (W 1 ) Attr 1 (W 2 ) Attr 1 (W 3 ) C d (n) ≤ p(n) + C d-1 (n 1 ) + … + C d-1 (n k ) ≤ p(n) + C d-1 (n 1 + … + n k ) ≤ p(n) + C d-1 (n-1) least priority 1

69. Complexity StrategyAlgorithmic Player 1Player 2complexity Energy gamesmemoryless NP  coNP Parity gamesmemoryless NP  coNP Energy parity games exponentialmemoryless NP  coNP

70. Algorithm Good-for-energy: All cycles are either >0, or =0 and even Algorithm for solving energy parity games ? Determine good-for-energy winning states – a story of cycles Reduction to (pure) energy games with modified weights: cycles =0 and even  >0 cycles =0 and odd  0 or <0

71. Algorithm Reduction to energy games with modified weights: cycles =0 and even  >0 cycles =0 and odd  0 or <0 Positive increment for transitions from even states Negative increment for transitions from odd states Δenergy priority 0 1 2 4 3 0 1/n -1/n 2

72. Algorithm Positive increment for transitions from even states Negative increment for transitions from odd states Increment is exponential (in nb. of priorities) Δenergy priority 0 1 2 4 3 0 1/n -1/n 2

73. Algorithm Algorithm for solving energy parity games ? Determine good-for-energy winning states by solving modified-energy game in O(E.Q d+2.W) Recursive fixpoint algorithm, flavour of McNaughton-Zielonka Note: a reduction to parity games (making energy explicit) would give complexity O(E.(Q 2 W) d ).

74. Relationship with mean-payoff parity games

75. Two-player games on graphs Parity gamesEnergy games Mean-payoff games Energy parity games Mean-payoff parity games QualitativeQuantitative Mixed qualitative-quantitative

76. Mean-payoff Mean-payoff value of a play = limit- average of the visited weights Optimal mean-payoff value can be achieved with a memoryless strategy. Decision problem: Given a rational threshold, decide if there exists a strategy for player 1 to ensure mean-payoff value at least.

77. Mean-payoff Memoryless strategy σ ensures nonnegative mean-payoff value iff all cycles are nonnegative in G σ iff memoryless strategy σ is winning in energy game Mean-payoff games with threshold 0 are equivalent to energy games.

78. Mean-payoff parity Mean-payoff parity games [CHJ05] Objective: - satisfy parity condition - maximize mean-payoff Infinite memory may be necessary ! However, finite-memory ε-optimal strategies exist.

79. Mean-payoff parity Mean-payoff parity games are polynomially equivalent to energy parity games. Reduction idea: from MPP game, construct EP game by incrementing all weights by ε=1/(n+1) If Player 1 wins MPP ≥0, then finite-memory ε-optimal strategy exists, which is winning in EP game. If Player 1 wins EP game, then he wins MPP ≥ -ε and then also MPP ≥0 since the value in MPP has denomiantor ≤n.

80. Complexity StrategyAlgorithmic Player 1Player 2complexity Energymemoryless NP  coNP Paritymemoryless NP  coNP Energy parityexponentialmemoryless NP  coNP Mean-payoff parity infinitememoryless NP  coNP By-product: a conceptually simple algorithm for mean-payoff parity games.

81. Thank you ! Questions ? The end

82. References [CdAHS03]A.Chakrabarti, L. de Alfaro, T.A. Henzinger, and M. Stoelinga. Resource interfaces, Proc. of EMSOFT: Embedded Software, LNCS 2855, Springer, pp.117-133, 2003 [EM79]A. Ehrenfeucht, and J. Mycielski, Positional Strategies for Mean-Payoff Games, International Journal of Game Theory, vol. 8, pp. 109-113, 1979 [BFL+08]P. Bouyer, U. Fahrenberg, K.G. Larsen, N. Markey, and J. Srba, Infinite Runs in Weighted Timed Automata with Energy Constraints, Proc. of FORMATS: Formal Modeling and Analysis of Timed Systems, LNCS 5215, Springer, pp. 33- 47, 2008 [CHJ05]K. Chatterjee, T.A. Henzinger, M. Jurdzinski. Mean-payoff Parity Games, Proc. of LICS: Logic in Computer Science, IEEE, pp. 178-187, 2005.

83. Energy parity

84. Mean-payoff parity

85. Complexity StrategyAlgorithmic Player 1Player 2complexity Energymemoryless NP  coNP Paritymemoryless NP  coNP Energy parityexponentialmemoryless NP  coNP Mean-payoffmemoryless NP  coNP Mean-payoff parity infinitememoryless NP  coNP