1. The Multiple Dimensions of Mean-Payoff Games
Laurent Doyen CNRS & LSV, ENS Paris-Saclay
RP 2017

2. The Multiple Dimensions of Mean-Payoff Games
Laurent Doyen CNRS & LSV, ENS Paris-Saclay
RP 2017

3. About Basics about mean-payoff games Algorithms & Complexity
Strategy complexity – Memory
Focus
Equivalent game forms
Techniques for memoryless proofs

4. Mean-Payoff

5. Mean-Payoff

6. Mean-Payoff

7. Mean-Payoff

8. Mean-Payoff
Switching policy to get average power (1,1) ?

9. Mean-Payoff
Switching policy to get average power (1,1) ?

10. Mean-Payoff
Mean-payoff value = limit-average of the visited weights

11. Mean-Payoff Switching policy
Infinite memory: (1,1) vanishing frequency in q0

12. Mean-Payoff
limit ?

13. Mean-payoff is prefix-independent
limit ?
limsup
liminf
Mean-payoff is prefix-independent

14. Mean-Payoff
Switching policy
Infinite memory: (1,1) for liminf

15. Mean-Payoff Switching policy
Infinite memory: (1,1) for liminf & (2,2) for limsup

16. Games

17. Two-player games

18. Two-player games Player 1 (maximizer) Player 2 (minimizer) Turn-based
Infinite duration

19. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

20. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

21. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

22. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

23. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

24. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

25. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

26. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

27. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

28. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

29. Two-player games Play: Player 1 (maximizer) Player 2 (minimizer)
Turn-based
Infinite duration
Play:

30. Two-player games Player 1 (maximizer) Player 2 (minimizer) Turn-based
Infinite duration
Strategies = recipe to extend the play prefix
Player 1:
Player 2:

31. Two-player games Player 1 (maximizer) Player 2 (minimizer) Turn-based
Infinite duration
Strategies = recipe to extend the play prefix
Player 1:
outcome of two strategies is a play
Player 2:

32. Mean-payoff games

33. Mean-payoff games Mean-payoff game:
positive and negative weights (encoded in binary)
Decision problem:
Decide if there exists a player-1 strategy to ensure mean-payoff value ≥ 0
Focus on decision problem of being positive
Two reasons:
Generalizes to multi-dimension
Allows to obtain the value in one dimension
Value problem:

34. Mean-payoff games Key ingredients:
identify memory requirement: infinite vs. finite vs. memoryless
solve 1-player games (i.e., graphs)
Key arguments for memoryless proof:
backward induction
shuffle of plays
nested memoryless objectives

35. Reduction to Reachability Games

36. Reduction to Reachability Games
Reachability objective: positive cycles (v ≥ 0)

37. Reduction to Reachability Games
Reachability objective: positive cycles (v ≥ 0)

38. Reduction to Reachability Games
Reachability objective: positive cycles (v ≥ 0)

39. Reduction to Reachability Games
Reachability objective: positive cycles (v ≥ 0)
If player 1 wins  only positive cycles are formed  mean-payoff value ≥ 0
If player 2 wins  only negative cycles are formed  mean-payoff value < 0
(Note: limsup vs. liminf does not matter)

40. Reduction to Reachability Games
Reachability objective: positive cycles (v ≥ 0)
Mean-payoff game  Ensuring positive cycles
Memoryless strategy transfers to finite-memory mean-payoff winning strategy

41. Strategy Synthesis
Memoryless mean-payoff winning strategy ?

42. Strategy Synthesis
Memoryless mean-payoff winning strategy ?

43. Strategy Synthesis Memoryless mean-payoff winning strategy ?
Progress measure: minimum initial credit to stay always positive

44. Strategy Synthesis Memoryless mean-payoff winning strategy ?
Progress measure: minimum initial credit to stay always positive

45. Strategy Synthesis Memoryless mean-payoff winning strategy ?
Progress measure: minimum initial credit to stay always positive

46. Strategy Synthesis Memoryless mean-payoff winning strategy ?
Progress measure: minimum initial credit to stay always positive

47. Strategy Synthesis Memoryless mean-payoff winning strategy ?
Choose successor to stay above minimum credit
minimum credit such that
Progress measure: minimum initial credit to stay always positive In
choose
such that

48. “Energy Game” (stay always positive, for some initial credit)
Strategy Synthesis
Memoryless mean-payoff winning strategy ?
“Energy Game” (stay always positive, for some initial credit)
Choose successor to stay above minimum credit
minimum credit such that
Progress measure: minimum initial credit to stay always positive In
choose
such that

49. Memoryless proofs Key arguments for memoryless proof:
backward induction
shuffle of plays
nested memoryless objectives

50. Energy Games
Energy: min-value of the prefix. (if positive cycle; otherwise ∞)
Mean-payoff: average-value of the cycle.

51. Energy Games
Winning strategy ?

52. Energy Games
Winning strategy ?
Follow the minimum initial credit !

53. Multi-dimension games

54. Multi-dimension games
Multiple resources
Energy: initial credit to stay always above (0,0)
Mean-payoff:

55. Multi-dimension games
Multiple resources
Energy: initial credit to stay always above (0,0)
Mean-payoff:
same ?
same as positive cycles ?

56. Multi-dimension games
Multiple resources
Energy: initial credit to stay always above (0,0)
Mean-payoff:
same ?
same as positive cycles ?
If player 1 can ensure positive simple cycles, then energy and mean-payoff are satisfied.
Not the converse !

57. Multi-dimension games
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient

58. Multi-dimension games
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient
Then σ’1 is winning
and finite memory
Let σ1 be winning
On each branch L1
...
stop and play
as from L1 !
... L2
...
...
...
...
...
...
...
...
...
...
With L1≤L2
(ℕd,≤) is well-quasi ordered
wqo + Koenig’s lemma

59. Multi-energy games
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient
For player 2 ?

60. Multi-energy games For player 2, memoryless strategies are sufficient
induction on player-2 states
if  initial credit against all memoryless strategies, then  initial credit against all arbitrary strategies.
‘left’ game
‘right’ game

61.  Multi-energy games cl cr
For player 2, memoryless strategies are sufficient
induction on player-2 states
if  initial credit against all memoryless strategies, then  initial credit against all arbitrary strategies. cl
‘left’ game cr
‘right’ game
cl+cr 
Play is a shuffle of left-game play and right-game play
Energy is sum of them

62.  Multi-energy games cl cr
For player 2, memoryless strategies are sufficient
induction on player-2 states
if  initial credit against all memoryless strategies, then  initial credit against all arbitrary strategies. cl
‘left’ game cr
‘right’ game
cl+cr 
Value against memoryless strategies
Value against arbitrary strategies
Play is a shuffle of left-game play and right-game play
Energy is sum of them
In general, we need

63. Memoryless proofs Key arguments for memoryless proof:
backward induction
shuffle of plays
nested memoryless objectives

64. Multi-energy games
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient
For player 2, memoryless strategies are sufficient
coNP ?

65. Multi-energy games not necessarily simple cycle!
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient
For player 2, memoryless strategies are sufficient
coNP ?
not necessarily simple cycle!
guess a memoryless strategy π for Player 2
Construct Gπ
check in polynomial time that Gπ contains no cycle with nonnegative effect in all dimensions

66. Multi-weighted energy games
Detection of nonnegative cycles  polynomial-time
Flow constraints using LP
Divide and conquer algorithm

67. Multi-weighted energy games
Detection of nonnegative cycles  polynomial-time
Flow constraints using LP

68. Multi-weighted energy games
Detection of nonnegative cycles  polynomial-time
Flow constraints using LP
Not connected !

69. Multi-weighted energy games
Detection of nonnegative cycles  polynomial-time
Flow constraints using LP
Divide and conquer algorithm
Mark the edges that belong to some (pseudo) solution.
Solve the connected subgraphs.

70. Multi-weighted energy games
Detection of nonnegative cycles  polynomial-time
Flow constraints using LP
Divide and conquer algorithm
Mark the edges that belong to some (pseudo) solution.
Solve the connected subgraphs.

71. Multi-dimension games
If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient
For player 2, memoryless strategies are sufficient
Equivalent with mean-payoff games (under finite-memory):
If player 1 wins  positive cycles are formed  mean-payoff value ≥ 0
Otherwise, for all finite-memory strategy of player 1 (with memory M), player 2 can repeat a negative cycle (in G x M)

72. Multi-dimension games
Player 1
Energy
MP - liminf
MP - limsup
Finite memory .
coNP-complete Player 2 memoryless
Infinite memory .

73. Multi-dimension games
Player 1
Energy
MP - liminf
MP - limsup
Finite memory .
coNP-complete Player 2 memoryless
Infinite memory .
coNP-complete Pl. 2 memoryless
Player 2 memoryless (shuffle argument)
Graph problem in PTIME (LP argument)
True for
False for

74. Multi-mean-payoff games
The winning region R of player 1 has the following characterization:
Player 1 wins from every state in R if and only if player 1 wins each from every state in R
(without leaving R)
Proof idea:

75. Multi-mean-payoff games
The winning region R of player 1 has the following characterization:
Player 1 wins from every state in R if and only if player 1 wins each from every state in R
(without leaving R)
Proof idea: L
Losing for player 1 for single objective
Attr2(L)
Winning for player 2, with memoryless strategy
By induction, player 2 is memoryless in the subgame

76. Memoryless proofs Key arguments for memoryless proof:
backward induction
shuffle of plays
nested memoryless objectives

77. Multi-dimension games
Player 1
Energy
MP - liminf
MP - limsup
Finite memory .
coNP-complete Player 2 memoryless
Infinite memory .
coNP-complete Pl. 2 memoryless
NP  coNP
Pl. 2 memoryless

78. Window games Issues with mean-payoff limsup vs. liminf
limit-behaviour, unbounded delay
complexity

79. Window games Issues with mean-payoff limsup vs. liminf
limit-behaviour, unbounded delay
complexity
unbounded window
Sliding window of size at most B
At every step, MP ≥ 0 within the window

80. Window games Window objective:
from some point on, at every step, MP ≥ 0 within window of B steps
prefix-independent
bounded delay
Implies the mean-payoff condition

81. Window games Window objective:
from some point on, at every step, MP ≥ 0 within window of B steps
prefix-independent
bounded delay
Implies the mean-payoff condition
Complexity, Algorithm ?
like coBüchi objective
min-max cost (for ≤B steps)
stable set (safety)
attractor & subgame iteration

82. Window games Window objective:
from some point on, at every step, MP ≥ 0 within window of B steps
prefix-independent
bounded delay
Implies the mean-payoff condition
Complexity, Algorithm ?
like coBüchi objective
multi-dimension: EXPTIME-complete

83. Hyperplane Separation
Multi-dimension mean-payoff (liminf): coNP-complete
Naive algorithm: exponential in number of states
Hyperplane separation: reduction to single-dimension mean-payoff games

84. Hyperplane Separation
Multi-dimension
Single dimension 

85. Hyperplane Separation

86. Hyperplane Separation
Player 1 cannot ensure MPλ ≥ 0 for some λ 
Player 1 loses the multi-dimension game

87. Hyperplane Separation
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
Player 1 wins the multi-dimension game

88. Hyperplane Separation
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
Player 1 wins the multi-dimension game

89. Hyperplane Separation
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
Player 1 wins the multi-dimension game

90. Hyperplane Separation
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
Player 1 wins the multi-dimension game

91. Hyperplane Separation
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
Player 1 wins the multi-dimension game

92. Hyperplane Separation
Multi-dimension mean-payoff (liminf): coNP-complete
Naive algorithm: exponential in number of states
Hyperplane separation: reduction to single-dimension mean-payoff games
Player 1 wins the multi-dimension game
Player 1 wins MPλ ≥ 0 for all λ  (R+)d 
In fact, it is sufficient for player 1 to win for all λ  {0,…,(dnW)d+1}d
Pseudo-polynomial for fixed dimension M
Fixpoint algorithm:
- remove states if losing for some λ
- remove attractor (for player 2) of losing states
Solving O(nMd) mean-payoff games in O(nmM)
O(n2mMd+1)

93. Conclusion Multiple dimensions of mean-payoff games Reachability game
Energy game
Cycle-forming game
Multi-dimension mean-payoff games
Memoryless proofs
Other directions: parity condition, stochasticity, imperfect information

94. Credits
Energy/Mean-Payoff Games is joint work with Lubos Brim, Jakub Chaloupka, Raffaela Gentilini, Jean-Francois Raskin.
Multi-dimension Games is joint work with Krishnendu Chatterjee, Jean-Francois Raskin, Alexander Rabinovich, Yaron Velner.
Window games is joint work with Krishnendu Chatterjee, Michael Randour, Jean-Francois Raskin.
Other important contributions:
[BFLMS08] Bouyer, Fahrenberg, Larsen, Markey, Srba. Infinite Runs in weighted timed automata with energy constraints. FORMATS’08.
[BJK10] Brazdil, Jancar, Kucera. Reachability Games on extended vector addition systems with states. ICALP’10.
[CV14] Chatterjee, Velner. Hyperplane Separation Technique for Multidimensional Mean-Payoff Games. FoSSaCS’14.
[Kop06] Kopczynski. Half-Positional Determinacy of Infinite Games. ICALP’06.
[KS88] Kosaraju, Sullivan. Detecting cycles in dynamic graphs in polynomial time. STOC’88.

96. The end
Thank you !
Questions ?