1. Game Theoretical Insights in Strategic Patrolling: Model and Analysis Nicola Gatti – ngatti@elet.polimi.it DEI, Politecnico di Milano, Piazza Leonardo da Vinci 32, Milano, 20133, Italy

2. Nicola GattiECAI 2008 2 Topic, Results, and Outline Topic Study of strategic models for capturing patrolling situations in presence of opponents Main results Modeling result: Problems in the current state-of-the-art Proposal of an alternative model Computational result: Exploitation of game theoretical analysis for reducing the solving algorithm complexity Outline Strategic patrolling state-of-the-art Proposal of an alternative model Towards integration between game theoretical analysis and algorithmic game theory Conclusions and future works

3. Nicola GattiECAI 2008 3 Game Theory Groundings for Strategic Patrolling Definition of game Protocol: rules of the game (e.g., number of players, sequential structure, available actions) Strategic-form games: the players act simultaneously (e.g., rock- paper-scissors) Extensive-form games: the players act according to a given sequential structure (e.g., chess) Strategies: players’ behavior in the game Solution: a strategy profile σ = (σ 1, …, σ n ) that is somehow in equilibrium Nash equilibrium: the players act simultaneously without meeting themselves before playing the game [Nash, 1950] Leader-follower equilibrium: a player can commit to a specific strategy and the follower acts on the basis of the commitment [von Stengel and Zamir, 2004]

4. Nicola GattiECAI 2008 4 von Neumann’s Hide-and-Seek Game 123 456 789 S H S H H

5. Nicola GattiECAI 2008 5 von Neumann’s Hide-and-Seek Game (2) Game protocol: Two–player: Seeker Hider Zero–sum: 1: if the robber is caught -1: otherwise Strategic–form: the players act simultaneously Actions: Seeker: a row or a column Hider: a single cell Solution concept: Nash equilibrium Strategies: players randomize over all the possible actions

6. Nicola GattiECAI 2008 6 Paruchuri et al.’s Strategic Patrolling (1) 123 456 789 G R

7. Nicola GattiECAI 2008 7 Paruchuri et al.’s Strategic Patrolling (2) Assumptions: Time is discretized in turns Time needed by the guard to patrol one area is exactly 1 turn Time needed by the guard to move between two areas is negligible with respect to time needed to patrol an area Time needed by the robber to rob an area is d turns The robber can observe the strategy of the guard Game protocol: Two–player: Guard Robber General–sum: each player assigns each area and the robber’s caught a value Strategic–form: the players act simultaneously Actions: Guard: a route of d areas, e.g. Robber: a single area

8. Nicola GattiECAI 2008 8 Paruchuri et al.’s Strategic Patrolling (3) Solution concept: leader-follower equilibrium Strategies: the guard randomizes over a portion of the actions, while the robber follows a pure strategy Multiple types: the payoffs of the robber could be known with uncertainty by the guard By Harsanyi transformation: the robber can be of different types (each type has a specific payoff) according to a given probability distribution Solving algorithms: Multi Linear Programming [Conitzer and Sandholm, 2005] Mixed Integer Linear Programming [Paruchuri et al., 2008]

9. Nicola GattiECAI 2008 9 Problems in Paruchuri et al.’s Strategic Patrolling (1) robber guard 123 / 1, -1 0.66, 1 / 1,-10.66, 11, -1 / 0.66, 11, -1 A simple setting 3 areas 1 type Two turns are needed by the robber to rob an area (d=2) Each player has the same evaluations over the areas

10. Nicola GattiECAI 2008 10 Problems in Paruchuri et al.’s Strategic Patrolling (2) 123 R G Guard’s optimal strategy (.16,.16,.16,.16,.16,.16 ) Robber’s optimal strategy (2) realization R G The robber’s expected utility is -.33 realization G

11. Nicola GattiECAI 2008 11 Problems in Paruchuri et al.’s Strategic Patrolling (2) 123 R G Guard’s optimal strategy (.16,.16,.16,.16,.16,.16 ) realization R G The robber’s expected utility is.33 realization G

12. Nicola GattiECAI 2008 12 Problems in Paruchuri et al.’s Strategic Patrolling (3) The model by Paruchuri et al. does not consider all the possible implications due to the observation of the robber According to the assumption of observation, the robber can enter an area when the guard is patrolling and not exclusively when the guard starts to patrol a route

13. Nicola GattiECAI 2008 13 An Alternative Strategic Patrolling Model The “natural” model is an extensive-form game wherein Guard: the next area to patrol Robber: the area to enter or wait In this work we search for a strategic-form model alternative to Paruchuri et al.’s model The proposed model is a strategic-form model wherein Guard: the next area to patrol Robber: the area to enter and the guard’s strategy will be the same at each turn In this way the robber cannot improve its expected utility by waiting In this model no “consistency“problem there is (the proof can be found in the paper)

14. Nicola GattiECAI 2008 14 An Alternative Strategic Patrolling Model (2) The proposed model reduces the complexity of the game since the number of actions of the guard are exactly the number of areas (while in Paruchuri et al. it is the number of possible routes of d areas) Preliminary further results shows that when either the topology is complete or time to move between two areas is negligible the extensive-form game and the strategic-form game are equivalent in term of optimal solution

15. Nicola GattiECAI 2008 15 Searching for a Nash Equilibrium We use the strategic patrolling as case study for the integration of game theoretical analysis and algorithmic game theory Idea Game theoretical analysis allows one to derive some insights Singularities: some strategy profiles are never of equilibrium independently of the values of the parameters (payoffs) Regularities: some strategy profiles are of equilibrium with a probability higher than others These insights can be exploited to improve searching efficiency and to make hard problems affordable

16. Nicola GattiECAI 2008 16 One Robber Type Analysis Proposition 1: Independently of the number of the robber’s types, at the equilibrium the guard will randomize over all the possible actions On the basis of Proposition 1, except for a null-measure subspace of the parameters, with one type of robber the Nash equilibrium: Is unique, and Prescribes that both the guard and the robber will randomize over all their available actions In this case the Nash equilibrium can be computed in closed form as a single problem of linear programming

17. Nicola GattiECAI 2008 17 More Robber Types Analysis (1) With more types, the equilibrium cannot be computed in closed form Anyway, game theoretical insights can be exploited to reduce the complexity of the search Searching in the space of the supports A complete method for searching a Nash equilibrium is to enumerate all the possible strategy supports and check them one by one (A strategy support is the set of actions over which agents randomize with a strict positive probability) Anyway, such a space rises exponentially in the number of players’ actions and then heuristics are needed [Porter et al., 2005] provides some heuristics for ordering the supports and shows that their approach is more efficient than Lemke-Howson algorithm

18. Nicola GattiECAI 2008 18 More Robber Types Analysis (2) By Proposition 1, the support of the guard will be the whole set of actions The supports of all the robber’s types can depict as a matrix M = By game theoretical analysis we can: Reduce the space of the matrices M Produce an ordering where the first Ms are the most probable to lead to an equilibrium Area 1…Area n Type 11…0 Type 20…1 ………… Type m1…0

19. Nicola GattiECAI 2008 19 Experimental Results We have studied random settings with 4, 5, 6, 7 areas and different number of robber’s types Our approach outperforms Porter et al. approach in term of computational time, dramatically reducing the space of the search Our approach outperforms Multi-LP algorithm, although the computation of a Nash equilibrium is harder than the computation of a leader-follower equilibrium types (with 4 areas) 6789101112 Porter23.1567.14132.31301.20621.41>1000 Ours0.1900.3520.7201.0151.5321.8522.231

20. Nicola GattiECAI 2008 20 Conclusions and Future Works Conclusions Analysis of state-of-the-art model of strategic patrolling Proposal of a strategic model in normal-form Attempt to exploit game theoretical analysis to improve the algorithm efficiency Future works Patrolling models and solving algorithms Exploiting game theoretical analysis in algorithmic game theory

21. Nicola GattiECAI 2008 21 Thank you for your attention!