F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Finding the Optimal Strategies in Robotic Patrolling with Adversaries in Topologically-Represented Environments Francesco Amigoni, Nicola Basilico, Nicola Gatti

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Robotic Patrolling A patrolling strategy determines the path followed by the robot, usually the next cell to move to

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Randomized Patrolling Strategies The patroller should adopt an unpredictable patrolling strategy, randomizing over cells and trying to reduce the intrusion risk (Pita et al., AAMAS08) Randomized strategy: the robot determines the next cell according to a probability distribution

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Example: Randomized Strategy

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Patrolling Strategies with Adversaries Considering a model of the adversary (Agmon et al., AAMAS08, Paruchuri et al., AAMAS08) can provide the patrolling robot a larger expected utility than not considering it, i.e., it can lead to better strategies (Amigoni et al., IAT2008) Model of the adversary can include: its preferences over the possible targets, its knowledge about the patrollers strategy, …

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano The Problem The problem we addressed in this work: finding the optimal randomized patrolling strategy in a arbitrary environment while considering a model of the adversary Our approach applies to environments with arbitrary topology generalizing (Agmon et al., ICRA08) Agmon et al., ICRA08

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano The Basic Patrolling Model Time is discrete Environment: represented by a directed graph, e.g., a grid of cells or a topological map (Carpin et al., IROS08) Single patrolling robot It can move between adjacent nodes It can detect a possible intruder in its current node Single intruder It knows the strategy of the patrolling robot, for example because it can observe the patroller movements before attempting to intrude It can directly enter any node Penetration time d i is required to successfully complete an intrusion in a node i When attempting to penetrate in a node i at time t, the intruder can be detected during {t,t+1,…,t+ d i }

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano The Basic Patrolling Model In this scenario there are two actors that compete one with each other Natural approach: to study their behaviors in a game-theoretical framework The patrolling problem is modeled as a two-player game and optimal behaviors can be derived by finding equilibria of such game The type of game that best describes this scenario is the leader- follower game Two players The leader commits to a strategy The follower observes such commitment and acts as a best responder

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano The Basic Patrolling Model Final States The indruder enters node i at time t: If the patroller does not visit cell i in the interval {t,t+1,…,t+ d i } the intruder wins Otherwise the intruder is captured and the patroller wins The intruder never enters Utilities X i,Y i (i {1, 2, …, 13}) : patrollers and intruders utilities when the intruder successfully attacks node i X 0,Y 0 : patrollers and intruders utilities when the intruder is captured P P I I I I I I move(10) move(12) move(7) … P P P P P P wait enter(13) enter(1) … … … …… 1 time unit

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Objective The proposed method finds the probability distribution over the patroller movements, i.e., given the current node, finding the probability of moving in each adjacent node

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Solving the Game Two competing actors: we study their behaviors in a game-theoretical framework The patrolling problem can be modeled as a leader-follower game Two players The leader commits to a strategy The follower observes such commitment and acts as a best responder Patrolling strategy: A = {α i,j }, where α i,j is the probability of doing move(j) when i is the current node The optimal A can be derived by computing the equilibrium of the leader-follower game resorting to a bilevel optimization problem (Conitzer and Sandholm, 2006)

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Solving Algorithm We safely assume that the game will end, i.e., the intruder will enter We compute A such that the patrollers expected payoff is maximum This amounts to solve a bilinear optimization problem for every possible action of the intruder Game Model Optimal patrolling strategy that maximizes patrollers expected utility Solving algorithm If the above problem does not admit a solution, Step 2: Step 1: is there any strategy A such that the game will never end? Single bilinear feasibility problem If a solution is found, it is the best patrolling strategy and the intruder will never attempt to enter

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano An Example X 1 = 0.8 Y 1 = 0.2 d 1 = 7 X 5 = 0.5 Y 5 = 0.5 d 5 = 7 X 0 = 1 Y 0 = -1 X 1 = 0.8 Y 1 = 0.2 d 1 = 7 X 5 = 0.5 Y 5 = 0.3 d 5 = With this strategy the game never ends, i.e., the intruder will never enter

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Another Example X 1 = 0.8 Y 1 = 0.2 d 1 = 5 X 5 = 0.5 Y 5 = 0.3 d 5 = 4 X 1 = 0.8 Y 1 = 0.2 d 1 = 5 X 5 = 0.5 Y 5 = 0.3 d 5 = X 0 = 1 Y 0 = -1 With this strategy the intruder will try to enter in cell 1 when the patroller is in cell 5, the expected utility of the patroller is 0.819

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Model Extensions Augmented sensing capabilities: we introduce the range parameter Synchronized multirobot setting: a single patroller able to sense an arbitrary subset of cells X 4 = 0.8 Y 4 = 0.4 X 6 = 0.7 Y 6 = 0.5 X 12 = 0.8 Y 12 = 0.4 expected utility penetration time X 0 = 1 Y 0 = -1

F. Amigoni, N. Basilico, N. Gatti DEI, Politecnico di Milano Conclusions and Future Works We presented an approach to find optimal randomized patrolling strategies in arbitrary environments with adversaries Future Works Accounting for intruders movements and limited observation capabilities Extending our framework with multiple non-synchronized patrollers