Decentralised Coordination of Mobile Sensors using the Max-Sum Algorithm Ruben Stranders, Alex Rogers, Nick Jennings School of Electronics and Computer Science University of Southampton {rs06r, acr,

2 This presentation focuses on the use of Max-Sum to coordinate mobile sensors Sensor Architecture & Max-Sum Empirical Evaluation Speeding up Max-Sum Model Value Coordinate

This work can be applied to improve situational awareness in dynamic scenarios Disaster Response Military Surveillance Climate Research

Our contribution is a coordination mechanism for a team of autonomous mobile sensors

These mobile sensors continuously monitor spatial phenomena

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

Limited Communication

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena No centralised control

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena No centralised control

To solve this coordination problem, we had to address three challenges 1.How to model the phenomena? 2.How to value potential samples? 3.How to coordinate to gather samples of highest value?

The three central challenges are clearly reflected in the architecture of our sensing agents Samples sent to neighbouring agents Samples received from neighbouring agents Information processing Model of Environment Outgoing negotiation messages Incoming negotiation messages Value of potential samples Action Selection Move Samples from own sensor Sensing Agent Raw samples Model Value Coordinate

These three challenges are clearly reflected in the architecture of our sensing agents Samples sent to neighbouring agents Samples received from neighbouring agents Information processing Model of Environment Outgoing negotiation messages Incoming negotiation messages Value of potential samples Action Selection Move Samples from own sensor Sensing Agent Raw samples Model

The sensors model the spatial phenomenon using the Gaussian Process Weak Strong Spatial Correlations

The sensors model the spatial phenomenon using the Gaussian Process Areas of Rapid Change

The sensors model the spatial phenomenon using the Gaussian Process Weak Strong Temporal Correlations

The value of a sample is determined how much it reduces uncertainty Samples sent to neighbouring agents Samples received from neighbouring agents Information processing Model of Environment Outgoing negotiation messages Incoming negotiation messages Value of potential samples Action Selection Move Samples from own sensor Sensing Agent Raw samples Value

The value of a sample is based on how much it reduces uncertainty But how to determine uncertainty reduction before collecting a sample?

The value of a sample is based on how much it reduces uncertainty But how to determine uncertainty reduction before collecting a sample? Prediction Confidence Interval Collected Sample Gaussian Process not only gives predictions, but also confidence intervals

The value of a sample is based on how much it reduces uncertainty But how to determine uncertainty reduction before collecting a sample? Prediction Confidence Interval Collected Sample Gaussian Process not only gives predictions, but also confidence intervals Potential Sample Location

The value of a sample is based on how much it reduces uncertainty But how to determine uncertainty reduction before collecting a sample? Prediction Confidence Interval Collected Sample Gaussian Process not only gives predictions, but also confidence intervals Measure of uncertainty

The value of a sample is based on how much it reduces uncertainty Prediction Confidence Interval Collected Sample Specifically, we can use information metrics such as Entropy, or Mutual Information

The sensor agents coordinate using the Max-Sum algorithm Samples sent to neighbouring agents Samples received from neighbouring agents Information processing Model of Environment Outgoing negotiation messages Incoming negotiation messages Value of potential samples Action Selection Move Samples from own sensor Sensing Agent Raw samples Coordinate

24 Max-Sum is a powerful algorithm for solving DCOPs Complete Algorithms DPOP OptAPO ADOPT Communication Cost Iterative Algorithms Best Response (BR) Distributed Stochastic Algorithm (DSA) Fictitious Play (FP) Max-Sum Algorithm Optimality

Max-Sum solves the social welfare maximisation problem in a decentralised way Mobile Sensors

Max-Sum solves the social welfare maximisation problem in a decentralised way Movement Parameters

Max-Sum solves the social welfare maximisation problem in a decentralised way Utility Functions

Max-Sum solves the social welfare maximisation problem in a decentralised way Localised Interaction

Max-Sum solves the social welfare maximisation problem in a decentralised way Social welfare: Mobile Sensors

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph Variable nodes Function nodes Agent 1 Agent 2 Agent 3

Max-Sum solves the social welfare maximisation problem by message passing Variable nodes Function nodes Agent 1 Agent 2 Agent 3

Max-Sum solves the social welfare maximisation problem by message passing From variable i to function j From function j to variable i

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph Sensor 1 Sensor 2 Sensor 3 Sensor 1 Sensor 2 Sensor 3

Variables represent the sensors’ movements Sensor 1 Sensor 2 Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample Sensor 1 Sensor 2 Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample Sensor 1 Sensor 2 Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample Sensor 1 Sensor 2 Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample Sensor 1 Sensor 2 Sensor 3

Functions represent the uncertainty reduction that results from collecting a sample Sensor 1 Sensor 2 Sensor 3

Unfortunately, the straightforward application of Max-Sum is too computationally expensive From variable i to function j From function j to variable i

Unfortunately, the straightforward application of Max-Sum is too computationally expensive From variable i to function j From function j to variable i Bottleneck!

Therefore, we developed two general pruning techniques that speed up Max-Sum Goal: Make as small as possible

Therefore, we developed two general pruning techniques that speed up Max-Sum Goal: Make as small as possible 1.Try to prune the action spaces of individual sensors 2.Try to prune joint actions

The first pruning technique prunes individual actions by identifying dominated actions

1. Neighbours send bounds ↑ [2, 2] ↓ [1, 1] ↑ [5, 6] ↓ [0, 1] ↑ [1, 2] ↓ [3, 4]

The first pruning technique prunes individual actions by identifying dominated actions 2. Bounds are summed [8, 10] [4, 7]

The first pruning technique prunes individual actions by identifying dominated actions 3. Dominated actions are pruned [8, 10] [4, 7]

We developed two general pruning techniques that speed up Max-Sum Goal: Make as small as possible 1.Try to prune the action spaces of individual sensors 2.Try to prune joint actions

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Action Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3

Action The second pruning technique reduces the joint action space because exhaustive enumeration is too costly Sensor 1Sensor 2Sensor 3 Etcetera…

The second pruning technique prunes the joint action space using branch and bound Sensor 1 Sensor 2 Sensor 3

The second pruning technique prunes the joint action space using branch and bound [7, 13][0, 4][2, 6] Sensor 1 Sensor 2 Sensor 3

The second pruning technique prunes the joint action space using branch and bound [7, 13][0, 4][2, 6] Sensor 1 Sensor 2 Sensor 3

The second pruning technique prunes the joint action space using branch and bound [7, 13][0, 4][2, 6] Sensor 1 Sensor 2 Sensor 3

The second pruning technique prunes the joint action space using branch and bound [7, 13][0, 4][2, 6] Sensor 1 Sensor 2 Sensor 3

This demonstration shows four sensors monitoring a spatial phenomenon

Sensors

This demonstration shows four sensors monitoring a spatial phenomenon Uncertainty Contours

This demonstration shows four sensors monitoring a spatial phenomenon

To empirically evaluate our algorithm, we measured speed up and prediction error Uncertainty Contours [7, 13][0, 4][2, 6]

The two pruning techniques combined prune 95% of the action space with 6 neighbouring sensors Number of neighbouring sensors % of joint actions pruned

Average RMSE Our Algorithm reduces Root Mean Squared Error of predictions up to 50% compared to Greedy

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors 1. Decentralised

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors 1. Decentralised 2. Fast % Pruned

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors 1. Decentralised 2. Fast 3. Accurate predictions % Pruned Prediction Error

For future work, we wish to extend the algorithm to do non-myopic planning

References R. Stranders, A. Farinelli, A. Rogers and N.R. Jennings (2009): Decentralised Coordination of Mobile Sensors Using the Max-Sum Algorithm. In: Proc 21st Int. Joint Conf on AI (IJCAI), Pasadena, USA. (In Press) R. Stranders, A. Farinelli, A. Rogers and N.R. Jennings (2009): Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm. 8th Proc. Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), Budapest. (In Press)

In conclusion, the use of Max-Sum leads to an effective coordination algorithm for mobile sensors 1. Decentralised 2. Fast 3. Accurate predictions % Pruned Prediction Error Questions?