1. Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick Jennings School of Electronics and Computer Science University of Southampton {rs06r, af2, acr, nrj}@ecs.soton.ac.uk

2. 2 This presentation focuses on the use of Max-Sum in coordination problems with continuous parameters From Discrete to Continuous Max-Sum for Decentralised Coordination Empirical Evaluation

3. 3 Max-Sum is a powerful algorithm for solving DCOPs Complete Algorithms DPOP OptAPO ADOPT Communication Cost Iterative Algorithms Best Response (BR) Distributed Stochastic Algorithm (DSA) Fictitious Play (FP) Max-Sum Algorithm Optimality

4. Max-Sum solves the social welfare maximisation problem in a decentralised way Agents

5. Max-Sum solves the social welfare maximisation problem in a decentralised way Control Parameters

6. Max-Sum solves the social welfare maximisation problem in a decentralised way Utility Functions

7. Max-Sum solves the social welfare maximisation problem in a decentralised way Localised Interaction

8. Max-Sum solves the social welfare maximisation problem in a decentralised way Agents Social welfare:

9. The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph Variable nodes Function nodes Agent 1 Agent 2 Agent 3

10. Max-Sum solves the social welfare maximisation problem by message passing Variable nodes Function nodes Agent 1 Agent 2 Agent 3

11. Max-Sum solves the social welfare maximisation problem by message passing From variable i to function j From function j to variable i

12. Until now, Max-Sum was only defined for discretely valued variables Graph Colouring

13. However, many problems are inherently continuous. Heading and Velocity Unattended Ground Sensor Activation Time Autonomous Ground Robot Thermostat Preferred Room Temperature

14. So, we extended the Max-Sum algorithm to operate in continuous action spaces Discrete Continuous

15. We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)

16. “Continuous” Graph Colouring

17. A CPLF is defined by a domain partitioning followed by value assignment

20. To make Max-Sum work on CPLFs, we need to define key two operations on them From variable i to function j From function j to variable i

21. To make Max-Sum work on CPLFs, we need to define key two operations on them From variable i to function j From function j to variable i 1.Addition of two CPLFs

22. To make Max-Sum work on CPLFs, we need to define key two operations on them From variable i to function j From function j to variable i 2. Marginal Maximisation to a single variable

23. Addition of two CPLFs involves merging their domains, and then summing their values

24. 1. Merge domains

25. Addition of two CPLFs involves merging their domains, and then summing their values

26. 2. Sum Values

27. Marginal maximisation is the operation of finding the maximum value of a function, if we fix all but one variable From function j to variable i:

28. Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable

29. Example: bivariate function:

30. Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction Project onto axis

31. Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction Project onto axis

32. Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction Project onto axis Result of projection

33. Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction Extract Upper Envelope

34. Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction Extract Upper Envelope

35. We empirically evaluated this algorithm in a wide- area surveillance scenario Dense deployment of sensors to detect activity within an urban environment. Unattended Ground Sensor

36. Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors time duty cycle Discrete time duty cycle time duty cycle Discretised time

37. Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors time duty cycle DiscreteContinuous time duty cycle time duty cycle time duty cycle time duty cycle time duty cycle

38. Continuous Max-Sum outperforms Discrete Max- Sum by up to 10% Discretisation Solution Quality (as fraction of optimal) Average Solution Quality over 25 Iterations

39. Total Message Size Continuous Max-Sum leads to more effective use of communication resources than Discrete Max-Sum Discretisation Total number of values exchanged between agents

40. In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum 1. No artificial discretisation time

41. In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum 1. No artificial discretisation 2. Better solutions time Solution Quality

42. In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum time 1. No artificial discretisation 2. Better solutions 3. Effective communication Solution Quality Message Size

43. For future work, we wish to extend the algorithm to arbitrary continuous functions For example, using Gaussian Processes

44. In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum time 1. No artificial discretisation 2. Better solutions 3. Effective communication Solution Quality Message Size Questions?