1. 19 th November 2008 Agent-Based Decentralised Control of Complex Distributed Systems Alex Rogers School of Electronics and Computer Science University of Southampton acr@ecs.soton.ac.uk http://users.ecs.soton.ac.uk/acr/

2. 19 th November 2008 Contents Agent-Based Decentralised Control Cooperative Systems –Local Message Passing Algorithms –Max-sum algorithm –Graph Colouring –Wide Area Surveillance Scenario Competitive Systems –Game Theory –Mechanism Design –Eliciting Effort in Open Information Systems Decentralised Energy Systems

3. 19 th November 2008 Electronics and Computer Science 5* for Electrical and Electronic Engineering 5* for Computer Science 100 academic staff 36 professors 150 research fellows 250 PhD students Research grant income: £15 million per annum £10 million from UK Research Councils

4. 19 th November 2008 Intelligence, Multimedia and Agents Research Group Design and application of computing systems for complex information and knowledge processing tasks Agent-Based Computing Digital Libraries Decentralised Information Systems E-Business Technologies Grid and Distributed Computing Human Computer Interaction Web Science Knowledge Technologies Trust and Provenance

5. 19 th November 2008 Contents Agent-Based Decentralised Control Cooperative Systems –Local Message Passing Algorithms –Max-sum algorithm –Graph Colouring –Wide Area Surveillance Scenario Competitive Systems –Game Theory –Mechanism Design –Eliciting Effort in Open Information Systems Decentralised Energy Systems

6. 19 th November 2008 Agent-Based Decentralised Control Agents Multiple conflicting goals and objectives Discrete set of possible actions

7. 19 th November 2008 Agent-Based Decentralised Control Sensors Multiple conflicting goals and objectives Discrete set of possible actions

8. 19 th November 2008 Agent-Based Decentralised Control Agents Multiple conflicting goals and objectives Discrete set of possible actions Some locality of interaction

9. 19 th November 2008 Agent-Based Decentralised Control Agents Maximise Social Welfare: Multiple conflicting goals and objectives Discrete set of possible actions Some locality of interaction

10. 19 th November 2008 Agent-Based Decentralised Control Cooperative Systems –All agents represent a single stakeholder –We have access to these agents (closed system) –We can design the strategies that the agents adopt and the mechanisms by which they interact Competitive Systems –Agents represent multiple stakeholders –We can not directly influence the strategies of the agents (open system) –We can only design the protocols and mechanisms by which they interact

11. 19 th November 2008 Cooperative Systems Agents Central point of control Decentralised control and coordination through local computation and message passing. Speed of convergence, guarantees of optimality, communication overhead, computability No direct communication Solution scales poorly Central point of failure

12. 19 th November 2008 Landscape of Algorithms Complete Algorithms DPOP OptAPO ADOPT Communication Cost Optimality Iterative Algorithms Best Response (BR) Distributed Stochastic Algorithm (DSA) Fictitious Play (FP) Greedy Heuristic Algorithms Message Passing Algorithms Sum-Product Algorithm

13. 19 th November 2008 Sum-Product Algorithm Variable nodes Function nodes Factor Graph A simple transformation: allows us to use the same algorithms to maximise social welfare: Find approximate solutions to global optimisation through local computation and message passing:

14. 19 th November 2008 Graph Colouring Agent function / utility variable / state Graph Colouring ProblemEquivalent Factor Graph

15. 19 th November 2008 Graph Colouring Equivalent Factor Graph Utility Function

16. 19 th November 2008 Max-Sum Calculations Variable to Function: Information aggregation Function to Variable: Marginal Maximisation Decision: Choose state that maximises sum of all messages

17. 19 th November 2008 Graph Colouring

18. 19 th November 2008 Optimality

19. 19 th November 2008 Communication Cost

20. 19 th November 2008 Robustness to Message Loss

21. 19 th November 2008 Hardware Implementation

22. 19 th November 2008 Wide Area Surveillance Scenario Dense deployment of sensors to detect pedestrian and vehicle activity within an urban environment. Unattended Ground Sensor

23. 19 th November 2008 Energy Constrained Sensors Maximise event detection whilst using energy constrained sensors: –Use sense/sleep duty cycles to maximise network lifetime of maintain energy neutral operation. –Coordinate sensors with overlapping sensing fields. time duty cycle t ime duty cycle

24. 19 th November 2008 Energy-Aware Sensor Networks

25. 19 th November 2008 Future Work Continuous action spaces –Max-sum calculations are not limited to discrete action space –Can we perform the standard max-sum operators on continuous functions in a computationally efficient manner? Bounded Solutions –Max-sum is optimal on tree and limited proofs of convergence exist for cyclic graphs –Can we construct a tree from the original cyclic graph and calculate an lower bound on the solution quality?

26. 19 th November 2008 Contents Agent-Based Decentralised Control Cooperative Systems –Local Message Passing Algorithms –Max-sum algorithm –Graph Colouring –Wide Area Surveillance Scenario Competitive Systems –Game Theory –Mechanism Design –Eliciting Effort in Open Information Systems Decentralised Energy Systems

27. 19 th November 2008 Competitive Systems Controlling open competitive systems is much more difficult –Global credit crisis Key challenges –Understanding the emerging macroscopic properties of a system of selfish competitive agents GAME THEORY –Designing protocols and ‘rules of the game’ such that these macroscopic properties are desirable COMPUTATIONAL MECHANISM DESIGN

28. 19 th November 2008 Game Theory For a given ‘game’ –What action should a rational player take? –What is the equilibrium action of all players? Nash equilibrium A Beautiful Mind: Genius and Schizophrenia in the Life of John Nash Sylvia Nasar Faber and Faber

29. 19 th November 2008 Nash Equilibrium Two strategies s 1 and s 2 are in Nash equilibrium if: 1.under the assumption that agent i plays s 1, agent j can do no better than play s 2 ; and 2.under the assumption that agent j plays s 2, agent i can do no better than play s 1. Neither agent has any incentive to deviate from a Nash equilibrium

30. 19 th November 2008 Nash Equilibrium Column Player LEFTMIDDLERIGHT Row Player UP 4, 35, 16, 2 MIDDLE 2, 18, 43, 6 DOWN 3, 09, 62, 8 1 2 3 4 NE

31. 19 th November 2008 Computational Mechanism Design Mechanism design concern the analysis and design of systems in which the interactions between strategic, autonomous and rational agents leads to predictable global outcomes. –Design interactions to ensure the system has desirable and predictable Nash equilibrium Computational mechanism design –Limited communication –Incomplete information –Bounded computation

32. 19 th November 2008 Nash Equilibrium Column Player LEFTMIDDLERIGHT Row Player UP 4, 35, 16, 2 MIDDLE 2, 18, 43, 6 DOWN 3, 09, 62, 8 1 2 3 4 NE

33. 16 th September, 2004

35. 19 th November 2008 First Price Auction Desirable properties Efficiency Allocation Item assigned to the highest bidder Payment Pay bid ( ) Bidding strategy Shade bid Bayes Nash

36. 19 th November 2008 Second Price (Vickrey) Auction Desirable properties Efficiency Allocation Item assigned to the highest bidder Payment Pay second bid Bidding strategy Bid true valuation Dominant strategy

37. 19 th November 2008 Open Information System Information buyer requires a prediction of an uncertain Tomorrow’s temperature Requires certain minimum precision or “certainty” θ0θ0 θ c(θ) θ θ Identify cheapest provider Make prediction of precision of at least θ 0 Truthfully report this prediction to buyer Ensure provider’s utility is positive in expectation

38. 19 th November 2008 Two Stage Mechanism Two stage Mechanism: 1.Ask information producers to declare their costs 2.Ask cheapest producer to make measurement and reward him with a payment using a ‘strictly proper scoring rule’ calculated from the second lowest cost Payment is made once the event is verified Desirable system wide properties –Dominant strategy to truthfully declare costs Information buyer can always identify cheapest supplier –Dominant strategy to commit effort and truthfully reveal prediction

39. 19 th November 2008 Challenges Solution concepts –Mechanisms with dominant strategy solutions are rare –How do we automate the design process? Decentralised Mechanisms –Remove need for a central auctioneer Payment Free Mechanism –Non-transferable utility –Induce cooperative behaviour through reciprocity Iterated Prisoner’s Dilemma Trust and reputation models Match making mechanisms to pair producers and buyers

40. 19 th November 2008 Contents Agent-Based Decentralised Control Cooperative Systems –Local Message Passing Algorithms –Max-sum algorithm –Graph Colouring –Wide Area Surveillance Scenario Competitive Systems –Game Theory –Mechanism Design –Eliciting Effort in Open Information Systems Decentralised Energy Systems

41. 19 th November 2008 Micro-CHP Flywheel Storage Wireless Sensors Plug-in Hybrid Appliances 2016 Zero Carbon Home

42. 19 th November 2008 Energy Exchange

43. 19 th November 2008 How to coordinate energy use and make optimal energy trading decisions within the home to minimise energy consumption / costs? –Load management through smart appliances –Predicting load (occupancy, activity, weather conditions) –Understanding and learning thermal characteristics of home –Price prediction in external and local markets –Optimal use of storage devices –Optimal decisions to buy electricity / use CHP Research Questions

44. 19 th November 2008 What protocols and trading mechanisms generate desirable system wide properties? –Stable, predictable and low prices –Minimise CO2 emissions through flattening demand One day Research Questions

45. 19 th November 2008 Publications Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices using the Max-Sum Algorithm. In: Proceedings of the Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2008), pp. 639-646, Estoril, Portugal. Papakonstantinou, A., Rogers, A., Gerding, E. and Jennings, N. (2008) A Truthful Two-Stage Mechanism for Eliciting Probabilistic Estimates with Unknown Costs. In: Proceedings of the Eighteenth European Conference on Artificial Intelligence (ECAI 2008), pp. 448-452, Patras, Greece. R. K. Dash, N. R. Jennings, and D. C. Parkes. (2003) Computational Mechanism Design: A Call to Arms. IEEE Intelligent Systems, pages 40–47.

46. 19 th November 2008 Questions Thank you for your attention.