1. Autonomous Learning Agents for Decentralised Data and Information Networks (ALADDIN) www.aladdinproject.org Dr. David Nicholson BAE Systems Bristol, UK David.Nicholson2@baesystems.com RISE 2008, January 7−8, Spain

2. Facts & Figures £5.5M funding (plus £1M in kind) –2/3 BAE Systems; 1/3 EPSRC –Started 1 st October 2005 –Duration 5 years (3 + 2) Funded manpower: –Research Fellows: 600 person months –Programmers: 100 person months –1 lecturer: Alex Rogers –13 PhD students

3. The Team School of Electronics and Computer Science (Lead: Prof. Nick Jennings, Director) Dept. Electrical and Electronic Engineering (Lead: Prof. Erol Gelenbe) & Dept. Mathematics (Lead: Prof. David Hand) Department of Engineering Science (Lead: Prof. Stephen Roberts) Department of Mathematics (Lead: Dr. David Leslie) Advanced Technology Centre Integrated System Technologies (Programme management, research, exploitation)

4. Overall Aims Develop techniques, methods and architectures for modelling, designing and building decentralised systems that can bring together information from variety of heterogeneous sources in order to take informed action Take total systems view on information and knowledge fusion and consider feedback between sensing, decision making and acting in such a system Achieve these objectives in environments in which: –Control is distributed. –Uncertainty, ambiguity, and bias are endemic. –Multiple (self-interested) stakeholders with different aims and objectives are present. –Resources are limited and continually vary during system’s operation. –Timeliness of action is important Demonstrate applicability in domain of disaster management

5. Conceptual Underpinning A4A4 A2A2 A5A5 A6A6 A3A3 A1A1 multi-agent processes: interacting with each other and their environment

6. Technical Approach Building Individual Agents –Information fusion –Bayesian inference –Decision theory –Reinforcement learning Building Multi-Agent Systems –Multi-agent systems –Game theory/Mechanism design –Mathematical modelling of collective behaviour Principled combination Total systems view Demonstrable applicability

7. Research Themes Individual Actors –Design of actors that can perform effectively in dynamic uncertain and multi-actor environments Multiple Actors –Way in which individual actors can interact in flexible ways to achieve individual and collective goals Architectures –Development of efficient and effective system architectures Applications –Development of disaster management demonstrators

8. 8 Applications

9. Urban rescue Building evacuation Demo Vignettes Information Agent Situational awareness

10. Situational Awareness Information agents to provide real-time environmental information around a disaster area: –Weather, temperature, visibility, noise, gas detection etc. –Exploit sensors already in the environment. –Built into an urban environment. –Scattered around a disaster area. –Access information through current internet protocols. Information Agent on Tablet

11. Solent Weather Sensors Analogue for future deployed sensors. –Wind speed, direction, air temperature, air pressure, tide height etc. –4 sensors in the Solent. –Updates every 5 minutes. –Data accessed through HTTP.

12. <sit:Reading rdf:about="&sit;sotonmet/winddir/reading/20060704113000" rdfs:value="235" sit:datetime="2006-07-04T11:30:00"> Data deployed on sensor websites in RDF format. Information agent collects data via HTTP - parses, stores and queries sensor reading using standard semantic web technology (Jena). Backgound mapping data collected in real-time from GoogleMaps. Current Implementation Information Agent

13. Demo Screenshot

14. RoboCup Rescue Initiated in the late 90s following the Hanshin/Awaji earthquake –Short term: support decision making in multi- agency response –Long term: realise human-robot response Open-source – universities, individuals Yearly competition –Agent strategies –Simulators / Infrastructure

15. RoboCup Rescue Simulator Models the aftermath of an earthquake Simulators – simulate civilian behaviour, fires, blockades. building collapse, traffic Agents – ambulance / police / fire Agents have to rescue civilians / clear blockades / put out fires

16. Platoon Agents Fire Brigades –put out fires –have limited water tank capacity –can collaborate to extinguish more quickly Ambulances –dig out civilians who are buried under rubble (takes a number of cycles)‏ –carry victims to a refuge –can collaborate to dig out more quickly Police Force –can remove obstructions on roads

17. Extended Simulator Viewer Fire Civilian Blockade Agents Kernel Simulators Omni-agent Omni-agent launches agents and all actions, sensing and communication passes through it. Actions forwarded to the kernel. Viewers can access global world view from kernel. Simulators model other actors (civilians) and evolving world state. Omni-agent connects with kernel as a viewer. Maintains compatibility with new versions of kernel, viewers and simulators. Can completely configure and tailor sensing and communication. Can extend and customise rescue scenarios.

18. Extended Capability Autorun, Strategies and Map Generation –Automatically launch strategies. –Run repeatable experiments. –Store statistics from different runs. Communication System –Implement completely configurable communication network. Broadcast, peer-to-peer. Lossy communication and corrupted messages. Bandwidth constraints and communication black-holes. Agent Sensing Abilities –Noisy sensing, different agent capabilities. Clustering –Automatic map clustering tools for implementing hierarchical strategies.

19. Demo Screenshots

20. Building Evacuation

21. 21 Architectures

22. Disaster Management Pyramid Disaster Management Enterprise Disaster Management Services Response Agencies Response Resources Non-Sentient Elements

23. Data Collection uncertain ambiguous incomplete Data Fusion alignment estimation combination Info Fusion context reasoning assessment RT1 Multi-Agent Decision-Making local goals resources coordination Strategic Planning global goals constraints RT2 reports and estimates information and decisions information

24. Decentralisation storage Info source peer-to peer communication network Info source inference and decision cycle

25. Data Collection uncertain ambiguous incomplete Data Fusion alignment estimation combination Info Fusion context reasoning assessment RT1 Multi-Agent Decision-Making local goals resources coordination Strategic Planning global goals constraints RT2 Research Focus: How to conflate functions and manage interactions to achieve global goals in a dynamic and uncertain environment ? centralised fully decentralised intra-function decentralised inter-function decentralised reports and estimates information and decisions information

26. 26 Individual Actors

27. Model the world view of an individual agent in the face of: –incomplete data –delayed data –bandwidth restrictions which then serves as basis on which: –to choose optimal action from a finite set of actions Taxonomy, experiments, evaluation of methods Aims from sensors or other agents

28. Specific Sub-Projects Develop separate approaches for constructing coherent world views with dynamic, multivariate streaming data subject to: –Missing values –Delays –Bandwidth restrictions Develop re-inforcement learning strategies to guide information acquisition Develop Bayesian approach to heterogeneous information representation in dynamic systems Develop an architecture for modelling exact inference for multi-agent systems in dynamic environments Make an empirical study of the decision making problems of exact and approximate inference mechanisms within a multi-agent system

29. Gaussian Processes missing correlated delayed active data selection.

30. Exploit Correlation

31. Faulty Sensors Faulty sensors 2 out of 6 sensors output temperatures 10% too high Faulty observations can spread, reinforce and thus create havoc in decentralised estimation networks basic fusion algorithm ALADDIN fusion algorithm

32. 32 Multiple Actors

33. Aims Incentivising Mechanisms –How to structure interactions so individuals are incentivised to behave in way that leads to desirable system properties –Use game theory and mechanism design Coordinated Problem Solving –How to form agile teams to tackle particular niches as and when they are needed –Use markets and coalition formation techniques to cope with unexpected resource allocation situations Modelling Collective Behaviour –Mathematical models of agent behaviours and emergent system properties resulting from the above technology –Use multi-dimensional Markov processes

34. Specific Sub-Projects Agents bidding in auctions Automated auction design Learning to share information to coordinate better Coalition formation techniques Distributed mechanisms for resource and task allocation

35. Optimisation Problem N L number of incidents I j number injured at incident j N U number of emergency units C i capacity of emergency unit i T ij response time for unit i to incident j 4 5 3 i j I j =5 T ij C i = 3

36. Auctions? Desirable Properties –Efficient means of dynamically allocating limited resources –in the presence of multiple stakeholders –with minimal communication requirement Auctions employed in a number of places: –Resource Allocation: Spectrum, Mining Tracts, Rare Items –Industrial Procurement –Novel applications Format –Agent submit bids –Auctioneer calculates winners (allocation) and payment

37. Distributed Auctions

38. Coalition Formation in Multi-Agent System Coalition formation process Agent 4 Task1 Task 2 Agent 2 Agent 1 Agent 5 Agent 3 Agent 6 Coalition 3 Coalition 1 Coalition 2

39. The guarantee provided by our algorithm on the quality of its solutions (21 agents) Previous state of the art Our algorithm 10% (of the optimal) 92% (of the optimal) 0.0000002% of the space 14% (of the optimal) 100% (of the optimal) 0.0000019% of the space Portion of the space searched Algorithm

40. Achievements Good level of collaboration –2 project workshops Winner of 1 st and 2 nd International Competitions on Agent Trust and Reputation (2006, 2007) Winner of Robocup Rescue Infrastructure Competition (2007) Winner of International Trading Agent Competition on Market Design (2007). 25 publications/submissions in journals, conferences and workshops –Including number of multi-institution ones 60 deliverables produced

41. Outreach Number of keynotes at international conferences Organised Workshop on Game Theory and Probabilistic Inference at NIPS (Rezek & Rogers). Organised 1st International Workshop on Agent Technology for Disaster Management at AAMAS-2006 (Jennings & Ramchurn) Organised special track on Autonomous Agents for Data and Information Fusion at Fusion-2006 conference (Johnston & Jennings) Organised agentcities workshop on Advanced Technologies for Disaster Management (Ramchurn & Jennings) Invited by EPSRC to present Aladdin at International Review of ICT (Jennings, Rogers & Ramchurn) Organised 1st International Workshop on Agent-Based Sensor Networks at AAMAS-2007 (Rogers & Dash). Participated in technical committee of Robocup Rescue 07 (Ramchurn)

42. Conclusions Exciting & challenging research agenda –Bringing together number of disciplines to produce end- to-end solutions to complex problems Fundamental research in basic techniques for individual and multiple actor systems Systems research on how to combine distinct components Demonstrations of technologies in disaster management scenarios