Design and Evaluation of a Multiagent Team for CLIMA Contest Çetin Meriçli Melih Çelik

Contents What is CLIMA Contest? – Massim Agent Infrastructure Why CLIMA? Proposed Approach – Simple Communicationless Role Assignment – Market-driven Task Allocation Algorithm – Q-Learning for parameter optimization Conclusions

What is CLIMA Contest? An attempt to stimulate research in the area of multi- agent systems by – Identifying key problems – Collecting suitable benchmarks Result in new approaches and techniques from computational logics Provided a scenario that is – Generic enough to be applicable to wide-range techniques – Precise enough to be tested using different approaches

What is CLIMA Contest? (contd) Components of Simulation Massim Server Massim Server Monitor Massim Agent Environment contains Two Teams Four agents each Depot Gold nuggets Bushes

Massim Agent Infrastructure AbstractAgent doAuthentication(…) processMessage(Element) processPing(String) processPong(String) processRequestAction(…) processSimulationEnd(…) processSimulationStart(…) receiveDocument() receivePacket() sendDocument(Document) start() ExampleAgent main(…) DemoGridAgent1 main(…) processRequestAction(…) processSimulationStart(..)

Why CLIMA? A good simulation environment for real-world problems like rescue operation, mine sweep Simulation Environment Gold Nuggets Locations not known a priori Pickup gold Carry gold to depot Bushes on the route Real World (Rescue) Trapped, injured people Locations not known Load to ambulance Carry to hospital Blocked rodes

Proposed Approach Simple Communicationless Role Assignment Market-driven Task Allocation Algorithm Q-Learning for parameter optimization

Simple Role Assignment Each agent acts as a single agent and select a proper role among the pool of available roles No explicit communication among agents Will be used to assess performance of Market-driven approach along with the sample agents of the simulator

Market-driven Task Allocation Each agent is a corporate A fixed payoff is offered for each subtask Agents bid on the task according to their cost of performing that task (resource, time, etc..) Task is assigned to the agent with minimum cost Decentralized system. Costs are broadcasted among teammates

Normally we think that assigning one task to each robot will give better result But, if we examine, giving both tasks to Robot A will resulted in a smaller cost

Q-Learning Will be used for estimating optimal parameters for Market-Driven Task Allocation Algorithm Why Q-Learning? – We need to assess a performance to an action in a certain state (in a grid cell, in our example) – Q-Learning is a very popular algorithm for such problems

Conclusions CLIMA domain is very suitable for developing algorithms for foraging missions So far, we have examined the simulator and development environment Currently we are working on implementing the algorithms to be evaluated