1. RoboCup Rescue Simulation Barış Eker April 2005

2. CONTENT  Robocup Rescue  RoboAKUT 2005  Discussion

3. RoboCup Rescue  Rescue League –Disaster environment is simulated in real world –Real robots are used –Final aim is to search&rescue –First step is searching and finding important points in disaster environment, civilians, fires, etc.

4. What is RoboCup Rescue?  Rescue Simulation League –Disaster environment is simulated in computer –Aims to simulate large urban areas –2D simulation (Although simulation is in 2D, 3D viewer is now available) –Agent programs are written to run on this environment –Aim is to both search&rescue

5. Rescue Simulation League  Multi-agent approach, heterogeneous agents, rescue strategies are tested  Seven different types of agent –Ambulance, fire fighter, police agents and control center agents for each of them, and civilians  Cooperation is needed among agents –Most cases, individual agents are insufficient to deal with problems –Communication is limited

6. Structure of the Simulation Environment  Real-time distributed simulation system –Several modules that connect to a network –Each module can run on a different computer to reduce the computational load –Each phenomenon is simulated by different simulators E.g. collapse of buildings, fire, etc. –GIS provides initial condition of disaster space –Viewer makes the visualization

7. Progress of RCRSS  The kernel sends individual vision info to each RCR agent  Each agent submits an action to kernel  The kernel sends these commands to sub- simulators  Sub-simulators submit the updated states to kernel  Kernel integrates the received states and send it to viewer  The kernel advances simulator clock

8. Progress of RCRSS  One cycle in simulation corresponds to one minute in disaster space  The kernel waits half a second for command/state submissions at steps 2 and 4, so a cycle takes a second  It occasionally takes a few seconds according to scale of the simulation and machine specs  All RCR agents must decide in half a second

9. RoboCupRescue Agents  An RCR agent controls act of an object in the disaster space  Seven classes –Civilian, AmbulanceTeam, FireBrigade, PoliceForce, AmbulanceCenter, FireStation and PoliceOffice  Platoon agents: Ambulance Team, Fire Brigade and Police Forces  Center agents: Ambulance center, Fire Station and Police Office  Rescue agents: Platoon agents + Center agents OR All agents – Civilian agents

10. RoboAkut  Reinforcement learning used in previous years  This year, all team is written from scratch  Task-based multiagent approach  Tasks are assigned from centers, however the agents can also act autonomously  Platoon agents collect the sensory information and send necessary ones to their centers

11. Ambulance Team  Start with exploring buildings assigned by center  If any civilian found, they are reported to center and the one that has the highest priority is started to be saved –Rescue agents has highest priority –Among civilians, having less estimated life(hp/damage) has more priority

12. Ambulance Team  Center opens auction for reported buried civilians and platoon agents respond by bidding  The bid values are determined by the distance from target, however this may be improved  Agents are assigned roles –Owner role : deals with job until it finished (with success or failure) –Helper role : deals with job until job finishes or owner role for another job is assigned –By that way, more than one agent is assigned to job when available, this increases efficiency

13. Ambulance Center  At the beginning, divides the disaster space into regions and assigns to platoons  Holds list of reported civilians and open auctions for them –Waits for two clock tick –Concludes the auction and inform the winners (owner and helper) –If owner does not send confirmation message, auction is re-opened for the civilian

14. Ambulance Center  Civilians are hold in a priority list –Reported by ambulance teams and from other centers –Priority is determined by Estimated remaining life –Having less life, has more priority Estimated rescue time –Having less rescue time, has more priority At first, we want to save the ones that has minimum life and easiest to save –What can be done ? Better remaining life estimation can be done via learning Better metrics can be proposed By that way, we may increase the overall efficiency of the team

15. Fire Brigades  Fire brigades also works with auction mechanism, but auction is not opened for each building –Fires should be extinguished in early stages, otherwise it becomes much harder –Therefore, quicker response required –Auction is opened for representative building for a region

16. Fire Brigades  All firebrigades assigned to a region deals with representative building at first –When fieryness property becomes 2, fires start spreading –In that case, helper firebrigades start extinguishing neighbour fires, because they can be extinguished more easily and by that way spreading can be prevented –Owner deals with representative building until it is extinguished or completely burned  Helpers are assigned index from center –The agent having index 1, deals with most recent fire, the agent having index 2, deals with next most recent

17. Fire Center  Holds the list of known fires  Fires represent regions –If a new fire is reported, it is checked whether it is in one of the known regions  After center assign a region to firebrigades, it sends clear region command to police center

18. Police Agents  Deals with the job assigned by their center –Clear region : If a region is assigned, all roads in this region are cleared –Clear road : If a road is assigned, it is reached and cleared  Report their position in each clock tick  Sends confirmation message when “clear region” message is taken  Informs center, when “clear region” job is finished  While path planning, other agents do not use blocked roads, but police agents use them and clear while passing

19. Police Center  Collect “clear road” and “clear region” messages from other centers  Assigns jobs to nearest polices  For “clear region” waits for confirmation message, if not taken, re-assignes the region

20. Communication  We used a common mechanism for message sending  Each agent has a “Message Sender”  “Message Sender” collects the messages in different queues according to their priority  “Message Sender” controls the maximum number of sendable messages in each clock tick  Unsent messages remain to be sent in next clock tick, by that way no message remains unsent  Each agent gives the message type while sending, “Message Sender” knows the priority of messages and puts them into related list

21. Problems  Need a better life estimator for ambulances  Need a better priority metrics for ambulance center, the rescue order of civilians is very important  Can auction or cost mechanism be improved?  Need a better way to extinguish boundary fires  Traffic jam  What if centers fail? –Communication between heterogeneous agents fail –Are platoon agents autonomous enough?