1. Multi-Agent Exploration in Unknown Environments Changchang Wu Nov 2, 2006

2. Outline Why multiple robots Design issues Basic approaches –Distributed –Centralized –Market-based

3. Why Multiple Robots Some tasks require a robot team Have potential to finish tasks faster Increase robustness w/ redundancy Compensate sensor uncertainty by merging overlapping information Multiple robots allow for more varied and creative solutions

4. A Good Multi-Robot System Is: Robust: no single point of failure Optimized, even under dynamic conditions Quick to respond to changes Able to deal with imperfect communication Able to avoid robot interference Able to allocate limited resources Heterogeneous and able to make use of different robot skills

5. Basic Approaches Distributed –Every robot goes for itself Centralized –Globally coordinate all robots Market-based –Analogy To Real Economy

6. Distributed Methods Planning responsibility spread over team Each robot basically act independently Robots use locally observable information to coordinate and make their plans

7. Example: Frontier-Based Exploration Using Multiple Robots (Yamauchi 1998) A highly distributed approach Simple idea: To gain the most new information about the world, move to the boundary between open space and uncertainty territory Frontiers are the boundaries between open space and unexplored space

8. Occupancy Grid World is represented as grid Each cell in the grid is assigned with a probability of being already occupied/observed The initial probability is all set to.5 Cell status can be Open ( 0.5) Bayesian rule is used to update cells by merging information from each sensor reading (sonar)

9. Frontier Detection Frontier = Boundary between open and unexplored space. Any open cell adjacent to unknown cell is frontier edge cell. Frontier cells grouped into frontier regions based on adjacency. Accessible frontier = Robot can pass through opening. Inaccessible frontier = Robot cannot pass through opening.

10. Multi-Robot Navigation Simple algorithm: Each robot goes along the shortest obstacle free path to a frontier region Robots share a common map: All information obtained by any robot is available to all robots Robots are planning path independently Use reactive strategy to avoid collisions Robots may waste time for the same frontiers

11. An Exploration Sequence

12. Distributed Methods: Pros & Cons Pros –Very robust. No single point failure –Fast response to dynamic conditions –Little or no communication is required –Easy ….Little computation required Cons –Plans only based on local information –Solutions are often sub-optimal

13. Centralized Methods Robot team treated as a single “system” with many degrees of freedom A single robot or computer is the “leader” Leader plans optimal tasks for groups Group members send information to leader and carry out actions

14. Example: Arena (Jia 2004) Robots share a common map and only communicate with a leader Robots compete for resources by their efficiency leader greedily assigns the most efficient tasks Leader coordinate robots to handle interference

15. Background World representation –Occupancy grid Cost unit –Moving forward one step = Turning 45 degrees Cost overflow –Similar to minimum cost spanning tree –Easy to compute the shortest path –Easy to handle obstacle

16. Cost Overflow Cost of 45° turning ＝ Cost of one cell’s step 12 2 3 3 3 3 4 4 4 44 4 5 5555 5 5 5 Direction priority 6 6 66666 7 7 7 77 8 8 8 8 88 8 99 9 9 9 9 9 10 11 12

17. Goal Candidates Detection A goal point P should satisfy i.P is passable (M ark the cells in warning range or obstacles/Wall/Unknown cells as impassable ) ii.Some unexplored cells lie in the circle with P as the center and (R + K) as the radium, where R is the warning radius and K is usually 1 Robot cell paths cell observation cell candidate goal

18. Goal Resource Reserved goal candidates –Robots obtained by competition Recessive goal candidates –The goal points in a given range to a reserved goal point –This distance can be adjusted Goal candidates Recessive goals candidates

19. Path Resource Path resource is a time-space term For a given time, the cells close to any robot are marked off for safety Looks just like a widened path Basically a reactive strategy goal path resource

20. Revenue and Utility Revenue –The expected gain of information that robots observe at a goal point Utility used by many other approaches –Utility = revenue – cost Utility in this paper –Utility = Revenue / Cost –Better connected to purpose of smallest cost –No need to care about unit conversion

21. Greedy Goal Selection Try to maximize the global utility Coordination: robots obtain goal and path resources exclusively Competition: repetitively select the pair of free agent and goal with highest utility Sub-optimal

22. Simple Algorithm Repeat until map is complete –Repeat #free robots times 1.Cost computation (Also make sure no interference with the busy robots) 2.Select the highest utility task (Compete) 3.Mark off the associated robot and goal points, and nearby goal points

23. 1 st Competition: 2 22 2 2 2 2 2 2 2222 2 2 2 2 2 2 333 3 3 3 3 3 3 333333 3 3 3 333 444 4 4 4 4 44 44444 1 1 11 1 1 1 1 1 11 5 5 555555 5 5 6 6 6 66 6 6 6 6 6 Interval = 3Competitor:

24. 1 st Competition Result: 44 6 6 5 5 3 2 2 1 4 4 1 22 44 33 22 4 Interval = 3Competitor:

25. 2 nd Competition 1 2 1 22 333 44 4 444 44 1 11 1 11 Interval = 3Competitor:Satisfied: 222 2 2 2 2 222 3333 3 3 3 3 3 3 3333 4 44444 4 4 4 4 4 4 44444 55555 555 5 5 5 5 6 6 6 666 6 6 6 6 6 6

26. 2 nd Competition Result 1 2 1 22 333 44 4 444 44 4 1 2 3 4 5 1 1 2 2 22 2 333 444 555 666 6 666 6 Interval = 3Competitor:Satisfied:

27. 3 rd Competition 1 2 1 22 333 44 4 444 44 4 1 2 3 4 5 1 2 2 22 2 333 444 555 666 6 666 66 Competitor:Satisfied:Interval = 3 1 11 2 2 222 3333 3 3 3 44444 4 455555 66 6 77 7 7 7 7 7 7 7 8 8 8 8 8 8 8 9 9 9 9 9 9 9 10 11 12 13

28. 3 rd Competition Result 1 32 4 4 1 2 3 4 5 1 2 3 4 666 Competitor:Satisfied:Interval = 3 1 2 3 47 8 9101112 56 13 4 6

29. Planning Issues Do not transfer a reserved goal point to another free agent (unless necessary). Frequent change of tasks can cause localization error. Quit an assigned task when the goal point is unexpectedly observed by other robots Schedule at most one task for each agent

30. Possible Variations Still keep busy agents in competition. Remove the goal resources they win from competition. –This prevents those goal resources being assigned to other agents –It is too early to burden a new task on a robot who has not achieved it current task No need to schedule them. –New resources probably will be found when they reach the goals

31. Handling Failure of Planning It may fail to plan safe paths –When some robot get to a place where it is almost too close to other robot it has no good space to detour –And it choose to just wait there for other robots to move away, which is not known by other robots Avoidance of unexpected obstacle –Robots have simple reactive mechanism –Release resources and try to gain new task

32. Fail to plan safe paths Competitor:Satisfied:Interval = 3 1 2 345678 9 10 1 2 34 111213141516 collision

33. Reactive Mechanism Competitor:Satisfied:Interval = 3

34. Exchange Tasks Competitor:Satisfied:Interval = 3 1234 5 1 2 3

35. Some Statistics

36. Demo

37. Centralized Methods : Pros Leader can take all relevant information into account for planning Optimal s islution possible! One can try different approximate solutions to this problem

38. Centralized Methods: Cons Optimal solution is computationally hard –Intractable for more than a few robots Makes unrealistic assumptions: –All relevant info can be transmitted to leader –This info doesn’t change during plan construction Vulnerable to malfunction of leader Heavy communication load for the leader

39. Market-Based Methods Based on market architecture Each robot seeks to maximize individual “profit” Robots can negotiate and bid for tasks Individual profit helps the common good Decisions are made locally but effects approach optimality –Preserves advantages of distributed approach

40. Why Is This Good? Robust to changing conditions –Not hierarchical –If a robot breaks, tasks can be re-bid to others Distributed nature allows for quick response Only local communication necessary Efficient resource utilization and role adoption Advantages of distributed system with optimality approaching centralized system

41. Architecture World is represented as a grid –Squares are unknown (0), occupied (+), or empty (-) Goals are squares in the grid for a robot to explore –Goal points to visit are the main commodity exchanged in market For any goal square in the grid: –Cost based on distance traveled to reach goal –Revenue based on information gained by reaching goal R = (# of unknown cells near goal) x (weighting factor) Team profit = sum of individual profits –When individual robots maximize profit, the whole team gains

42. Example World

43. Goal Selection Strategies Possible strategies: –Randomly select points, discard if already visited –Greedy exploration: Choose goal point in closest unexplored region –Space division by quadtree

44. Exploration Algorithm Algorithm for each robot: 1. Generate goals (based on goal selection strategy) 2. If OpExec (human operator) is reachable, check with OpExec to make sure goals are new to colony 3. Rank goals greedily based on expected profit 4. Try to auction off /bid goals to each reachable robot –If a bid is worth more than you would profit from reaching the goal yourself (plus a markup), sell it

45. Exploration Algorithm 5. Once all auctions are closed, explore highest-profit goal 6. Upon reaching goal, generate new goal points –Maximum # of goal points is limited 7. Repeat this algorithm until map is complete

46. Bidding Example R1 auctions goal to R2

47. Expected vs. Real Robots make decisions based on expected profit –Expected cost and revenue based on current map Actual profit may be different –Unforeseen obstacles may increase cost Once real costs exceed expected costs by some margin, abandon goal –Don’t get stuck trying for unreachable goals

48. Information Sharing If an auctioneer tries to auction a goal point already covered by a bidder: –Bidder tells auctioneer to update map –Removes goal point Robots can sell map information to each other –Price negotiated based on information gained –Reduces overlapping exploration When needed, OpExec sends a map request to all reachable robots –Robots respond by sending current maps –OpExec combines the maps by adding up cell values

49. Advantages of Communication Low-bandwidth mechanisms for communicating aggregate information Unlike other systems, map info doesn’t need to be communicated repeatedly for coordination

50. What Is a Robot Doing Goal generation and exploration Sharing Information with other robots Report information to OpExec at some frequency

51. Experimental Setup 4 or 5 robots –Equipped with fiber optic gyroscopes –16 ultrasonic sensors

52. Experimental Setup Three test environments –Large room cluttered with obstacles –Outdoor patio, with open areas as well as walls and tables –Large conference room with tables and 100 people wandering around Took between 5 and 10 minutes to map areas

53. Experimental Results

55. Successfully mapped regions Performance metric (exploration efficiency): –Area covered / distance traveled [m 2 / m] –Market architecture improved efficiency over no communication by a factor of 3.4

56. Conclusion Market-based approach for multi-robot coordination is promising –Robustness and quickness of distributed system –Approaches optimality of centralized system –Low communication requirements Probably not perfect –Cost heuristics can be inaccurate –Much of this approach is still speculative Some pieces, such as leaders, may be too hard to do

57. In Sum Distributed vs. centralized mapping Distributed vs. centralized planning Revenue/Cost vs. Revenue – Cost Often sub-optimal solutions No common evaluation system for comparisons

58. References Yamauchi, B., "Frontier-Based Exploration Using Multiple Robots," In Proc. of the Second International Conference on Autonomous Agents (Agents98), Minneapolis, MN., 1998. Menglei Jia, Guangming Zhou,Zonghai Chen, "Arena—an Architecture for Multi-Robot Exploration Combining Task Allocation and Path Planning,“ 2004 Zlot, R., Stentz, A., Dias, M. B., and Thayer, S. “ Multi-Robot Exploration Controlled By A Market Economy. ” Proceedings of the IEEE International Conference on Robotics and Automation, 2002. http://voronoi.sbp.ri.cmu.edu/presentations/motionplanning2001Fall/FrontierExp loration.ppt http://voronoi.sbp.ri.cmu.edu/presentations/motionplanning2001Fall/FrontierExp loration.ppt http://www.ai.mit.edu/courses/16.412J/lectures/advanced%20lecture_11.6.ppt http://www.ai.mit.edu/courses/16.412J/lectures/advanced%20lecture_11.6.ppt http://mail.ustc.edu.cn/~jml/jml.files/Arena.ppt