Cognitive Colonization The Robotics Institute Carnegie Mellon University Bernardine Dias, Bruce Digney, Martial Hebert, Bart Nabbe, Tony Stentz, Scott Thayer

Presentation Outline  Requirements  Software Architecture  Perception and Mapping  Communal Learning  Robot Test Bed  Status and Future Work

Requirements  is robust to individual robot failure;  does not depend on reliable communications;  can perform global tasks given the limited sensing and computational capabilities of individual robots;  learn to perform better through experience. Distributed robotics for small-scale mobile robots calls for a software system that:

Cognitive Colonization Paradigm  dynamically assigning robots to tasks and checkpointing data;  treating communication as an opportunistic resource;  aggregating resources by distributing the computational and perceptual load across the group of robots;  sharing learned behaviors (both individual and group) between all robots. The proposed software system addresses these requirements by:

Software Architecture Models Centralized Distributed optimal intractable brittle sluggish communication heavy suboptimal tractable robust nimble communication light

Free Market Architecture  Robots in a team are organized as an economy  Team mission is best achieved when the economy maximizes production and minimizes costs  Robots interact with each other to exchange money for tasks to maximize profit  Robots are both self-interested and benevolent, since it is in their self interest to do global good

Simple Reasoning Robot 1 Robot 2 Task A = 120 Task B = Robot 1 profit = 20 Robot 2 profit = 30

More Complex Reasoning Robot 1 Robot 2 Task A = 120 Task B = Subcontract: ( ) / 2 = 130 Robot 1 profit: 40 (20) Robot 2 profit: 50 (30)

Distributed Mapping Example Operator Exec <-- Revenue paid Tasks performed -->

Distributed Mapping Roles Unattached Robot Single Robot Command Unit Mapping Squad Mapping Squad Communications Squad

Architectural Framework Roles Resources Negotiations Locomotor Sensors CPU Mapper Comm Leader Exec Radio

Simple Mapping Example InitialFinal

Complex Mapping Example Initial Final

Simulated Mapping X X X X X X X X X X X X X X X X X X X

Simulator Movie

Simulated Mapping X X X X X X X X X X X X X X X X X X X

Architecture Features  Revenue, cost and profit  Negotiation and price  Competition vs. cooperation  Role determined via comparative advantage  Self organization  Learning and adaptation

Map Reconstruction Objectives  Reconstruction of 3-D map from multiple robots  Unknown or imprecise relative position  Recovery of positions and structure  Map reconstruction for operators Robot A Robot B

Typical Environment

Map Reconstruction: Approach  Approach:  Feature extraction  Initial feature matches  Recovery of epipolar geometry  Filtering of matches by re-projection  Recovery of motion and 3D structure

Feature Matching and Depth Recovery

Feature Map

Feature Matching and Depth Recovery

Feature Map

Communal Learning

Robot Death and Sacrifice  Quickly learning causes of robot death required for colony survival  ‘Buddy System’ used to preserve fatal situations and actions  When robot sacrifice is required maximal cause of death information will be extracted  Causes of death are high value commodities and quickly disseminated through the colony

Robots in Action

Current Status  Five working robot test beds with navigation, obstacle avoidance, point-to-point communication, and image streaming  First version of software architecture working for distributed coverage tasks  First version of cooperative stereo implemented with automatic feature selection  Prototype colony interface designed

Next Steps  Port the architecture to the real robots  Extend the architecture to support all robot roles needed for distributed mapping  Add learning for behavior parameters and transaction confidences  Perform map integration from multiple sensing points