Cognitive Colonization Tony Stentz, Martial Hebert, Bruce Digney, Scott Thayer Robotics Institute Carnegie Mellon University

Requirements Distributed robotics for small-scale mobile robots calls for a software system that: is robust to individual robot failure; does not depend on reliable communications; can perform global tasks given the limited sensing and computational capabilities of individual robots; learn to perform better through experience.

Cognitive Colonization Paradigm The proposed software system addresses these requirements by: dynamically assigning robots to tasks and checkpointing data; treating communication as an opportunistic resource; aggregating resources by distributing the computational and perceptual load across the group of robots; sharing learned behaviors (both individual and group) between all robots.

Software Architecture Command Unit Mobile Robot Mobile Robot Mobile Robot Mobile Robot Mobile Robot Mobile Robot Squad Level Mobile Robot Mobile Robot Mobile Robot Squad Level Mobile Robot Mobile Robot Mobile Robot Mobile Robot

Example: Distributed Mapping Unattached Robot Single Robot Command Unit Mapping Squad Mapping Squad Communications Squad

Colony Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - directives - positions - health - world map - merge - split - grow - regions to map - comm areas to maintain - all known behaviors - aggregate adapted behaviors

Squad Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - map a portion - commlink - positions - data quality - comm quality - local map - more comm squads - positions - fields of view - maintain internal comm - maintain external comm - re-establish comm - map a region - better mapping strategies - better comm strategies

Robot Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - take sensor data - relay data - position - health - sensor data - comm data - safeguarding - maintain comm - re-establish comm - bootstrap colony - better nav - better comm

Deliverables Reactive behaviors Communications protocols and strategies Resource allocation strategies Learning algorithms Planning algorithms Demonstration systems

Schedule Robust Colonization Port to Military Platforms Colonization Dynamics Static Colonization

DARPA Demo II Project

GRAMMPS: Initial Plan

GRAMMPS: Swapping Goals

GRAMMPS: Plan Completed

Chornobyl Nuclear Power Plant

Pioneer Robot

3-D Map Data

New Ideas Distributed control architecture for formation of new robot colonies. Probabilistic model of robot existence. “In-situ” group learning, behavior exchange, skill swapping. Multiple colony dynamics.

Learning Hierarchical Control Structures Primitive Actions Sensors and Reinforcement Signals Flat StructureHierarchical Structure Learning Control System Behavior Sensors and Reinforcement Signals Primitive Actions Behavior

Generating Lower Skills New Task World Change TIME PERFORMANCE HIERARCHICAL FLAT

Benefits of Hierarchical Learning Decomposed behaviors transfer between tasks, environments and robots Confines disruptions to only levels affected Generates levels of abstraction Suitable for robot pretraining

Behavior Learning in Colonies Can use shared experiences/information to speed learning Applicable to both individual robots, squads and command units Suitable for exploration with robot death/sacrifice