1. Cognitive Colonization Tony Stentz, Martial Hebert, Bruce Digney, Scott Thayer Robotics Institute Carnegie Mellon University

2. Requirements Distributed robotics for small-scale mobile robots calls for a software system that: is robust to individual robot failure; does not depend on reliable communications; can perform global tasks given the limited sensing and computational capabilities of individual robots; learn to perform better through experience.

3. Cognitive Colonization Paradigm The proposed software system addresses these requirements by: dynamically assigning robots to tasks and checkpointing data; treating communication as an opportunistic resource; aggregating resources by distributing the computational and perceptual load across the group of robots; sharing learned behaviors (both individual and group) between all robots.

4. Software Architecture Command Unit Mobile Robot Mobile Robot Mobile Robot Mobile Robot Mobile Robot Mobile Robot Squad Level Mobile Robot Mobile Robot Mobile Robot Squad Level Mobile Robot Mobile Robot Mobile Robot Mobile Robot

5. Example: Distributed Mapping Unattached Robot Single Robot Command Unit Mapping Squad Mapping Squad Communications Squad

6. Colony Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - directives - positions - health - world map - merge - split - grow - regions to map - comm areas to maintain - all known behaviors - aggregate adapted behaviors

7. Squad Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - map a portion - commlink - positions - data quality - comm quality - local map - more comm squads - positions - fields of view - maintain internal comm - maintain external comm - re-establish comm - map a region - better mapping strategies - better comm strategies

8. Robot Level Objectives: Adaptations: Planning: Data Acquired: Subordinates: Behaviors: - take sensor data - relay data - position - health - sensor data - comm data - safeguarding - maintain comm - re-establish comm - bootstrap colony - better nav - better comm

9. Deliverables Reactive behaviors Communications protocols and strategies Resource allocation strategies Learning algorithms Planning algorithms Demonstration systems

10. Schedule Robust Colonization Port to Military Platforms Colonization Dynamics 2000200120022003 Static Colonization

11. DARPA Demo II Project

12. GRAMMPS: Initial Plan

13. GRAMMPS: Swapping Goals

14. GRAMMPS: Plan Completed

15. Chornobyl Nuclear Power Plant

16. Pioneer Robot

17. 3-D Map Data

18. New Ideas Distributed control architecture for formation of new robot colonies. Probabilistic model of robot existence. “In-situ” group learning, behavior exchange, skill swapping. Multiple colony dynamics.

19. Learning Hierarchical Control Structures Primitive Actions Sensors and Reinforcement Signals Flat StructureHierarchical Structure Learning Control System Behavior Sensors and Reinforcement Signals Primitive Actions Behavior

20. Generating Lower Skills New Task World Change 0 -50 -100 -150 -200 -250 -300 0 -50 -100 -150 -200 -250 -300 050100150200250300350 050100150200250300350 TIME PERFORMANCE HIERARCHICAL FLAT

21. Benefits of Hierarchical Learning Decomposed behaviors transfer between tasks, environments and robots Confines disruptions to only levels affected Generates levels of abstraction Suitable for robot pretraining

22. Behavior Learning in Colonies Can use shared experiences/information to speed learning Applicable to both individual robots, squads and command units Suitable for exploration with robot death/sacrifice