1. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 1 Workshop Introduction: Modular Perception and Control for Autonomous Robotics Chad Jenkins Assistant Professor Computer Science Department Brown University Robotics 2005 Workshop on Modular Foundations for Perception and Control June 11, 2005 Cambridge, MA USA

2. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 2 Common Background ● Brooks’ Subsumption Architecture introduced the use of modular components for autonomous robot perception and control during the mid-1980’s ● Since then, modularity has been a crucial aspect of robotics systems, including: – Behavior-based and hybrid autonomous control ● Trajectory formation, manipulation, robot teams – Parsimonious state spaces ● Reinforcement learning and decision making – Human-robot interaction ● Gesture recognition and imitation learning

3. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 3 Common Background ● Modularity is typically cast in terms of behaviors or models at two levels of abstraction: – Primitive modules that encode basic low-level behavior – Compositional modules that compose existing modules to provide greater functionality ● Through modularity, we are able to – Limit the cost of searching for high-DOF systems – Respond promptly in potentially dynamic environments – Provide scalability over time in developing perception and control systems

4. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 4 A General View for Primitive Modules Low-level Sensing Modular Vocabulary of Primitives Motor-level Actuation Set desired motion trajectory Task-level Controller Internal Planner Human Communication Encode observed motion into vocabulary Interact through common vocabulary Plan using modular capabilities Execute modules to achieve objectives Task-level Skill-level Motor-level Compositional Modules Compose existing modules into new functions

5. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 5 The Greater Challenge for Modularity ● These issues get to a greater questions: – Why do roboticists continually reinvent modular architectures and components for practical applications/platforms? ● Programming languages analogy: – Java code is platform independent – Any computer with a virtual machine can run Java code – Libraries are reused to construct greater functionality ● Should this be our goal for robotics? – Obviously, our problem is harder

6. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 6 Underlying Issues ● To this point, there is little unified consensus the following four interrelated issues – Principles for modular design and development – General module representation – Common interface structure – Refinement of modules over time

7. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 7 Issue 1: Modular Development ● What modules are necessary? – Size, scope, and interaction among modules ● How should the modules be developed? ● Robotics has not adopted a general methodology for designing and developing modular controllers. – Instead, ad hoc decisions or biased domain knowledge is applied to a specific implementation. – I believe modules should be learned in a data-driven fashion from the real-world.

8. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 8 Issue 2: Module Representation ● Is there a general formulation for encoding a wide spectrum of primitive modules? ● Can compositional operators be defined for sequencing and superposition? ● Can we avoid coding limited-purpose modules or learning overly specific policies?

9. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 9 Issue 3: Common Interface Structure ● Ideally, modules are used in a wide variety of applications – Neural prosthesis, humanoid robots, dynamics-based interactive entertainment, human-robot teams, assistive robotics, robotic telepresence, smart spaces – Involving a variety of problems: low-bandwidth communication, mixed-initiative interaction, user modeling, gesture recognition, distributed inference, exteroceptive information, attention/saliency ● Modules should be generally applicable and not engineered for specific objective ● Is there a general interface structure capable of expressing the output of a wide spectrum of modules? – Avoiding complex data structure for handling many special cases ● Some architectures exist, but are domain specific – Player/Stage (mobile), Orocos (industrial), Yarp (humanoid)

10. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 10 Issue 4: Refinement over time ● Aspects of control and perception change over a robot’s useful lifetime. ● How amenable are modules to modification? ● Can modules be added or removed without complication? ● Which modules should be refined and when? – Should primitives be developed and then remain static? ● How much to the primitives need to account for subtle changes to the robot’s dynamics over time?

11. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 11 My Approach ● Prediction is the key element – Modules provide the expectation for future states ● Learn predictive modules from data – Categorize and generalize in an unsupervised fashion to a NLDS – Gradient field s.t. \hat{x}_m[n+1] = f_m(x[n],u[n]) ; y[n] = g(x[n]) ● Predictions directional vectors in state space – Applications access modules at each time step for control or perception tasks ● Learned modules are exemplar-based – Modifying an example redefines the behavior of the module ● Currently working on composition operators and modules

12. Jenkins — Modular Perception and Control Brown Computer Sciencecjenkins@cs.brown.edu — ROUGH DRAFT (05.19.2005) 12... Develop training programs Human performance Matrix Analogy: Jujitsu Punch Stand ready Kick Jump Download to human Capture data from performance Learn behaviors Human-like robot control Enhanced human performance Perception of human activity