Thrust ID: Peer-to-Peer HRI Training and Learning with Humans Rod Grupen (lead) Cynthia Breazeal Nicholas Roy MURI 8 Kickoff Meeting 2007

MURI 8 Kickoff Meeting 2007 Interactive Training in Human-Robot Teams MIT-Vanderbilt-Stanford UW-UMASS Amherst  Learning from human demonstration  common ground - project proprietary sensory and motor policy spaces into a common frame  asking questions and providing explanations  combine learned policies flexibly and effectively in response to new run-time situations  perform peer-to-peer policies jointly with humans and other robots

MURI 8 Kickoff Meeting 2007 Thrust ID Objectives MIT-Vanderbilt-Stanford UW-UMASS Amherst  build libraries of schema for component tasks underlying triage, hazmat, and HRI  acquaint human partners with capabilities and limitations of robot partners  establish common knowledge about strategies, procedures, and practices  negotiate roles and preferences for joint activity  master joint activities through practice

MURI 8 Kickoff Meeting 2007 Actions MIT-Vanderbilt-Stanford UW-UMASS Amherst  g  g  K  g  T  g  g’

MURI 8 Kickoff Meeting 2007 Developmental Programming MIT-Vanderbilt-Stanford UW-UMASS Amherst l native control basis l temporal structure for development l integration - hard problems use previous solutions for easier problems teleological hypothesis

MURI 8 Kickoff Meeting 2007 Action Schemas - Hierarchy MIT-Vanderbilt-Stanford UW-UMASS Amherst computational model of infant development  stage 1 - touch what you see  stage 2 - the length of your arm  stage 3 - grasp affordances  stage 4 - human collaboration Vgotskian pointing multi-body objects: simultaneous trackability attributes: motion (scale, multi-body kinematics), topological/geometrical attributes, hue, saturation, intensity, texture

MURI 8 Kickoff Meeting 2007 Action Schemas - Generative Models, Teleology, and Transfer Learning MIT-Vanderbilt-Stanford UW-UMASS Amherst teleoperator sorting instruction sorting replay with prior knowledge (1) parse events to find a matching schema. (2) associate goals with schema (3) Replicate demonstration with contingencies

MURI 8 Kickoff Meeting 2007 Commodity Mobile Manipulators MIT-Vanderbilt-Stanford UW-UMASS Amherst …nature routinely selects for dynamics to combine speed and agility with light weight and low power… strength, performance, safety

MURI 8 Kickoff Meeting 2007 Whole-Body Primate/Hominid/Human Model MIT-Vanderbilt-Stanford UW-UMASS Amherst postural stability prehensile skills tool use social organization

MURI 8 Kickoff Meeting 2007 Contributions MIT-Vanderbilt-Stanford UW-UMASS Amherst  multi-agent, and human-robot schema for coordinated action  interactive, socially-guided learning from demonstration (question/explain)  hierarchical composition of skills  communicative actions to convey states, objects, and actions

MURI 8 Kickoff Meeting 2007 Year 1 Demonstrations MIT-Vanderbilt-Stanford UW-UMASS Amherst  component schema for initial triage client sideremote network client side remote network

MURI 8 Kickoff Meeting 2007 Year 1 Demonstrations MIT-Vanderbilt-Stanford UW-UMASS Amherst sample acquisition, and cataloging load carrying strategies that do not violate stability constraints of the platform tool use hazard containment

MURI 8 Kickoff Meeting 2007 Year 2 (and onward) MIT-Vanderbilt-Stanford UW-UMASS Amherst  role engagement and switching in multi-robot, and human-robot strategies  remote humans, prior knowledge, maps, run- time situational awareness, mental models, asymmetric beliefs, affect on communicative actions  UGV/UAV/human coalitions