1. Breakout session B questions

2. Research directions/areas Multi-modal perception cognition and interaction Learning, adaptation and imitation Design and evaluation of interfaces User studies, metrics, monitoring Tasks and coordination

3. Research directions/areas Multi-modal perception cognition and interaction Perception Perception is important - computational vision, haptics Biological models of computation, neuroscience approaches several models already tested - good adaptability Spatial representations, attentional mechanisms in the context of HRI tasks object detection, recognition Understanding dynamically changing environments - human activity modeling (gestures, activities - recognize person is falling, doing the exercise properly) how to benefit from recent advances in neuroscience and computer vision Haptics Sensory motor bandwidth - minimum update rate - understanding motor control feedback of biological systems (such as minimum update rates)

4. Research Directions Natural language processing incrementality in conversation, break up utterances, clarify parts of utterances, understanding speech flow, what a spontaneous speech look how to give robot instructions not give in perfect language Alan’s example - good challenge for NPL - needing robots understand understanding how people talk when they do work robot can be learning how to person speaks - spontaneous speech otherwise difficult - might be easier in this context Overcome divide in language technologies (Visual) Perception and Language Semantic context - language and vision - in the context of actitivity Describe the action as you do it - incremental adaptive of learning spatial Representations and object/word/relation associations

5. Research Directions Control safety of complex (coupled) control system both robot and human conflicting requirements - how to handle safety Traditional control - study coupling between human and robot control how robots can train humans (rehabilitation) How humans can train robots (better controllers) modeling of the coupled systems from control perspective (aka block diagrams) Recent developments in dynamically stable mobile robots - new avenues for human robot interactions (push robot - machine have to understand physically language design (sensing forces inertial) Cognitive architectures testbeds - HRI tasks motivation for ‘cognitive’ architectures Can serve as good checkpoints - for the big research challenge

6. Roadmaps and Experiments considered to show achievements Landmarks on the roadmaps collaborative assembly - human and the robot together assemble Lego model of the airplane (known the pieces) - how language and vision can be used together Learn how to recognize new objects with human interface (pointing and naming) adaptive representations Robot meet you at the door - collaborative navigation (RoboDog) Guide for the blind - navigation, language instructions, but also Dynamic interaction (stop and go, understanding subtle gestures, forces, commands) Rehabilitation domain tighter coupling (robot/human) - getting out of bed Intelligent walker adapts to persons need

7. What other fields should be considered How to foster collaborations Money for multidisciplinary collaboration - cognitive psychology, neuroscience, linguistics Most of it in place - better incentives provided by funding Multi-disciplinary training Interdisciplinary areas - standalone support for junior faculty (sabbaticals) Multi-disciplinary PHD programs Curriculum development

8. Intellectual Merit Development new computational models of biological systems Development of new (learning) algorithms for acquiring adaptive representations (vision, multi- modal sensing, language) of real-world dynamically changing environments Development of new algorithms for learning and/or modeling behaviors/control strategies of complex high DOF systems (humans and robots) Development of coupled control strategies, motion interaction languages, techniques for interaction with dynamically stable robots

9. Broader Impacts Impact on human lives NL human robot interaction - collaborative assembly tasks - human and robot engaged in common task (exercise, assembly, education, therapy) NL + vision - impact on (incremental) learning adaptive representations of objects and actions - improved interfaces, interactions for robotic household assistant, service robotics

10. Broader Impacts Haptics - enriched sensory experience for variety of applications/interactions (rehabilitation, surgery, entertainment) Coupled Control - various aspects of health care industry - rehabilitiation, surgery, surgical training

11. Broader impacts Impact on education Education Outreach (high schools, Promoting interdisciplinary research