Human-robot interaction Michal de Vries

Humanoid robots as cooperative partners for people Breazeal, Brooks, Gray, Hoffman, Kidd, Lee, Lieberman, Lockerd and Mulanda International Journal of Humanoid Robots Submitted 2003, published in 2004

Overview 1) Introduction 2) Understanding others 3) Social robotics 4) Meet Leonardo 5) Task learning 6) Discussion

1) Introduction The goal of the paper: developing robots with social abilities. Such robots can understand natural human instructions (such as natural language, gestures, emotional expressions). Learning new skills should be done quickly. Create robots that play a role in the daily lives of ordinary people.

2) Understanding others Theory of Mind: People attribute mental states (beliefs, desires, goals) to understand and predict behavior. This is even the case with non-living things of sufficient complexity (Breitenberg). Although this is far from scientific, it is suprisingly useful (Dennett, 1987). Mirror neurons are a possible neural mechanism for the Theory of Mind (Gallese & Goldman, 1998).

3) Social robots Human-robot collaboration: No master-slave relation between human and robot, but cooperating partners. Joint Intention Theory: Doing something together as a team where the teammates share the same goal and same plan of execution.

Robots understanding others Most robots interact with people as objects or socially impaired people. Social robots must be capable of understanding the intentions, beliefs, goals and desires of people. It must also understand social cues (and vice versa). Such robots must be able to take multiple points of view. Common vs. partial knowledge.

How should social robots learn? It is a trend in machine learning to eschew built-in structure or a priori knowledge of the environment. The main focus is on statistical learning techniques. Such techniques need hunderds or thousands examples to learn something succesfully.

How should robots learn? Learning without built-in structure is a problem: A robot needs to learn quickly. Learning in biology is robust and fast. Furthermore, humans are also born with innate cognitive and behavioral machinery which develops in an environment. So the authors use a combination of bottom- up and top-down processing.

4) Meet Leonardo Leonardo: a robot with 65 degree of freedom

Leonardo's architecture Leo's computational architecture

Understanding speech Leo cannot speak, but has a natural language understanding system called Nautilus. Nautilus supports: For instance, a basic vocabulary, simple contexts and spatial relations.

The vision system Leo percieves the environment with 3 camera systems. A camera behind the robot to track people and objects in Leo's environment (peripheral information). An overhead camera mounted in the ceiling that faces vertically down to track gestures and objects (color, position, shape, size). The third camera system is in Leo's eyes and processes face recognition and facial features.

Attention Leo's attentional system computes the level of saliency (interest) for objects and events. Three factors compute saliency: perceptual properties internal states (belief system) socially directed reference

Beliefs Seeing reflects the state of the world as it is directly precieve. Beliefs are representational and are held even if they do not happen to agree with immediate perceptual experience. Leo's belief system gets input (visual, tactile information and speech) and merges this information into a coherent set of beliefs.

Beliefs Beliefs must be processed and updated correctly. Leo can compare his beliefs with beliefs of others. It must make a distinction between his beliefs and the beliefs of others, but also know which beliefs are common knowledge. Leo can represent beliefs of others by monitoring people's looks at objects, their gestures and talks over time.

5) Task learning Leo can learn from natural human instructions. “Leo, this is a hammer.” By hearing its own name, the word “this” and in combination with a pointing gesture to a hammer, the speech understanding passes this knowledge to the spatial reasoning and belief system. Leo can show whether he understands the instructions. “Leo, show me the hammer.” Leo can evaluate its own capabilities.

Push the button Leo learns pushing buttons

How Leo learns pushing buttons Task:“Buttons-On-and-Off” Leo indicates that he does not know such task and goes into learning mode. Subtask:“Button-On” The same reaction. A person learns Leo this by demonstrating it. It says “Press button 1” and turns it on. Leo encodes the goal state associated with an action performed in the tutorial setting by comparing the world state before and after its execution. The same holds for the subtask “Button-Off”.

A schematic overview All subtasks must be learned in order to master the overall task.

6) Discussion Social skills (performing/recognizing) are important for robots in relation with humans. No master-slave relation, but collaboration. Knowing what matters: attention. An action can be determined in more ways: Reinforcement learning (large state-action spaces -> large number of trials)‏ Learning by imitation (a lot faster, but requires innate knowlegde)‏

Some remarks Leo cannot speak, but speech is very important in social interaction. The practical and biological implausibility of 3 camera systems, especially an overhead camera. The biological implausibility of innate knowledge. Of course, we are biased towards some behaviour, but we do not have an a priori vocabulary. No role for mirror neurons?

Extra Info More information about Leo can be found at: eonardo/overview/overview.htmlhttp://robotic.media.mit.edu/projects/robots/l eonardo/overview/overview.html Questions?