1. The Next Generation of Robots?
Rodney Brooks
and
Una-May O’Reilly

2. Our Objectives How can biology inform robotic competence?
How can aspects of human development and social behavior inform robotic competence?

3. Our Approach Exploit the advantages of the robot’s physical embodiment
Integrate multiple sensory and motor systems to provide robust and stable behavioral constraints
Capitalize on social cues from an instructor
Build adaptive systems with a developmental progression to limit complexity

4. Our Humanoid Platforms
Cog and Kismet

5. Biological Inspiration for Cog
Cog has simulated musculature in its arms
Cog has an implementation of a human model of visual search and attention
Cog employs context-based attention and internal situations influence action
Cog uses a naïve model of physics to distinguish animate from inanimate

6. Social Inspiration for Cog
A theory of mind
A theory of body
Mimicry

7. Human <—> Robot Cameras Gaze direction Microphones Facial
Neck pan
Neck tilt
Neck lean
Eye tilt
Left eye pan
Right eye pan
Camera with wide field of
view
Camera with narrow field of
Axis of rotation
Microphones
Facial
features
Speech synthesizer
Head orientation

8. Levels of Control robot responds to human human responds to robot
Social Level
Behavior Level
perceptual
feedback
current goal
Skills Level
coordination
between motor
modalities
current primitive(s)
Primitives Level

9. Kismet’s Competencies
Direct Visual Attention
Recognize Socially Communicated Reinforcement
Communicate Internal State to Human
Regulation of Social Interaction

10. No One in Charge QNX L NT 11 400-500 MHz PCs Linux QNX (vision)
speech synthesis
affect recognition
Linux
Speech
recognition
Face
Control
Emotive
Response
Percept
& Motor
Drives &
Behavior L
Tracker
Attn
system
Dist. to
target
Motion
filter
Eye
finder
Motor
ctrl
audio
speech
comms
Skin
Color
QNX
CORBA
dual-port
RAM
Cameras
Eye, neck, jaw motors
Ear, eyebrow, eyelid,
lip motors
Microphone
Speakers
4 Motorola micro-controllers
L, multi-threaded lisp
higher-level perception, motivation, behavior, motor skill integration & face control
MHz PCs
QNX (vision)
Linux (speech recognition)
NT (speech synthesis & vocal affect recognition

11. Visual Attention skin tone color motion habituation attention
Frame Grabber
skin tone
color
motion
habituation w w w w
attention
inhibit
reset
Top down,
task-driven
influences
Eye Motor
Control

12. Visual Search

13. Social Constraints Person backs off Person draws closer
Comfortable interaction speed
Too fast – irritation response
Too fast,
Too close – threat response
Comfortable interaction distance
Too close – withdrawal response
Too far – calling behavior
Person draws closer
Person backs off
Beyond sensor range

14. Evidence for 4 contours in Kismet-directed speech
Cross Cultural Affect
Evidence for 4 contours in
Kismet-directed speech
time (ms)
pitch, f (kHz) o
approval
That’s a
good bo-o-y!
No no
baby.
prohibition
Can you
get it?
attention
MMMM
Oh, honey.
comfort

15. Low-Intensity Neutral High Intensity Neutral
Affect Recognizer
Soothing &
Low-Intensity
neutral vs
Everything Else
Soothing
Low-Intensity Neutral
Approval & Attention
Prohibition
High Intensity Neutral
approval
attention
soothing
prohibition
neutral
prohibition
attention & approval
energy variance
soothing & low-energy neutral
pitch mean

16. Naive Subjects 5 female subjects 4 naive subjects 1 caregiver
Four contours and neutral speech
praise, prohibition, attention, soothing
Multiple languages
French, German, Indonesian, English, Russian
Driven by Human

17. Facial Expressions arousal sleep displeasure pleasure neutral
excitement
depression
stress
calm
afraid
angry
frustrated
relaxed
content
elated
bored
sad
fatigued
happy
surprise
sleepy

18. Facial Postures in Affect Space
Open stance
Low
arousal
fear
accepting
Negative
valence
tired
unhappy
content
surprise
Positive
valence
disgust
stern
High
arousal
anger
Closed stance

19. Face, Voice, Posture

20. Turn-Taking / Proto-Dialog
Naïve subjects
Told to “talk to the robot”
Engage in turn taking
No understanding (on either side) of content

21. Implemented Model of Visual Search and Attention
Color w
Motor
System
Motion  w
Activation Map
Skin w
Motivation
System
Habituation w

23. Hardware – Cog’s Arms 6 DOF in each arm Series elastic actuator
Force control
Spring law

24. Hardware – Cog’s Head 7 degrees of freedom
Human speed and range of motion

25. Visual and Inertial Sensors
3-axis inertial sensor
Peripheral View
Peripheral View
Foveal View
Foveal View

26. Computational System Designed for real-time responses
Network of 24 PC’s ranging from MHz
QNX real-time operating system
Implementation shown today consists of
~26 QNX processes
~75 QNX threads