1. Peeping into the Human World
Sociable Robots
Peeping into the Human World

2. An Infant’s Advantages
Non-hostile environment
Actively benevolent, empathic caregiver
Co-exists with mature version of self

3. Baby Scheme Physical form can evoke nurturing response
Caregiver exaggerates voice, gestures

4. Kismet – a Baby Robot

5. Requirements Robot needs to perceive human state
Computer vision, speech processing
Human needs to perceive robot state
Animatronics, speech generation
Closed loop interaction requires both directions

6. Readable locus of attention
Attention can be deduced from behavior
Or can be expressed more directly

7. Expressing locus of attention
Animatronic expression of locus of attention
Small distance between cameras with wide and narrow field of views, simplifying mapping between the two
Central location of wide camera allows head to be oriented accurately independently of the distance to an object
Allows coarse open-loop control of eye direction from wide camera – improves gaze stability

8. Computing locus of attention
“Cyclopean” camera
Stereo pair
Useful to have relatively stable view for computer vision tasks.
Deviation from human arrangement.
Retinotopic view is pretty unstable
Need narrow field of view for precise foveation
Could add wider field of view resource to eyes – solution on Cog
Tricky aesthetically with cameras at time (and still now for wide field of view and not too expensive)
So place on head
Auxiliary benefit – not affected at all by eye-movement
Makes for more stable system
Open-loop control of eyes based on head-fixed imaging, refined by closed-loop control based on eye-fixed imaging

9. Computing locus of attention
Visual input
Pre-attentive filters
Saliency map
Pursuit
(Cyclopean camera)
Perceptual biases.
Don’t have to exact – decision will be made attentively.
Analogous to pop-out, pre-attentive processes in humans.
Massively parallel, relatively simple operations that filter and prioritize the scene.
Coarse behavioral influence.
Primary information flow
Modulatory influence
Behavior system
Eye movement

10. Looking Preference Internal influences bias how salience is measured
“Seek toy” –
low skin gain, high saturated-color gain
Looking time 28% face, 72% block
“Seek face” –
high skin gain, low color saliency gain
Looking time 80% face, 20% block
Internal influences bias how salience is measured
The robot is not a slave to its environment
Prefers behaviorally relevant stimuli

11. Example (video)

12. Example (video) Robot’s search is task-specific
Still opportunistic when appropriate
Visual behavior conveys degree of commitment
Gaze direction, expression conveys interest

13. Interpersonal Distance
Comfortable interaction distance

14. Interpersonal Distance

15. Interpersonal Distance

16. Interpersonal Distance

17. Interpersonal Distance

18. Examples (video) “Back off buster!” “Come hither, friend”
Robot backs away if person comes too close
Cues person to back away too – social amplification
Robot makes itself salient to call a person closer if too far away

19. Negotiating object showing
Comfortable interaction speed
Too fast –
irritation response
Too fast,
Too close – threat response

20. Example (video)

21. Facial expressions

22. Facial Expressions (Russell, Scott&Smith)
arousal
surprise
afraid
elated
angry
stress
excitement
happy
frustrated
displeasure
pleasure
neutral
sad
content
depression
calm
fatigued
relaxed
Emotions can be mapped to affect dimensions (Russell)
Facial postures are related in a systematic way to these affective dimensions (Smith & Scott)
bored
sleepy
sleep

23. Generating Posture, Expressions
Open stance
Low
arousal
anger
fear
tired
Negative
valence
content
Positive
valence
High
arousal
Closed stance

24. Example Facial Expression

25. With Posture (video)

26. Emotive Voice Quality

27. Synthesized Emotive Speech

28. With Vocalizations (video)

29. But…
Is there more to life than being really, really, really, ridiculously good-looking?

30. Affective Intent (video)

31. Fernald’s Results
Four cross-cultural contours of infant-directed speech
Exaggerated prosody matched to infant’s innate responses
time (ms)
pitch, f (kHz) o
approval
That’s a
good bo-o-y!
No no
baby.
time (ms)
pitch, f (kHz) o
prohibition
Can you
get it?
time (ms)
pitch, f (kHz) o
attention
time (ms)
pitch, f (kHz) o
MMMM
Oh, honey.
comfort

32. Evidence for Fernald-like Contours
Approval
Prohibition
Attention
Soothing

33. Performing Recognition
prohibition &
high-energy neutral
attention & approval
energy variance
soothing & low-energy neutral
pitch mean
Breazeal & Aryananda, Humanoids 2000

34. Examples (video)

35. Turn-Taking
Cornerstone of human-style communication, learning, and instruction
Four phases of turn cycle
relinquish floor
listen to speaker
reacquire floor
speak
Integrates
visual behavior & attention
facial expression & animation
body posture
vocalization & lip synchronization

36. Example (video)

37. Kismet’s really, really, ridiculously detailed web-pages:
Conclusions
Robots can partake in “infant-caregiver” interactions
These interactions are rich with scaffolding acts
Prerequisite for socially situated learning
Kismet’s really, really, ridiculously detailed web-pages: