1. Using Robots in Autism Therapy: A Survey of Ongoing Research
Marjorie Skubic
Associate Professor
Electrical and Computer Engineering Dept.
Computer Science Dept. (joint apt.)

2. Outline Motivation – How I got interested Autistic disorders
A survey of the research
Why robots might help
The field of researchers
Conclusions

3. How I got interested Research in Human-Robot Interaction
Looking for a killer application
Better – How can we use robots to help people?
Talks at the IEEE RO-MAN 2005 Workshop

4. Autistic Disorders
1 of 300 children diagnosed with autism with rates rising
1 of 800 children diagnosed with Down syndrome
1 of 450 children diagnosed with juvenile diabetes
1 of 333 children will develop cancer by age 20
Diagnosis currently made through behavioral observation
No blood test or genetic screening is available although there is evidence of a genetic link

5. Autistic Disorders: Characteristics
Inability to relate to other people
Little use of eye contact with other people
Difficulty understanding gestures and facial expressions
Difficulties with verbal & non-verbal communication
Difficulty understanding other’s intentions, feelings, and mental states

6. Why Use Robots?
Most children, including children with autism, are attracted to robots.
This natural affinity is exploited, and the robot is used as an interactive toy.
Robots may provide a less threatening environment than interacting with people.
Robots can provide a repetitive and more predictable environment.
This “safe” environment can gently push a child with autism towards human interaction.

7. The Connection to Imitation
One theory: Autism may be caused by early impairments in imitation and shared attention (Rogers & Pennington, 1991) (Baron-Cohen, 1995)
Imitation is a format of communication, a means to express interest and engage others in interaction (Nadel, 1999)
Idea: Use a doll-like robot to engage children with autism and teach basic imitative interaction skills
From: K. Dautenhahn, and A. Billard, Games Children with Autism Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

8. Robota
A six-year old autistic boy playing with Robota. He seemed curious about Robota's head movements and so he touches the doll.
From: K. Dautenhahn, and A. Billard, Games Children with Autism Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

9. Imitation Using Robota
“Robota … allows the child to understand that the doll’s movement originates from his own movement (sense of agency) and is limited to a restricted category of movement (enhances intentional action)”
From: J. Nadel, “Early Imitation and a Sense of Agency,” Proc. 4th Intl. Workshop on Epigenetic Robots, 2004

10. An autistic child playing “chasing” games with the mobile robot
From: K. Dautenhahn, and A. Billard, Games Children with Autism Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

11. Joint Attention Using Robota
Robota is controlled via teleoperation by the investigator.
From: B. Robins, P. Dickerson, and K. Dautenhahn, “Robots as
Embodied Beings – Interactionally Sensitive Body Movements
In Interactions Among Autistic Children and a Robot,”
Proc. RO-MAN 2005

12. Two autistic children:
Note Andy’s gaze at
Jack.
The investigator encourages the children to show each other how
they can interact with the robot.
The robot will not move unless the children show the same movement,
i.e., they must work together.

13. Andy and Jack touch each other to balance themselves while each raising a leg.

14. Adam shows no interest in his classmates
and usually tries to avoid the rest of the
children. But Adam is interested in Robota.
Adam takes Rob’s hand to show him how to interact with Robota.

15. Interacting with Keepon
Keepon is controlled via teleoperation.
From: H. Kozima, C. Nakagawa, and Y. Yasuda, “Interactive
Robots for Communication-Care: A Case Study in Autism
Therapy,” Proc. RO-MAN 2005

16. Views from Keepon’s camera eyes

17. Attentive action
Emotive action
Keepon's kinematic mechanism. Two gimbals are connected by four wires; the lower gimbal is driven by two motors. Another motor rotates the whole inner-structure; yet another drives the skull downward for bobbing.

18. Enabling Interaction
Eye-contact: Referring to each other's mental states
Joint attention: Sharing the perceptual information
Enables people to exchange intention and emotion toward a target.

19. Emergence of dyadic interaction
Emergence of dyadic interaction. Spontaneous actions to Keepon (left) and actions copied from others (right).
Emergence of triadic interaction. The child discovers excitement in Keepon (left) and then looks at the adult to share the excitement (right).

20. Using Robots for Autism Diagnosis
ESRA
Playtest
From: B. Scassellati, “Quantitative Metrics of Social Response
for Autism Diagnosis,” Proc. RO-MAN 2005

21. Autism Diagnosis Methods
Reaction to the ESRA robot with and without the face configuration
Can generate facial expressions
using 5 servo motors

22. Autism Diagnosis Methods
Measure listening preferences to speech sounds
At the press of a button,
an audio clip is played.
The interaction is logged
in non-volatile memory.

23. Autism Diagnosis Methods
Vocal prosody, i.e., how something is said
Features F24 vs. F1
Mean pitch * energy vs. mean pitch
Separation of two features used in a Bayesian classifier distinguishes low energy categories (neutral and soothing) from high energy categories (approval, attention, and prohibition).

24. Autism Diagnosis Methods
Position tracking relative to another person

25. Autism Diagnosis Methods
Gaze direction and focus of attention
Red – adolescents with autism
Blue – typical adolescents

26. Linear discriminant analysis of autistic (au) and typical (nc) gaze patterns. Linear filters F(x) are trained to reproduce the gaze pattern G(x) of each individual x and then applied to predict the gaze patterns of any other individual.
For example, F(au)*G(self) indicates a filter trained on an individual with autism and tested on that same individual while F(nc)*G(au) indicates a filter trained on a control individual and tested on an individual with autism. The mean performance of this data (y-axis) is a function of the response percentile of individual pairings. Significant differences (all p<0.01 for a two-tailed t-test) are seen between the following classes: (1) F(nc)*G(self), (2) F(au)*G(self), (3) F(nc)* G(other nc), and (4) the three other conditions.

27. University of Sherbrooke
Project for engineering students:
Design a robotic toy for an autistic child
Educational value
Real world problem
Students work together in a team
Students must first investigate autistic disorders

28. University of Sherbrooke
Pushing Jumbo around the play area.
Rolling game with Roball.
From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and autism", Socially Intelligent Agents - Creating Relationships with Computers and Robots, Kluwer, pp

29. University of Sherbrooke
Assembling the arms and tail of C-Pac.
Girl showing signs of interest toward Bobus.
From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and autism", Socially Intelligent Agents - Creating Relationships with Computers and Robots, Kluwer, pp

31. The Field of Researchers
Francois Michaud
University of Sherbrooke, Canada
Kerstin Dautenhahn & Ben Robbins
University of Hertfordshire, UK
Aude Billard
Swiss Federal Institute of Technology (EPFL)
Jacqueline Nadel
French National Centre of Scientific Research

32. The Field of Researchers
Brian Scassellati and Bob Schultz
Yale University
Javier Movellan
University of California – San Diego
Hideki Kozima
National Institute of ICT, Japan
Michio Okada
ATR, Kyoto, Japan

33. Conclusions
The use of robots for autism therapy and diagnosis is just beginning.
There is anecdotal evidence that robot therapy can help children with autism
How can we start here at MU with the new Thompson Family Center for Autism and Neurodevelopmental Disorders?

34. Maybe the Tiger Kitty
The iCat by Philips Research

35. Acknowledgements
Thanks to Brian Scassellati, Francois Michaud, Ben Robins, and Hideki Kozima for helpful discussions.