1. A Comparison of Children Learning New Words From Robots, Tablets, and People
Jacqueline Kory Westlund, Leah Dickens, Sooyeon Jeong,
Paul Harris, David DeSteno, & Cynthia Breazeal
Hi, I’m Jackie. I'm a PhD student in the Personal Robots Group at the MIT Media Lab, which is led by Cynthia Breazeal.
Today I’m going to share an early study about children’s word learning from robots, tablets, and humans that I did with our collaborators: Leah Dickens and David DeSteno from Northeastern University, Paul Harris from the Harvard Graduate School of Education, as well as Sooyeon Jeong who is another student in Cynthia’s lab.

2. Word learning Hart & Risley, 1995; Snow et al., 2007
The reason we looked at word learning is because
Hart & Risley, 1995; Snow et al., 2007

3. Early language development
Word learning
Early language development
it is an easily measurable component of children’s early language development. Children’s early language development is important because it is the basis for much learning throughout life.
Hart & Risley, 1995; Snow et al., 2007

4. Early language development
Word learning
Early language development
Literacy skills are crucial for success in school
Literacy
Duranti & Goodwin, 1992; Hart & Risley, 1995; Snow et al., 2007; Vygotsky, 1978

5. Early language development
Word learning
Early language development
Academic
success
Literacy
Duranti & Goodwin, 1992; Hart & Risley, 1995; Snow et al., 2007; Vygotsky, 1978

6. Early language development
Word learning
Early language development
Life success
And beyond. Language is our primary means of human knowledge transfer.
A decent amount of literature explores how children who grow up with an impoverished exposure to English – such as fewer total words heard by age 3 or less access to vocabulary-enrichment activities in preschool – may have trouble "catching up" later. This is the infamous “word gap.” Vocabulary ability early on predicts language skills later.
Thus a lot of effort goes into finding new ways to support children’s early language education
Academic
success
Literacy
Duranti & Goodwin, 1992; Hart & Risley, 1995; Snow et al., 2007; Vygotsky, 1978

7. Technology for Language Education
iPads, tablets
Computers
Robots
Some of those new ways is through the use of technology – we create ipads, tablets, computers, TV, virtual agents, and even robots to help children learn new words.
I mean, that’s why some of us are here, right? We see the potential of robots in education?
Cassell, 2004; Judge et al. 2015; Naigles & Mayeux, 2001; Wartella & Lauricella, 2014;
Willoughby et al., 2015; Wolf et al. 2014

8. Children learn from robots
Breazeal et al., in press
Chang et al., 2010;
Tanaka & Matsuzoe, 2012
Here are a few of the robots. These robots played vocabulary games, acted out new verbs, and shared stories that contained key vocabulary words.
This prior work shows that children treat these robots as companions and learn from them in ways similar to how they learn from people.
One thing to see here is that many of these learning scenarios involved explicit instruction of words and word meanings. But a lot of children’s usual language learning occurs rapidly and without feedback from a teacher.
Kory Westlund et al.,
2015; Gordon et al., in
review
Cassell, 2004
Movellan et al., 2009

9. Fast mapping Rapid Approximate Little or no feedback
So in this work, we focused on “fast mapping” – rapid, approximate word learning that happens without feedback.
Here’s an example of fast mapping from the work of Carey and colleagues. A teacher asks 3- and 4-year-olds to get one of two nearby trays: “Bring me the chromium one, not the red one.” Children infer that the new word “chromium” refers to the new color.
Obviously, it can take time to really get the full meaning of a new word, but the initial mapping – the fast mapping – happens pretty quickly.
So that brings us to our research questions.
Carey, 2010; Carey & Bartlett, 1978; Corriveau et al., 2009;
Kucker et al., 2015; Markson & Bloom, 1997

10. Research Questions
Will children learn new words via fast mapping from a robot and/or from a tablet as effectively as from a human?
Who/what do children prefer as a learning partner?
How do children conceptualize the robot?
Will children learn new words via fast-mapping from a robot and/or from a tablet as effectively as from a human?
Who or what do children prefer as a learning partner?
How do children conceptualize the robot in this task – as an agent more like a person, or more like a tablet?

11. Hypotheses
Children will learn equally well from the robot and the human, but less well from the tablet.
Children will prefer the human.
Children will construe the robot as between human and tablet.
Because prior work has shown the importance of social cues for language learning, we expected that children would learn equally well from the robot and person, but less well from the tablet, due to its lack of social embodiment.
We expected children would prefer to learn with the robot, because it was an exciting new character.
We expected children would construe the robot as something in-between a machine and a person, based on prior work showing that children may conceptualize robots as a new category of social other in between machines, pets, and people.
1: Harris, 2007; Kuhl, 2007; Meltzoff et al., 2009; Naigles & Mayeux, 2001; Sage & Baldwin, 2010
3: Kahn et al., 2012; Severson & Carlson, 2010

12. Study design Within-subjects (tablet x robot x person)
The study followed a within-subjects design.
Each child interacted with all three interlocutors: tablet, person, robot. The order of interactions was counterbalanced.
A second experimenter acted as the master of ceremonies, and led the child through all three interactions.

13. Two sessions Learning task + recall + questions Recall + questions
Each child also participated in two sessions.
The first session included the learning task, recall tests, and questions gauging the child’s construal of the robot.
The second session – about a week later -- repeated the recall tests and the questions, to see whether children retained what they had learned and whether their construal of the robot changed when they remembered the robot later.

14. Learning task 10 animal pictures with each interlocutor
2 animals named
8 positive but uninformative comment
Children looked at a series of 10 unfamiliar animals with each interlocutor. Each picture was shown on the tablet screen for 10 seconds.
During 8 of the 10 pictures, the interlocutor commented positively but uninformatively – “Look at that!” “Cool animal!”
During the other 2 pictures, the animals were named – “Ooh, a kinkajou. See the kinkajou?” This provided the opportunity for fast mapping to occur.

15. Recall test 5 animals shown at a time 1 that was named
4 that were seen, but unnamed
After children viewed a set of pictures with an interlocutor, we did a receptive recall test. They were shown five pictures at a time: one target named animal plus four distractors – animals that had been seen, but were not named. They were asked to point to the named animal: “Which one is the kinkajou?”
We did the test again in the second session.

16. Construal Questions
6 questions: “When a robot ____, is it more like a person or more like an iPad?”
Answers a question
Remembers something
Teaches you something
Thinks about something
Interested in something
Tells you something
We asked children questions about their construal of the robot as more like a person or more like an iPad or various activities – answering a question, teaching you, thinking.
We called the tablet an ‘iPad” during the study because the kids were more familiar with iPads. ? ?

17. Preference Questions
6 questions: “If you want to find out ____, who would you ask – person, robot, or iPad?”
Name of animal
What gadget is called
What animal eats
What gadget does
Where animal lives
Where gadget is found
We asked children who they would seek out as a source of new information. We showed the children two pictures and asked questions such as “If you wanted to find out the name of this animal, who would you ask – the person, the robot, or the iPad?”
We also asked children which of the three interlocutors they would most like to play with again.

18. Technology DragonBot (Setapen, 2012) Tablet Wizard-of-Oz
Recorded dialogue
Tablet
We used a DragonBot robot and a Samsung Galaxy tablet for the study.
The person’s voice was recorded for the robot’s speech and the tablet’s speech, then pitch-shifted to sound reasonably distinct, so that all three would use the same tone and intonation of speech.
The robot was teleoperated for the study. The teleoperator triggered the robot’s actions according to a script of behavior, and the same speech and expressions were used by the person. Given the simplicity of the scenario, the teleoperator primary had to attend to timing to trigger speech at the right times.

19. Participants 19 children (10 female, 9 male) Ages 3-5 (M=4.6, SD=.57)
Greater Boston area preschool
19 children ages 3-5 from a Greater Boston area preschool serving a predominately middle class population participated in the study.
Briefly, before we go on to results,

20. Hypotheses
Children will learn equally well from the robot and the human, but less well from the tablet.
Children will prefer the human.
Children will construe the robot as between human and tablet.
Let’s review our hypotheses. We tested them as follows:
1: Harris, 2007; Kuhl, 2007; Meltzoff et al., 2009; Naigles & Mayeux, 2001; Sage & Baldwin, 2010
3: Kahn et al., 2012; Severson & Carlson, 2010

21. Hypotheses
Children will learn equally well from the robot and the human, but less well from the tablet. Recall tests
Children will prefer the human.
Children will construe the robot as between human and tablet.
The recall tests told us about children’s learning
1: Harris, 2007; Kuhl, 2007; Meltzoff et al., 2009; Naigles & Mayeux, 2001; Sage & Baldwin, 2010
3: Kahn et al., 2012; Severson & Carlson, 2010

22. Hypotheses
Children will learn equally well from the robot and the human, but less well from the tablet.
Children will prefer the human. Questions
Children will construe the robot as between human and tablet.
We can learn about preferences through the questions about who the children would seek as a source of new information
1: Harris, 2007; Kuhl, 2007; Meltzoff et al., 2009; Naigles & Mayeux, 2001; Sage & Baldwin, 2010
3: Kahn et al., 2012; Severson & Carlson, 2010

23. Hypotheses
Children will learn equally well from the robot and the human, but less well from the tablet.
Children will prefer the human.
Children will construe the robot as between human and tablet. Questions
And we learned about how they perceive the robot with the “more like a person or more like a tablet” questions.
1: Harris, 2007; Kuhl, 2007; Meltzoff et al., 2009; Naigles & Mayeux, 2001; Sage & Baldwin, 2010
3: Kahn et al., 2012; Severson & Carlson, 2010

24. Results: Learning
Children learned a mean of 4.3 of the 6 named animals correctly overall.
As you can see here, children correctly identified about the same number of animals during the recall with each interlocutor. They retained this information too – there was not much dropoff during the follow-up test a week later.
This was surprising – we had expected children would learn better with the robot and person. That said … this was a pretty simple learning task, and did not really require attending to social cues to do well, so maybe we should have anticipated that it would not be that difficult for children to pick up the animal names from any of these interlocutors.

25. Results: Source of new information
This graph shows children’s mean preference for picking each interlocutor, across all six questions.
In general, there was a trend toward asking the person for all the kinds of information, followed by the robot, then the iPad. This was especially true regarding the name of the animal, what the animal eats, and what the gadget does questions.
Only for the question what the gadget is called were children split on whether they would pick the robot or person.
Children’s answers were pretty stable from session 1 to the followup, with the exception of where the animal lives and where you find the gadget – so in session 2 they were split.

26. Results: Preference
We asked children directly who they would like to learn with again.
You can see a very strong preference for the robot.
The “other” answers were usually children declaring a tie, and not picking a favorite.
This could be related to the robot’s relative novelty – it was a pretty exciting robot, compared to the people and tablets that were more familiar. It was also a robot dragon, and some kids may be predisposed to think favorably of both dragons and robots, given recent movies like “How to Train Your Dragon” and “Big Hero 6”.

27. Results: Perception of robot
Switch from tablet-like to person-like:
Answers a question
Teaches you something
More person-like:
Interested in things
Remembers things
Thinks about things
More tablet-like:
Tells you something
For the questions asking children whether the robot was more like a person or like an iPad, we saw some interesting patterns.

28. Results: Perception of robot
Switch from tablet-like to person-like:
Answers a question
Teaches you something
More person-like:
Interested in things
Remembers things
Thinks about things
More tablet-like:
Tells you something
After interacting with the robot, children’s perceptions on a few dimensions shifted from thinking the robot would be like other technology to thinking it was more like a person. This was especially evident for two questions that invited children to appraise the robot as an active social partner – when a robot answers a question and teaches you things.

29. Results: Perception of robot
That's shown here, you can see the change from pretest to posttest. The switch in session 2 may be to how children remembered the robot - as different from how they expected it would be and how it was when they had just played with it.

30. Results: Perception of robot
The same change is seen when a robot teaches you: more like a person later.

31. Results: Perception of robot
Switch from tablet-like to person-like:
Answers a question
Teaches you something
More person-like:
Interested in things
Remembers things
Thinks about things
More tablet-like:
Tells you something
They thought the robot was more personlike when it is interested in things, when it remembers things, and when it thinks about things.

32. Results: Perception of robot
Switch from tablet-like to person-like:
Answers a question
Teaches you something
More person-like:
Interested in things
Remembers things
Thinks about things
More tablet-like:
Tells you something
But more tablet-like when it tells you something.

33. Results: Perception of robot
Shown in this graph. When a robot tells you something – immediately after interacting, children thought the robot was more like an ipad. Perhaps this was because the phrasing of the question led children to focus on the robot’s voice, which may have ended up more similar to the tablet’s than the person’s, though all three were the same person originally. Later, they remembered the robot as more like a person.
But these results were in line with what we expected: the robot was viewed as somewhat like a person, but with some technological qualities.

34. Main findings Learned equally from robot, person, tablet
Simple task
Did not require social information
Preferred person as source of new information
High enthusiasm for robot
Construed robot as person-like with some technological qualities
So to summarize:
Our main findings were that in this simple case, children learned from all three interlocutors equally well.

35. Main findings Learned equally from robot, person, tablet
Simple task
Did not require social information
Preferred person as source of new information
High enthusiasm for robot
Construed robot as person-like with some technological qualities
They preferred the person as a source of new information about the animal and gadget, but some children did say they’d seek out the robot or tablet.

36. Main findings Learned equally from robot, person, tablet
Simple task
Did not require social information
Preferred person as source of new information
High enthusiasm for robot
Construed robot as person-like with some technological qualities
Even though they usually preferred the person, children had very high enthusiasm for learning or playing with the robot again.

37. Main findings Learned equally from robot, person, tablet
Simple task
Did not require social information
Preferred person as source of new information
High enthusiasm for robot
Construed robot as person-like with some technological qualities
Finally, children perceived the robot as fairly person-like – thinking, being interested, teaching – but still viewed it as having some technological attributes.

38. Future work
More complex tasks requiring social information for learning
What are humans vs. robots best at?
Children’s construal of robots
This study left a lot of questions open.
In what cases are a human’s abilities required?
Some work has found that embodied robot tutors are more effective than virtual tutors.
What learning is best suited to a tablet or a robot?
Would a human fair better than a robot for more complex tasks requiring social or physical information for learning?
We’re done some follow-up work since this study looking at some more complex learning tasks and probing how kids view robots. Talk to me at the break if you’re curious!
Questions?

39. A Comparison of Children Learning New Words From Robots, Tablets, and People
Jacqueline Kory Westlund, Leah Dickens, Sooyeon Jeong,
Paul Harris, David DeSteno, & Cynthia Breazeal

40. So are the robots autonomous?
Autonomy = hard problem!
child speech recognition, social interaction
We’re working on it
In the video, you saw a lot of robots.
They’re not autonomous, well, not all of them.
Ideally, autonomous robot will act similar to human-controlled robot
In the couple of studies I’m going to describe, we used the dragonbot as a learning companion for young kids.
In these studies, we remote-controlled the robot. It wasn’t necessary to have a fully autonomous robot because what we were interested in is potential interactions – how kids would behave and learn if they got to play with an ‘ideal’ robot, a robot that did everything right.
Don't need autonomy to study how people interact with robots

41. Robots are not people! Not a replacement for teachers or caregivers!
Support interactions:
Ask questions, spark conversation
Model beneficial behaviors, conversation strategies, more advanced language
Natalie Freed: Sophie study
Robot as facilitator
David Nuñez: Tinkerbook
Robot as “parent trainer”
Prompts for parents
Ask questions, spark conversation
Model beneficial behaviors, conversation strategies, more advanced language
Natalie Freed: Sophie study
Robot as facilitator
David Nuñez: Tinkerbook with prompts
Robot as “parent trainer”

42. Early language impact
Low SES kids heard ~30million words less than high SES kids (Hart & Risley, 1995)
Unfamiliar words, cognitive challenge -> higher language ability entering kindergarten (Snow et al., 2007)
Impoverished exposure to novel English words or rich vocab-building curricula -> deficits in language ability (Fish & Pinkerman, 2003; Paez, Tabors, & Lopez, 2007)
Fish, M., & Pinkerman, B. (2003). Language skills in low-SES rural appalachian children: Normative development and individual differences, infancy to preschool. Journal of Applied Developmental Psychology, 23(5),
Páez, M. M., Tabors, P. O., & López, L. M. (2007). Dual language and literacy development of spanish-speaking preschool children. Journal of Applied Developmental Psychology, 28(2),