1. VEHICLE INTELLIGENCE LAB
Chapter 11. A Bayesian model of imitation in infants and robots (2/2) in Imitation and Social Learning in Robots, Humans and Animals, Nehaniv & Dautenhahn Course: Robots Learning from Humans
Dong-Kyoung Kye
Vehicle Intelligence Laboratory
School of Electrical and Computer Engineering
Seoul National University

2. Contents Bayesian imitative learning
Example : learning to solve a maze task through imitation
Learning a forward model for the maze task
Imitation using the learned forward model and learned priors
Inferring the intent of the teacher
Further applications in robotic learning
Towards a probabilistic model for imitation in infants
Conclusion

3. Bayesian imitative learning - Forward model
It maps state, action to next state :
𝑃 𝑆 𝑡 𝑆 𝑡 , 𝐴 𝑡 )
Learned from exploring the state-space at random
Body babbling
Supervised process (Assuming proprioception)

4. Bayesian imitative learning - Inverse model
Probability that an action is chosen given the desired next state, and the goal
𝑃 𝐴 𝑡 𝑆 𝑡 , 𝑆 𝑡+1 , 𝑆 𝑔 )
𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒕+𝟏 , 𝑺 𝒈 ) ∝𝑷 𝑺 𝒕+𝟏 𝑺 𝒕 , 𝑨 𝒕 )∗𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒈 )
Forward model
Prior

5. Learning to solve a maze task through imitation
Learning a forward model for the maze task
20 x 20 grid of squares
States 𝑆 𝑡 : Grid locations in the maze
Five actions available
- North(N), East(E), South(S), West(W) or remain in place(X)
The noisy ‘forward dynamics’ of the environment
- Actual and learned probabilistic forward models
- Simulated maze environment

6. Learning to solve a maze task through imitation
Imitation using the learned forward model and learned priors
The imitator can use ‘inverse model’ to select appropriate actions to imitate the teacher and reach the goal state.
𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒕+𝟏 , 𝑺 𝒈 ) ∝𝑷 𝑺 𝒕+𝟏 𝑺 𝒕 , 𝑨 𝒕 )∗𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒈 )
Inverse model
Forward model
Prior

7. Learning to solve a maze task through imitation
Imitation using the learned forward model and learned priors
The imitator can use ‘inverse model’ to select appropriate actions to imitate the teacher and reach the goal state.
𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒕+𝟏 , 𝑺 𝒈 ) ∝𝑷 𝑺 𝒕+𝟏 𝑺 𝒕 , 𝑨 𝒕 )∗𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒈 )
Prior
- Simulated maze environment

8. Learning to solve a maze task through imitation
Imitation using the learned forward model and learned priors

9. Learning to solve a maze task through imitation
Inferring the intent of the teacher
The intent inference algorithm provides an estimate of the distribution over the instructor’s possible goals for each time step.
𝑷 𝑺 𝒈 𝑨 𝒕 , 𝑺 𝒕 , 𝑺 𝒕+𝟏 )∝𝑷 𝑺 𝒕+𝟏 𝑺 𝒕 , 𝑨 𝒕 , 𝑺 𝒈 )∗𝑷 𝑨 𝒕 𝑺 𝒕 , 𝑺 𝒈 )∗𝑷 𝑺 𝒕 𝑺 𝒈 )∗𝑷 (𝑺 𝒈 )
Learned
forward model
Learned prior over actions

10. Learning to solve a maze task through imitation
With a maze task example..
It shows that how the abstract probabilistic framework proposed in this chapter can be used to solve a concrete sensorimotor problem.

11. Further applications in robotic learning
E.g.) Box lifting (HOAP-2)

12. Further applications in robotic learning
E.g.) Box lifting (HOAP-2)
Learning forward models from motion capture.
(a) Forward models learned by the system after observing 3 different actions performed by the human
(b) Forward model learned by the system for the box lift experiment

13. Further applications in robotic learning
E.g.) Box lifting (HOAP-2)

14. Towards a probabilistic model for imitation in infants
- Active Intermodal Mapping (AIM)

15. Towards a probabilistic model for imitation in infants
- Match and correction process is the Bayesian action selection method.

16. Conclusion
Bayesian approach is well-suited to imitation learning in real-world robotic environments which are noisy and uncertain
Bayesian probabilistic framework can also be applied to better understand the stages of infant imitation learning.