1. Introduction to Imitation Learning谷雨
03/12

2. ICML2018: Imitation Learning

3. Background What to predict in imitation learning?
A distribution of actions (or simply an action) given a state
Relation between imitation learning and RL
Methodology (i.e., demonstrations / rewards…)
Scenario (different level of freedom)
Relation between imitation learning and supervised learning

4. Imitation Learning in a Nutshell
Given: demonstrations or demonstrator
Goal: train a policy to mimic demonstrations

5. Components

6. Some Applications

7. Notation

9. Running Example

10. The Simplest Setting of Imitation Learning
Behavioral Cloning

11. General Imitation Learning vs Behavioral Cloning

12. Limitations of Behavioral Cloning

13. When to use Behavioral Cloning

14. Types of Imitation Learning

15. Comparison

16. Interactive Direct Policy Learning

17. Learning Reductions

19. A Naïve Attempt
Not guaranteed to converge!

20. Sequential Learning Reductions

21. Data Aggregation (DAgger)

22. Policy Aggregation

23. Interactive Direct Policy Learning

24. Inverse Reinforcement Learning
Background for RL

25. Inverse Reinforcement Learning

26. Inverse Reinforcement Learning

27. Simplified version

31. More Complicated Situations…

32. Example

37. Recommended Reading ICML2018: Imitation Learning Tutorial
Imitation Learning: A Survey of Learning Methods
Learning to Search in Branch and Bound Algorithms (NIPS’2014) …