1. Model-Free Episodic Control
Name
Ze Liu
Data

2. CONTENTS 01 02 03 04 05 INTRODUCTION INSPIRATION AND WHAT TO SOLVE
ALGORITHMS 04
EXPERIMENTAL RESULTS 05
REFERENTIAL VALUE

3. Model-Based or
Model-Free
Episodic
Control
INTRODUCTION
PART 01

4. MDP S: a set of states A: a set of actions
PART ONE INTRODUCTION
MDP
S: a set of states
A: a set of actions
Ps′s,a: the probability that action a in state s will lead to state s'
Rs,a: the immediate reward received after
transitioning from state s to state s', due to action a
γ: the discount factor

5. PART ONE INTRODUCTION
Model-Free
Model-based

6. PART ONE INTRODUCTION
Episodic Control

7. INSPIRATION AND
WHAT TO SOLVE
PART 02

8. Traditional RL is data inefficient and too slow to train A
PART TWO INSPIRATION AND WHAT TO SOLVE
Traditional RL is data inefficient
and too slow to train A
Traditional RL algorithms take many millions of interactions to attain human-level performance.
Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. B
Research on the brain
Study find that the hippocampal system may be used to guide sequential decision-making by co-representing environment states with the returns achieved from the various possible actions

9. ALGORITHMS
PART 03

10. PART THREE ALGORITHMS

11. PART THREE ALGORITHMS
Writing
Look-up

12. PART THREE ALGORITHMS

13. PART THREE ALGORITHMS RP

14. PART THREE ALGORITHMS
VAE

15. EXPERIMENTAL RESULTS
PART 04

16. PART FOUR EXPERIMENTAL RESULTS

17. REFERENTIAL VALUE
PART 05

18. THANK YOU FOR WATCHING