Download presentation
Presentation is loading. Please wait.
1
Reinforcement Learning Presented by: Kyle Feuz
3
Outline Motivation MDPs RL Model-Based Model-Free Q-Learning SARSA Challenges
4
Examples Pac-Man Spider
5
MDPs 4-tuple (State, Actions, Transitions, Rewards).
6
Important Terms Policy Reward Function Value Function Model
7
Model-Based RL Learn transition function Learn expected rewards Compute the optimal policy
8
Model-Free RL Learn expected rewards/values Skip learning transistion function Trade-offs?
9
Basic Equations
10
Examples Pac-Man Spider Mario
11
Q-Learning Q(s, a) = = (1 − α)Q(s, a) + α[R(s, s′ ) + Max Q(s′, a′ )]
12
Q-Learning Demo Video
13
SARSA Q-Learning Q(s, a) = = (1 − α)Q(s, a) + α[R(s, s′ ) + Q(s′, a′ )]
14
Challenges Explore vs. Exploit State Space representation Training Time Multiagent Learning Moving Target Competive or Cooperative
15
Transfer Learning for Reinforcement Learning on a Physical Robot Applied TL and RL on Nao robot TL using the q-value reuse approach RL uses SARSA variant State space is represented via CMAC Neural Network inspired by the cerebellum Acts as an associative memory Allows agents to generalize the state space
16
Agent Model
17
SARSA Update Rule Q(s, a) = = (1 − α)Q(s, a) + α[R(s, s′ ) + γe(s, a)Q(s′, a′ )]
18
Q-Value Reuse Q(s, a) = = Qsource (χX (s), χA (a)) + Qtarget (s, a)
19
Experimental Setup Seated Nao robot Hit the ball at 45 angle 5 Actions in Source – 9 Actions in Target
20
Robot Results
21
Simulator Results
22
Advanced Combinations
23
Examples Pac-Man Spider Mario Q-Learning Penalty Kick Others
24
References and Resources rl repository rl-community rl on PBWorks rl warehouse Reinforcement Learning: An Introduction Artificial Intelligence: A Modern Approach How to Make Software Agents do the Right Thing How to Make Software Agents do the Right Thing
25
Questions?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.