Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning Integrated Symbolic and Continuous Action Models Joseph Xu & John Laird May 29, 2013.

Published byPatience Peters Modified over 8 years ago

Similar presentations

Presentation on theme: "Learning Integrated Symbolic and Continuous Action Models Joseph Xu & John Laird May 29, 2013."— Presentation transcript:

1 Learning Integrated Symbolic and Continuous Action Models Joseph Xu & John Laird May 29, 2013

2 Action Models ?

3 Benefits Accurate action models allow for – Internal simulation – Backtracking planning – Learning policies via trial-and-error without incurring real-world cost 3 AgentWorld AgentModel Exploration Reward Exploration World Policy

4 Requirements Model learning should be Accurate Predictions made by model should be close to reality Fast Learn from few examples General Models should make good predictions in many situations Online Models shouldn’t require sampling entire space of possible actions before being useful 4

5 Continuous Environments Discrete objects with continuous properties – Geometry, position, rotation Input and output are vectors of continuous numbers Agent runs in lock-step with environment Fully observable 5 Output -9.0 5.8 Input EnvironmentAgent 0.21.20.0 px py rx ry A B 0.00.2 pz rz 3.43.90.0 px pypz 0.0 rx ryrz AB

6 Action Modeling in Continuous Domains 6

7 Locally Weighted Regression 7

8 ? x k nearest neighbors Weighted Linear Regression 8

9 LWR Shortcomings LWR generalizes based on proximity in pose space Smoothes together qualitatively distinct behaviors Generalizes with examples that are closer in absolute coordinates instead of similarity in object relationships 9 ?

10 LWR Shortcomings LWR generalizes based on proximity in pose space Smoothes together qualitatively distinct behaviors Generalizes with examples that are closer in absolute coordinates instead of similarity in object relationships 10

11 Our Approach Type of motion depends on relationships between objects, not absolute positions Learn models that exploits relational structure of the environment – Segment behaviors into qualitatively distinct linear motions (modes) – Classify which mode is in effect using relational structure Flying mode (no contact) Ramp rolling mode (touching ramp) Bouncing mode (touching flat surface) 11

12 Learning Multi-Modal Models 12 ~intersect(A,B) above(A,B) ball(A) Relational State intersect(A,B) above(A,B) ball(A) Time mode I mode II Segmentation Classification Relational Mode Classifier Continuous state B A B A 0.21.20.0 0.20.30.90.0 0.2 RANSAC + EMFOIL Scene Graph

13 Predict with Multi-Modal Models 13 ~intersect(A, B) Relational State mode I mode II Relational Mode Classifier Continuous state B A 0.21.20.0 0.2 Scene Graph prediction ~intersect(A,B) above(A,B) ball(A)

14 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise ball bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 RANSAC t = vy – 0.98

15 RANSAC Discover new modes 1.Choose random set of noise examples 2.Fit line to set 3.Add all noise examples that also fit line 4.If set is large (>40), create a new mode with those examples 5.Otherwise, repeat. 15 New mode Remaining noise 1.2.3.4.

16 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise ball bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 RANSAC t = vy – 0.98

17 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 ball EM

18 Expectation Maximization Simultaneously learn: – Association between training data and modes – Parameters for mode functions Expectation – Assume mode functions are correct – Compute likelihood that mode generated data point Maximization – Assume likelihoods are correct – Fit mode functions to maximize likelihood Iterate until convergence to local maximum 18

19 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 ball bx07, by07, vy07(b,p), ~(b,r)t07 Clause: ~(b,p) FOIL

20 Learn classifiers to distinguish between two modes (positives and negatives) based on relations Outer loop: Iteratively add clauses that cover the most positive examples – Inner loop: Iteratively add literals that rule out negative examples Object names are variablized for generality 20 Clause# pos. ex. ~intersect(target, any)16221 ~z-overlap(target, any)6162 east-of(target, x)36 ~x-overlap(target, any)21 east-of(x, target)9 ~ontop(target, any) & above(target, x)2

21 FOIL FOIL learns binary classifiers, but there can be many modes Use one-to-one strategy: – Learn classifier between each pair of modes – Each classifier votes between two modes – Mode with most votes wins

22 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 ball bx09, by09, vy09(b,p), ~(b,r)t09 bx08, by08, vy08(b,p), ~(b,r)t08 bx07, by07, vy07(b,p), ~(b,r)t07 RANSAC t = vy Clause: ~(b,p)

23 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 bx13, by13, vy13(b,p), ~(b,r)t13 bx12, by12, vy12(b,p), ~(b,r)t12 bx11, by11, vy11(b,p), ~(b,r)t11 bx10, by10, vy10(b,p), ~(b,r)t10 bx09, by09, vy09(b,p), ~(b,r)t09 bx08, by08, vy08(b,p), ~(b,r)t08 bx07, by07, vy07(b,p), ~(b,r)t07 t = vy ball Clause: (b,p) FOIL Clause: ~(b,p)

24 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 bx13, by13, vy13(b,p), ~(b,r)t13 bx12, by12, vy12(b,p), ~(b,r)t12 bx11, by11, vy11(b,p), ~(b,r)t11 bx10, by10, vy10(b,p), ~(b,r)t10 bx09, by09, vy09(b,p), ~(b,r)t09 bx08, by08, vy08(b,p), ~(b,r)t08 bx07, by07, vy07(b,p), ~(b,r)t07 t = vy ball bx16, by16, vy16~(b,p), (b,r)t16 bx15, by15, vy15~(b,p), (b,r)t15 bx14, by14, vy14~(b,p), (b,r)t14 RANSAC t = vy – 0.7 Clause: (b,p)Clause: ~(b,p)

25 staterelationstargstaterelationstargstaterelationstargstaterelationstarg platform ramp noise bx06, by06, vy06~(b,p), ~(b,r)t06 bx05, by05, vy05~(b,p), ~(b,r)t05 bx04, by04, vy04~(b,p), ~(b,r)t04 bx03, by03, vy03~(b,p), ~(b,r)t03 bx02, by02, vy02~(b,p), ~(b,r)t02 bx01, by01, vy01~(b,p), ~(b,r)t01 t = vy – 0.98 bx13, by13, vy13(b,p), ~(b,r)t13 bx12, by12, vy12(b,p), ~(b,r)t12 bx11, by11, vy11(b,p), ~(b,r)t11 bx10, by10, vy10(b,p), ~(b,r)t10 bx09, by09, vy09(b,p), ~(b,r)t09 bx08, by08, vy08(b,p), ~(b,r)t08 bx07, by07, vy07(b,p), ~(b,r)t07 t = vy ball bx16, by16, vy16~(b,p), (b,r)t16 bx15, by15, vy15~(b,p), (b,r)t15 bx14, by14, vy14~(b,p), (b,r)t14 t = vy – 0.7 Clause: (b,r)Clause: (b,p)Clause: ~(b,p) FOIL

26 Demo Physics simulation with ramp, box, and ball Learn models for x and y velocities link 26

27 Physics Simulation Experiment 2D physics simulation with gravity 40 possible configurations Training/Testing blocks run for 200 time steps 40 configs x 3 seeds = 120 training blocks Test over all 40 configs using different seed Repeat with 5 reorderings gravity random offset origin 27

28 Learned Modes ModeExpectedLearned X velocity flying or rolling on flat surface rolling or bouncing on ramp bouncing against vertical surface Y velocity rolling and bouncing on flat surface flying under influence of gravity rolling or bouncing on ramp 28

29 Prediction Accuracy Compare overall accuracy against single smooth function learner (LWR) 29

30 Classifier Accuracy Compare FOIL performance against classifiers using absolute coordinates (SVM, KNN) 30 KK

31 Nuggets Multi-modal approach addresses shortcomings of LWR – Doesn’t smooth over examples from different modes – Uses relational similarity to generalize behaviors Satisfies requirements – Accurate. New modes are learned for inaccurate predictions – Fast. Linear modes are learned from (too) few examples – General. Each mode generalizes to all relationally analogical situations – Online. Modes are learned incrementally and can immediately make predictions 31

32 Coals Slows down with more learning – keeps every training example Assumes linear modes RANSAC, EM, and FOIL are computationally expensive

Download ppt "Learning Integrated Symbolic and Continuous Action Models Joseph Xu & John Laird May 29, 2013."

Similar presentations

Artificial Neural Networks

Artificial Neural Networks
Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning
Neural Networks  A neural network is a network of simulated neurons that can be used to recognize instances of patterns. NNs learn by searching through.

Neural Networks  A neural network is a network of simulated neurons that can be used to recognize instances of patterns. NNs learn by searching through.
Data Mining Classification: Alternative Techniques

Data Mining Classification: Alternative Techniques
Support Vector Machines and Margins

Support Vector Machines and Margins
CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.
K Means Clustering , Nearest Cluster and Gaussian Mixture

K Means Clustering , Nearest Cluster and Gaussian Mixture
Indian Statistical Institute Kolkata

Indian Statistical Institute Kolkata
DARPA Mobile Autonomous Robot SoftwareMay 2000 1 Adaptive Intelligent Mobile Robotics William D. Smart, Presenter Leslie Pack Kaelbling, PI Artificial.

DARPA Mobile Autonomous Robot SoftwareMay Adaptive Intelligent Mobile Robotics William D. Smart, Presenter Leslie Pack Kaelbling, PI Artificial.
Statistical Classification Rong Jin. Classification Problems X Input Y Output ? Given input X={x 1, x 2, …, x m } Predict the class label y  Y Y = {-1,1},

Statistical Classification Rong Jin. Classification Problems X Input Y Output ? Given input X={x 1, x 2, …, x m } Predict the class label y  Y Y = {-1,1},
CMPUT 466/551 Principal Source: CMU

CMPUT 466/551 Principal Source: CMU
Classification Neural Networks 1

Classification Neural Networks 1
Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.

Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.
Formation et Analyse d’Images Session 8

Formation et Analyse d’Images Session 8
Motion Tracking. Image Processing and Computer Vision: 82 Introduction Finding how objects have moved in an image sequence Movement in space Movement.

Motion Tracking. Image Processing and Computer Vision: 82 Introduction Finding how objects have moved in an image sequence Movement in space Movement.
MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 

MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 
Topological Mapping using Visual Landmarks ● The work is based on the Team Localization: A Maximum Likelihood Approach paper. ● To simplify the problem,

Topological Mapping using Visual Landmarks ● The work is based on the "Team Localization: A Maximum Likelihood Approach" paper. ● To simplify the problem,
The value of kernel function represents the inner product of two training points in feature space Kernel functions merge two steps 1. map input data from.

The value of kernel function represents the inner product of two training points in feature space Kernel functions merge two steps 1. map input data from.
Iterative Optimization of Hierarchical Clusterings Doug Fisher Department of Computer Science, Vanderbilt University Journal of Artificial Intelligence.

Iterative Optimization of Hierarchical Clusterings Doug Fisher Department of Computer Science, Vanderbilt University Journal of Artificial Intelligence.
1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.

1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.

Similar presentations

Ads by Google