Helsinki University of Technology Adaptive Informatics Research Centre Finland Variational Bayesian Approach for Nonlinear Identification and Control Matti.

Slides:



Advertisements
Similar presentations
1 -Classification: Internal Uncertainty in petroleum reservoirs.
Advertisements

Bayesian Belief Propagation
Introduction to Data Assimilation NCEO Data-assimilation training days 5-7 July 2010 Peter Jan van Leeuwen Data Assimilation Research Center (DARC) University.
State Estimation and Kalman Filtering CS B659 Spring 2013 Kris Hauser.
Neural networks Introduction Fitting neural networks
Salvatore giorgi Ece 8110 machine learning 5/12/2014
Computer vision: models, learning and inference Chapter 8 Regression.
1 Approximated tracking of multiple non-rigid objects using adaptive quantization and resampling techniques. J. M. Sotoca 1, F.J. Ferri 1, J. Gutierrez.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 10: The Bayesian way to fit models Geoffrey Hinton.
Probabilistic Robotics SLAM. 2 Given: The robot’s controls Observations of nearby features Estimate: Map of features Path of the robot The SLAM Problem.
On-Line Probabilistic Classification with Particle Filters Pedro Højen-Sørensen, Nando de Freitas, and Torgen Fog, Proceedings of the IEEE International.
Observers and Kalman Filters
Bayesian Robust Principal Component Analysis Presenter: Raghu Ranganathan ECE / CMR Tennessee Technological University January 21, 2011 Reading Group (Xinghao.
Presenter: Yufan Liu November 17th,
Pattern Recognition and Machine Learning
280 SYSTEM IDENTIFICATION The System Identification Problem is to estimate a model of a system based on input-output data. Basic Configuration continuous.
Reliable Range based Localization and SLAM Joseph Djugash Masters Student Presenting work done by: Sanjiv Singh, George Kantor, Peter Corke and Derek Kurth.
CS 547: Sensing and Planning in Robotics Gaurav S. Sukhatme Computer Science Robotic Embedded Systems Laboratory University of Southern California
Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE.
SLAM: Simultaneous Localization and Mapping: Part I Chang Young Kim These slides are based on: Probabilistic Robotics, S. Thrun, W. Burgard, D. Fox, MIT.
Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.
Tracking using the Kalman Filter. Point Tracking Estimate the location of a given point along a sequence of images. (x 0,y 0 ) (x n,y n )
Particle Filters for Mobile Robot Localization 11/24/2006 Aliakbar Gorji Roborics Instructor: Dr. Shiri Amirkabir University of Technology.
Arizona State University DMML Kernel Methods – Gaussian Processes Presented by Shankar Bhargav.
Particle Filtering for Non- Linear/Non-Gaussian System Bohyung Han
Estimation and the Kalman Filter David Johnson. The Mean of a Discrete Distribution “I have more legs than average”
CSC2535: 2013 Advanced Machine Learning Lecture 3a: The Origin of Variational Bayes Geoffrey Hinton.
Gaussian process regression Bernád Emőke Gaussian processes Definition A Gaussian Process is a collection of random variables, any finite number.
Kalman filtering techniques for parameter estimation Jared Barber Department of Mathematics, University of Pittsburgh Work with Ivan Yotov and Mark Tronzo.
Computer vision: models, learning and inference Chapter 19 Temporal models.
Jamal Saboune - CRV10 Tutorial Day 1 Bayesian state estimation and application to tracking Jamal Saboune VIVA Lab - SITE - University.
Probabilistic Robotics Bayes Filter Implementations.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.
Young Ki Baik, Computer Vision Lab.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 11: Bayesian learning continued Geoffrey Hinton.
Mean Field Variational Bayesian Data Assimilation EGU 2012, Vienna Michail Vrettas 1, Dan Cornford 1, Manfred Opper 2 1 NCRG, Computer Science, Aston University,
Human-Computer Interaction Kalman Filter Hanyang University Jong-Il Park.
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 1 Dynamic Sensor Resource Management for ATE MURI.
17 May 2007RSS Kent Local Group1 Quantifying uncertainty in the UK carbon flux Tony O’Hagan CTCD, Sheffield.
-Arnaud Doucet, Nando de Freitas et al, UAI
Mobile Robot Localization (ch. 7)
Processing Sequential Sensor Data The “John Krumm perspective” Thomas Plötz November 29 th, 2011.
Real-Time Simultaneous Localization and Mapping with a Single Camera (Mono SLAM) Young Ki Baik Computer Vision Lab. Seoul National University.
Chapter 3: Maximum-Likelihood Parameter Estimation l Introduction l Maximum-Likelihood Estimation l Multivariate Case: unknown , known  l Univariate.
ECE-7000: Nonlinear Dynamical Systems Overfitting and model costs Overfitting  The more free parameters a model has, the better it can be adapted.
An Introduction to Kalman Filtering by Arthur Pece
NCAF Manchester July 2000 Graham Hesketh Information Engineering Group Rolls-Royce Strategic Research Centre.
Gaussian Processes For Regression, Classification, and Prediction.
Short Introduction to Particle Filtering by Arthur Pece [ follows my Introduction to Kalman filtering ]
Tracking with dynamics
Nonlinear State Estimation
The Unscented Particle Filter 2000/09/29 이 시은. Introduction Filtering –estimate the states(parameters or hidden variable) as a set of observations becomes.
Machine Learning CUNY Graduate Center Lecture 6: Linear Regression II.
École Doctorale des Sciences de l'Environnement d’ Î le-de-France Année Modélisation Numérique de l’Écoulement Atmosphérique et Assimilation.
École Doctorale des Sciences de l'Environnement d’Île-de-France Année Universitaire Modélisation Numérique de l’Écoulement Atmosphérique et Assimilation.
Ch 1. Introduction Pattern Recognition and Machine Learning, C. M. Bishop, Updated by J.-H. Eom (2 nd round revision) Summarized by K.-I.
CSC321: Lecture 8: The Bayesian way to fit models Geoffrey Hinton.
Probabilistic Robotics Bayes Filter Implementations Gaussian filters.
8 Sept 2006, DEMA2006Slide 1 An Introduction to Computer Experiments and their Design Problems Tony O’Hagan University of Sheffield.
ICS 280 Learning in Graphical Models
Ch3: Model Building through Regression
Particle Filtering for Geometric Active Contours
PSG College of Technology
Unscented Kalman Filter
Introduction to particle filter
Introduction to particle filter
Probabilistic Robotics
Kalman Filtering COS 323.
Neural networks (3) Regularization Autoencoder
Linear Discrimination
Presentation transcript:

Helsinki University of Technology Adaptive Informatics Research Centre Finland Variational Bayesian Approach for Nonlinear Identification and Control Matti Tornio and Tapani Raiko October 9, 2006

Introduction System identification and control in nonlinear state-space models Continues the work by Rosenqvist and Karlström (Automatica 2005) Our background is in machine learning Uncertainties taken explicitly into account by using Variational Bayesian treatment

Why nonlinear state-space models? System identification using a hidden state has many benefits: More resistant to noise Observations (without history) do not always carry enough information about the system state Finds a representation of the state that is more suitable for approximating the dynamics

System identification in nonlinear state-space models We use a state-of-the-art tool by Valpola and Karhunen (Neural Computation 2002) Parameters, states, and observations are modelled with Gaussian distributions Less prone to overfitting (than the prediction error method) Can determine the dimensionality of the state space etc.

Properties of the method The model scales well to higher dimensions Can model very complex dynamics Natural conjugate gradient algorithm is used for fast system identification

Nonlinearities by MLP networks f(x(t),θ)=B tanh[Ax(t)+a] + b + noise The parameters θ include the weight matrices, bias vectors, noise variances etc. Note that the policy mapping does not fix the control signal (because of the noise model)

Variational Bayesian treatment Posterior probability p(x,θ|y) is approximated by q(x,θ) q is assumed to be Gaussian with limited dependencies The fit of q to p is measured by a cost function Both identification and prediction can be done by minimising the misfit by adjusting the parameters defining q (means, variances, covariances)

Control Current state is estimated with extended Kalman filter (EKF) Control signals u(t) are selected to minimise the expected cost E{J} over the distribution q Quasi-Newton algorithm for optimisation Compare to dual control: estimation errors increase the expected cost

Control (cont.) Prediction with variances is ~5 times slower  too slow for some applications, the method can still be used for system identification Learning is done offline  online learning possible as well, leads to different exploration strategies

Optimistic inference control Alternative control scheme Observations at some point in the future are fixed and the states leading to this desired future are inferred Allows the direct use of inference algorithms Conceptually very simple, but not as versatile as NMPC  constraints hard to model

Experiments Assume the cart-pole system to be unknown Dynamics are identified from only 2500 samples 6-dimensional state space x(t) was used

Results Very high success rate was reached even under high noise Partially observed system is hard to control Low noiseHigh noise

Results (initialisation) Good initialisations are important  Local minima are the biggest problem Internal forward model can provide reasonable initialisations without significant extra computation

Conclusion Learning nonlinear state-space models seems promising when  observations about the system state are incomplete or  the dynamics of the system are not well known Variational Bayesian treatment helps  against overfitting  to determine the model order