10/28 Temporal Probabilistic Models. Temporal (Sequential) Process A temporal process is the evolution of system state over time Often the system state.

Slides:



Advertisements
Similar presentations
State Estimation and Kalman Filtering CS B659 Spring 2013 Kris Hauser.
Advertisements

SA-1 Probabilistic Robotics Planning and Control: Partially Observable Markov Decision Processes.
Dynamic Bayesian Networks (DBNs)
Lirong Xia Hidden Markov Models Tue, March 28, 2014.
Lirong Xia Approximate inference: Particle filter Tue, April 1, 2014.
Hidden Markov Models Reading: Russell and Norvig, Chapter 15, Sections
Introduction of Probabilistic Reasoning and Bayesian Networks
Chapter 15 Probabilistic Reasoning over Time. Chapter 15, Sections 1-5 Outline Time and uncertainty Inference: ltering, prediction, smoothing Hidden Markov.
1 Slides for the book: Probabilistic Robotics Authors: Sebastian Thrun Wolfram Burgard Dieter Fox Publisher: MIT Press, Web site for the book & more.
Advanced Artificial Intelligence
1 Reasoning Under Uncertainty Over Time CS 486/686: Introduction to Artificial Intelligence Fall 2013.
10/24  Exam on 10/26 (Lei Tang and Will Cushing to proctor)
CS 188: Artificial Intelligence Fall 2009 Lecture 20: Particle Filtering 11/5/2009 Dan Klein – UC Berkeley TexPoint fonts used in EMF. Read the TexPoint.
CS 547: Sensing and Planning in Robotics Gaurav S. Sukhatme Computer Science Robotic Embedded Systems Laboratory University of Southern California
CS 547: Sensing and Planning in Robotics Gaurav S. Sukhatme Computer Science Robotic Embedded Systems Laboratory University of Southern California
Part 2 of 3: Bayesian Network and Dynamic Bayesian Network.
Probabilistic Robotics Introduction Probabilities Bayes rule Bayes filters.
CPSC 322, Lecture 31Slide 1 Probability and Time: Markov Models Computer Science cpsc322, Lecture 31 (Textbook Chpt 6.5) March, 25, 2009.
CS 188: Artificial Intelligence Spring 2007 Lecture 14: Bayes Nets III 3/1/2007 Srini Narayanan – ICSI and UC Berkeley.
11/14  Continuation of Time & Change in Probabilistic Reasoning Project 4 progress? Grade Anxiety? Make-up Class  On Monday?  On Wednesday?
CS 188: Artificial Intelligence Fall 2006 Lecture 17: Bayes Nets III 10/26/2006 Dan Klein – UC Berkeley.
CPSC 322, Lecture 32Slide 1 Probability and Time: Hidden Markov Models (HMMs) Computer Science cpsc322, Lecture 32 (Textbook Chpt 6.5) March, 27, 2009.
10/22  Homework 3 returned; solutions posted  Homework 4 socket opened  Project 3 assigned  Mid-term on Wednesday  (Optional) Review session Tuesday.
Announcements Homework 8 is out Final Contest (Optional)
Computer vision: models, learning and inference Chapter 10 Graphical Models.
CS 188: Artificial Intelligence Fall 2009 Lecture 19: Hidden Markov Models 11/3/2009 Dan Klein – UC Berkeley.
Dynamic Bayesian Networks CSE 473. © Daniel S. Weld Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanningLearning.
1 Bayesian Networks Chapter ; 14.4 CS 63 Adapted from slides by Tim Finin and Marie desJardins. Some material borrowed from Lise Getoor.
Learning Models of Relational Stochastic Processes Sumit Sanghai.
Markov Localization & Bayes Filtering
QUIZ!!  T/F: The forward algorithm is really variable elimination, over time. TRUE  T/F: Particle Filtering is really sampling, over time. TRUE  T/F:
Recap: Reasoning Over Time  Stationary Markov models  Hidden Markov models X2X2 X1X1 X3X3 X4X4 rainsun X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3.
1 Robot Environment Interaction Environment perception provides information about the environment’s state, and it tends to increase the robot’s knowledge.
Temporal Models Template Models Representation Probabilistic Graphical
CPSC 322, Lecture 32Slide 1 Probability and Time: Hidden Markov Models (HMMs) Computer Science cpsc322, Lecture 32 (Textbook Chpt 6.5.2) Nov, 25, 2013.
UIUC CS 498: Section EA Lecture #21 Reasoning in Artificial Intelligence Professor: Eyal Amir Fall Semester 2011 (Some slides from Kevin Murphy (UBC))
Bayes’ Nets: Sampling [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available.
Processing Sequential Sensor Data The “John Krumm perspective” Thomas Plötz November 29 th, 2011.
The famous “sprinkler” example (J. Pearl, Probabilistic Reasoning in Intelligent Systems, 1988)
QUIZ!!  In HMMs...  T/F:... the emissions are hidden. FALSE  T/F:... observations are independent given no evidence. FALSE  T/F:... each variable X.
Tractable Inference for Complex Stochastic Processes X. Boyen & D. Koller Presented by Shiau Hong Lim Partially based on slides by Boyen & Koller at UAI.
1 Chapter 15 Probabilistic Reasoning over Time. 2 Outline Time and UncertaintyTime and Uncertainty Inference: Filtering, Prediction, SmoothingInference:
CS 188: Artificial Intelligence Bayes Nets: Approximate Inference Instructor: Stuart Russell--- University of California, Berkeley.
1 CMSC 671 Fall 2001 Class #20 – Thursday, November 8.
1 Chapter 17 2 nd Part Making Complex Decisions --- Decision-theoretic Agent Design Xin Lu 11/04/2002.
The Unscented Particle Filter 2000/09/29 이 시은. Introduction Filtering –estimate the states(parameters or hidden variable) as a set of observations becomes.
Rao-Blackwellised Particle Filtering for Dynamic Bayesian Network Arnaud Doucet Nando de Freitas Kevin Murphy Stuart Russell.
CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov
Reasoning over Time  Often, we want to reason about a sequence of observations  Speech recognition  Robot localization  User attention  Medical monitoring.
Probabilistic Robotics Introduction Probabilities Bayes rule Bayes filters.
Probabilistic Robotics Probability Theory Basics Error Propagation Slides from Autonomous Robots (Siegwart and Nourbaksh), Chapter 5 Probabilistic Robotics.
Instructor: Eyal Amir Grad TAs: Wen Pu, Yonatan Bisk Undergrad TAs: Sam Johnson, Nikhil Johri CS 440 / ECE 448 Introduction to Artificial Intelligence.
CS 541: Artificial Intelligence Lecture VIII: Temporal Probability Models.
CS 541: Artificial Intelligence Lecture VIII: Temporal Probability Models.
CS498-EA Reasoning in AI Lecture #23 Instructor: Eyal Amir Fall Semester 2011.
HMM: Particle filters Lirong Xia. HMM: Particle filters Lirong Xia.
Probabilistic reasoning over time
Artificial Intelligence
Markov ó Kalman Filter Localization
CS 4/527: Artificial Intelligence
Probabilistic Reasoning Over Time
CAP 5636 – Advanced Artificial Intelligence
Inference Inference: calculating some useful quantity from a joint probability distribution Examples: Posterior probability: Most likely explanation: B.
Instructors: Fei Fang (This Lecture) and Dave Touretzky
Instructors: Fei Fang (This Lecture) and Dave Touretzky
CS 188: Artificial Intelligence
CS 188: Artificial Intelligence Fall 2008
Chapter14-cont..
Hidden Markov Models Lirong Xia.
Probabilistic reasoning over time
Presentation transcript:

10/28 Temporal Probabilistic Models

Temporal (Sequential) Process A temporal process is the evolution of system state over time Often the system state is hidden, and we need to reconstruct the state from the observations Relation to Planning: –When you are observing a temporal process, you are observing the execution trace of someone else’s plan…

State Estimation… In non-deterministic scenarios, observation from a state had to be certain I.e., P(o|s) must be either 0 or 1. Saying “state s may give observation o doesn’t make sense since there is no way of exploiting that information. In stochastic scenarios, P(o|s) can be a number between 0 and 1, and this makes for pretty interesting reasoning opportunities..

Stationarity assumption over time is akin to “relational” assumption over objects

Dynamic Bayes Networks are “templates” for specifying the relation between the values of a random variable across time-slices  e.g. How is Rain at time t related to Rain at time t+1? We call them templates because they need to be expanded (unfolded) to the required number of time steps to reason about the connection between variables at different time points

While DBNs are special cases of B.N.’s there are a certain inference tasks that are particularly frequently useful for them (Notice that all of them involve estimating posterior probability distributions—as is done in any B.N. inference) Most inference tasks in DBNs will be about posterior distributions over single or multiple state variables.

Can do much better if we exploit the repetitive structure Both Exact and Approximate B.N. Inference methods can be made to take the temporal structure into account.  Specialized variable-elimination method  Unfold t+1 th level, and roll-up t th level by variable elimination  Specialized Likelihood-weighting methods that take evidence into account  Particle Filtering Techniques

Can do much better if we exploit the repetitive structure Both Exact and Approximate B.N. Inference methods can be made to take the temporal structure into account.  Specialized variable-elimination method  Unfold t+1 th level, and roll-up t th level by variable elimination  Specialized Likelihood-weighting methods that take evidence into account  Particle Filtering Techniques

Notice that to attach the likelihood to the evidence, we are using the CPTs in the bayes net. (Model-free empirical observation, in contrast, either gives you a sample or not; we can’t get fractional samples)

Normal LW takes each sample through the network one by one Idea 1: Take then all from t to t+1 lock-step  the samples are the distribution Normal LW doesn’t do well when the evidence is downstream (the sample weight will be too small) In DBN, none of the evidence is affecting the sampling! EVEN MORE of an issue

Idea: Put samples back into high probability regions (particle ~ sample)

A Robot localizing itself using particle filters

Factored action representations for MDPs Based on Section 4.2 of Boutilier, Hanks & Dean

What does IPPC use?

2-TBNs model effect on each each variable separately  Need “persistence” links over unaffected variables PSO’s (probabilistic STRIPS operators) Explicitly represent complete outcomes  bad when an action has multiple independent stochastic effects Synchronic vs. diachronic dependencies Simple 2-TBN have only diachronic dependencies

Special Cases of DBNs are well known in the literature Restrict number of variables per state –Markov Chain: DBN with one variable that is fully observable –Hidden Markov Model: DBN with only one state variable that is hidden and can be estimated through evidence variable(s) Restrict the type of CPD –Kalman Filters: DBN where the system transition function as well as the observation variable are linear gaussian The advantage of Gaussians is that the posterior distribution remains Gaussian

Class Ended here.. Slides beyond this not discussed