1. Probabilistic Reasoning Over Time
CSE P573
Autumn 2004

2. questions What is the Markov assumption?
What is difference between filtering and smoothing?
Is finding the most likely sequence of states the same as finding the sequence of most likely states? What algorithm do you use?

3. questions
Is a Kalman filter appropriate for discrete or for continuous variables?
What kinds of distributions does it handle?

4. questions
What is the main advantage of an using an HMM (hidden Markov model) over using a DBN (Dynamic Bayesian Network)?
What is the main advantage of using a DBN over an HMM?

5. questions
What is a "particle" as used in particle filtering algorithms?
Go on to Ch 15 slides…
Go on to Robotics slides…

6. Track the Robot

7. Particle Filtering: Core Idea
Initialize particles S randomly with weight 1
For each observation yt {
For each particle s S {
Choose a sample s’ according to P(Xt=s’|Xt-1=s)
s = s’
w(s) = P(Yt=yt|Xt=s) * w(s)
} }

8. Particle Filtering: Resampling
After every K-th observation is processed:
Randomly select (with replacement) a new set of particles S’ according to the distribution {w(s) | s  S}
S = S’
For all s  S { w(s)=1 }
Resampling KILLS unlikely particles
Resampling DUPLICATES likely particles

9. Particle Filtering: Computing the Belief State
Compute P(Xt=x | y1, …, yt) as:
Sum( w(s) | s  S & value(s)=x ) /
Sum( w(s) | s  S )

10. Shakey
Shakey Video