Probabilistic Reasoning Over Time CSE P573 Autumn 2004

questions What is the Markov assumption? What is difference between filtering and smoothing? Is finding the most likely sequence of states the same as finding the sequence of most likely states? What algorithm do you use?

questions Is a Kalman filter appropriate for discrete or for continuous variables? What kinds of distributions does it handle?

questions What is the main advantage of an using an HMM (hidden Markov model) over using a DBN (Dynamic Bayesian Network)? What is the main advantage of using a DBN over an HMM?

questions What is a "particle" as used in particle filtering algorithms? Go on to Ch 15 slides… Go on to Robotics slides…

Track the Robot

Particle Filtering: Core Idea Initialize particles S randomly with weight 1 For each observation yt { For each particle s S { Choose a sample s’ according to P(Xt=s’|Xt-1=s) s = s’ w(s) = P(Yt=yt|Xt=s) * w(s) } }

Particle Filtering: Resampling After every K-th observation is processed: Randomly select (with replacement) a new set of particles S’ according to the distribution {w(s) | s  S} S = S’ For all s  S { w(s)=1 } Resampling KILLS unlikely particles Resampling DUPLICATES likely particles

Particle Filtering: Computing the Belief State Compute P(Xt=x | y1, …, yt) as: Sum( w(s) | s  S & value(s)=x ) / Sum( w(s) | s  S )

Shakey Shakey Video