BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003

Problem The goal is to track an unknown number of blobs from static camera video.

Solution The Bayesian Multiple-BLob (BraMBLe) tracker is a Bayesian solution. It estimates State at frame t Image Sequence Number, Positions, Shapes, Velocities, …

Bayes Rule Posterior State Distribution Observation Likelihood Prior

Sequential Bayes Posterior State Distribution Observation Likelihood Prior Instead of modeling directly, BraMBLe models and.

Update Algorithm

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

Image Observations We want to choose our observations so that we can compute quickly: Individual observations are conditionally independent.

Image Observations We want. A bank of filters is applied at each grid point. Y Cr Cb Filter plots from Mexican Hat Gaussian

Image Observations We want to choose our model so that we can compute quickly: Observation depends on membership of grid point.

Image Observations We want to choose our model so that we can compute quickly: We can precompute and quickly evaluate any state x.

Appearance Models The appearance models are learned from training data. Training Data

Observation Likelihood

Observation Likelihood Review We defined our image observations so that We defined our observation model so that We can precompute and quickly evaluate for many choices of x.

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

Object Model The blob configuration is Number of objects Object State

Object Model The blob configuration is The object state is Identity Location Velocity Shape

Person Model Generalized-Cylinder Model Calibrated Camera

Prediction Model The number of objects can change:  Each object has a constant probability of remaining in the scene.  There is a constant probability that a new object will enter the scene. In this formulation, hypotheses with different numbers of objects can be compared directly.

Prediction Model Damped constant location velocity:

Prediction Model Damped constant location velocity: Auto-regressive shape:

Model Review The observation likelihood is fast to compute for different hypotheses. The prediction model allows generation of from Estimating requires an efficient way of  Representing.  Computing the multiplications and integrations.

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

Efficient Representation is represented by a set of particles, : Sampling from the set using the weights approximates sampling from N Points: N Weights:

Efficient Representation is represented by a set of particles, : Sampling from the set using the weights approximates sampling from

Efficient Computation The particle set representing is computed from by C ONDENSATION : 1.Apply dynamics to the particle set: 2.Multiply by the observation likelihood:

Applying Dynamics Given particle set, compute Image from

Applying Dynamics Given particle set, compute 1.Resample into Image from

Applying Dynamics Given particle set, compute 1.Resample into 2.Predict, generating to give

Multiplication by Likelihood Given particle set, compute Weight particles, setting Image from

Efficient Computation Review The particle set representing is computed from by C ONDENSATION: Resample Predict Reweight Image from

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

People Tracking

Tracking was successful in real time on this 53s clip except when two people crossed in front of a third.

Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

Algorithm Summary The models chosen  Are a smooth integration of foreground and background models.  Allow hypotheses with differing numbers of objects to be compared directly.  Can be quickly evaluated in a particle filtering implementation.

Relationship to Previous Work Static camera blob tracking:  Classifies pixels as foreground or background. Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts Video from

Relationship to Previous Work Static camera blob tracking:  Classifies pixels as foreground or background.  Static camera blob tracking:  Finds the position in the search area Made up of foreground pixels. Matching the blob in the previous frame. Frame t - 1 Frame t Frame t + 1 Predicted position in frame t + 1 Search area

Relationship to Previous Work Improvements over blob tracking:  Integrates the foreground and background modeling.  Multiple objects can be tracked through occlusions. Video from from

Weaknesses The algorithm is sensitive to reflections. The algorithm sometimes switches the labels when one object passes in front of another. There are a lot of parameters to assign.