EECS 274 Computer Vision Tracking. Motivation: Obtain a compact representation from an image/motion sequence/set of tokens Should support application.

Slides:



Advertisements
Similar presentations
CSCE643: Computer Vision Mean-Shift Object Tracking Jinxiang Chai Many slides from Yaron Ukrainitz & Bernard Sarel & Robert Collins.
Advertisements

CSCE643: Computer Vision Bayesian Tracking & Particle Filtering Jinxiang Chai Some slides from Stephen Roth.
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Dynamic Bayesian Networks (DBNs)
Introduction To Tracking
Visual Recognition Tutorial
Formation et Analyse d’Images Session 8
Lecture 4 Linear Filters and Convolution
Prof. Trevor Darrell Lecture 19: Tracking
Tracking Objects with Dynamics Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 04/21/15 some slides from Amin Sadeghi, Lana Lazebnik,
Motion Tracking. Image Processing and Computer Vision: 82 Introduction Finding how objects have moved in an image sequence Movement in space Movement.
Mean Shift Theory and Applications
Image processing. Image operations Operations on an image –Linear filtering –Non-linear filtering –Transformations –Noise removal –Segmentation.
Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE.
Ensemble Tracking Shai Avidan IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE February 2007.
Nonlinear and Non-Gaussian Estimation with A Focus on Particle Filters Prasanth Jeevan Mary Knox May 12, 2006.
Maximum Likelihood (ML), Expectation Maximization (EM)
Visual Recognition Tutorial
Computer Vision I Instructor: Prof. Ko Nishino. Today How do we recognize objects in images?
© 2003 by Davi GeigerComputer Vision November 2003 L1.1 Tracking We are given a contour   with coordinates   ={x 1, x 2, …, x N } at the initial frame.
Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.
Novel approach to nonlinear/non- Gaussian Bayesian state estimation N.J Gordon, D.J. Salmond and A.F.M. Smith Presenter: Tri Tran
Dorin Comaniciu Visvanathan Ramesh (Imaging & Visualization Dept., Siemens Corp. Res. Inc.) Peter Meer (Rutgers University) Real-Time Tracking of Non-Rigid.
Tracking with Linear Dynamic Models. Introduction Tracking is the problem of generating an inference about the motion of an object given a sequence of.
Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)
Lecture II-2: Probability Review
Particle Filters++ TexPoint fonts used in EMF.
Radial Basis Function Networks
Elec471 Embedded Computer Systems Chapter 4, Probability and Statistics By Prof. Tim Johnson, PE Wentworth Institute of Technology Boston, MA Theory and.
Crash Course on Machine Learning
PATTERN RECOGNITION AND MACHINE LEARNING
Mean Shift : A Robust Approach Toward Feature Space Analysis - Bayesian Background Modeling
Mean-shift and its application for object tracking
TP15 - Tracking Computer Vision, FCUP, 2013 Miguel Coimbra Slides by Prof. Kristen Grauman.
Human-Computer Interaction Human-Computer Interaction Tracking Hanyang University Jong-Il Park.
Computer vision: models, learning and inference Chapter 19 Temporal models.
SIS Sequential Importance Sampling Advanced Methods In Simulation Winter 2009 Presented by: Chen Bukay, Ella Pemov, Amit Dvash.
Computer vision: models, learning and inference Chapter 19 Temporal models.
Computer Vision - A Modern Approach Set: Tracking Slides by D.A. Forsyth The three main issues in tracking.
ECE 8443 – Pattern Recognition LECTURE 07: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Class-Conditional Density The Multivariate Case General.
EECS 274 Computer Vision Segmentation by Clustering II.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: ML and Simple Regression Bias of the ML Estimate Variance of the ML Estimate.
CS654: Digital Image Analysis Lecture 30: Clustering based Segmentation Slides are adapted from:
ECE-7000: Nonlinear Dynamical Systems Overfitting and model costs Overfitting  The more free parameters a model has, the better it can be adapted.
Boosted Particle Filter: Multitarget Detection and Tracking Fayin Li.
Real-Time Tracking with Mean Shift Presented by: Qiuhua Liu May 6, 2005.
Mean Shift ; Theory and Applications Presented by: Reza Hemati دی 89 December گروه بینایی ماشین و پردازش تصویر Machine Vision and Image Processing.
Tracking with dynamics
Lecture 3: MLE, Bayes Learning, and Maximum Entropy
Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION Kalman Filter with Process Noise Gauss- Markov.
Instructor: Mircea Nicolescu Lecture 7
Computational Intelligence: Methods and Applications Lecture 26 Density estimation, Expectation Maximization. Włodzisław Duch Dept. of Informatics, UMK.
ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Mixture Densities Maximum Likelihood Estimates.
Tracking We are given a contour G1 with coordinates G1={x1 , x2 , … , xN} at the initial frame t=1, were the image is It=1 . We are interested in tracking.
Tracking Objects with Dynamics
Particle Filtering for Geometric Active Contours
A segmentation and tracking algorithm
Statistical Methods For Engineers
Hidden Markov Models Part 2: Algorithms
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
LECTURE 21: CLUSTERING Objectives: Mixture Densities Maximum Likelihood Estimates Application to Gaussian Mixture Models k-Means Clustering Fuzzy k-Means.
Announcements Project 2 artifacts Project 3 due Thursday night
Mean Shift Theory and Applications
CHAPTER – 1.2 UNCERTAINTIES IN MEASUREMENTS.
Kalman Filter: Bayes Interpretation
Tracking Many slides adapted from Kristen Grauman, Deva Ramanan.
Nome Sobrenome. Time time time time time time..
CHAPTER – 1.2 UNCERTAINTIES IN MEASUREMENTS.
Presentation transcript:

EECS 274 Computer Vision Tracking

Motivation: Obtain a compact representation from an image/motion sequence/set of tokens Should support application Broad theory is absent at present Some slides from R. Collins lecture notes Reading: FP Chapter 17

Tracking Very general model: –We assume there are moving objects, which have an underlying state X –There are measurements Y, some of which are functions of this state –There is a clock at each tick, the state changes at each tick, we get a new observation Examples –object is ball, state is 3D position+velocity, measurements are stereo pairs –object is person, state is body configuration, measurements are frames, clock is in camera (30 fps)

Three main steps

Simplifying assumptions

Tracking as induction Assume data association is done –we’ll talk about this later; a dangerous assumption Do correction for the 0’th frame Assume we have corrected estimate for i’th frame –show we can do prediction for i+1, correction for i+1

General model for tracking The moving object of interest is characterized by an underlying state X State X gives rise to measurements or observations Y At each time t, the state changes to X t and we get a new observation Y t X1X1 X2X2 Y1Y1 Y2Y2 XtXt YtYt … Can be better explained with graphical model

Base case

Induction step Given

Induction step

Explanation with graphical model

Linear dynamic models Use notation ~ to mean “has the pdf of”, N(a, b) is a normal distribution with mean a and covariance b. Then a linear dynamic model has the form This is much, much more general than it looks, and extremely powerful D: transition matrix M: observation (measurement) matrix

Propagation of Gaussian densities

Examples Drifting points –we assume that the new position of the point is the old one, plus noise. –For the measurement model, we may not need to observe the whole state of the object e.g. a point moving in 3D, at the 3k’th tick we see x, 3k+1’th tick we see y, 3k+2’th tick we see z in this case, we can still make decent estimates of all three coordinates at each tick. –This property, which does not apply to every model, is called Observability

Examples Points moving with constant velocity Periodic motion etc. Points moving with constant acceleration

Moving with constant velocity We have –(the Greek letters denote noise terms) Stack (u, v) into a single state vector, x i –which is the form we had above

Moving with constant acceleration We have –(the Greek letters denote noise terms) Stack (u, v) into a single state vector –which is the form we had above

Constant velocity dynamic model state 1 st component of state position measurement and 1 st component of state position velocity time

Constant acceleration dynamic model position velocity position time

Kalman filter Key ideas: –Linear models interact uniquely well with Gaussian noise make the prior Gaussian, everything else Gaussian and the calculations are easy –Gaussians are really easy to represent once you know the mean and covariance, you’re done

Kalman filter in 1D Dynamic Model Notation Predicted mean Corrected mean Before/after i-th measurement

Prediction for 1D Kalman filter Because the new state is obtained by –multiplying old state by known constant –adding zero-mean noise Therefore, predicted mean for new state is –constant times mean for old state Predicted variance is –sum of constant^2 times old state variance and noise variance –old state is normal random variable, multiplying normal rv by constant implies mean is multiplied by a constant variance by square of constant, adding zero mean noise adds zero to the mean, adding rv’s adds variance

Correction for 1D Kalman filter Pattern match to identities given in book –basically, guess the integrals, get: Notice: –if measurement noise is small, we rely mainly on the measurement, –if it’s large, mainly on the prediction

Example: Kalman filtering

In higher dimensions, derivation follows the same lines, but isn’t as easy. Expressions here. Kalman filter: general case

the measurement standard deviation is small, so the state estimates are rather good

Smoothing Idea –We don’t have the best estimate of state - what about the future? –Run two filters, one moving forward, the other backward in time. –Now combine state estimates The crucial point here is that we can obtain a smoothed estimate by viewing the backward filter’s prediction as yet another measurement for the forward filter –so we’ve already done the equations

Forward filter

Backward filter

Forward-backward filter

Data association Also known as correspondence problem Given features detectors (measurements) are not perfect, how can one find correspondence between measurements and features in the model?

Data association Determine which measurements are informative (as not every measurement conveys the same amount of information) Nearest neighbors –choose the measurement with highest probability given predicted state –popular, but can lead to catastrophe Probabilistic Data Association (PDA) –combine measurements, weighting by probability given predicted state –gate using predicted state

Data association Constant velocity model measurements are plotted with standard deviation (dash line)

Data association with NN The blue track represents the actual measurements of state, and red tracks are noise The noise is all over the place, and the gate around each prediction typically contains only one measurement (the right one). This means that we can track rather well ( the blue track is very largely obscured by the overlaid measurements of state) Nearest neighbors (NN) pick best measurements (consistent with predictions) work well in this case Occasional mis-identifcaiton may not cause problems

Data association with NN If the dynamic model is not sufficiently constrained, choosing the measurement that best may lead to failure

Data association with NN But actually the tracker loses tracks These problems occur because error can accumulate it is now relatively easy to continue tracking the wrong point for a long time, and the longer we do this the less chance there is of recovering the right point

Probabilistic data association Instead of choosing the region most like the predicted measurement, we can exclude all regions that are too different and then use the others, weighting them according to their similarity to prediction superscipt indicates the region

Nonlinear dynamics As one repeats iterations of this function --- there isn’t any noise --- points move towards or away from points where sin(x)=0. This means that if one has a quite simple probability distribution on x_0, one can end up with a very complex distribution (many tiny peaks) quite quickly. Most easily demonstrated by running particles through this dynamics as in the next slide.

Nonlinear dynamics Top figure shows tracks of particle position against iteration for the dynamics of the previous slide, where the particle position was normal in the initial configuration (first graph at the bottom) As the succeeding graphs show, the number of particles near a point (= histogram = p(x_n)) very quickly gets complex, with many tiny peaks.

Propagation of general densities

Factored sampling Represent the state distribution non-parametrically –Prediction: Sample points from prior density for the state, p(x) –Correction: Weight the samples according to p(y|x) Representing p(y|x) in terms of {s t-1 (n), π t-1 (n) }

Particle filtering We want to use sampling to propagate densities over time (i.e., across frames in a video sequence) At each time step, represent posterior p(x t |y t ) with weighted sample set Previous time step’s sample set p(x t-1 |y t-1 ) is passed to next time step as the effective prior

Particle filtering M. Isard and A. Blake, “CONDENSATION -- conditional density propagation for visual tracking”, IJCV 29(1):5-28, 1998 Start with weighted samples from previous time step Sample and shift according to dynamics model Spread due to randomness; this is predicted density p(x t |y t-1 ) Weight the samples according to observation density Arrive at corrected density estimate p(x t |y t )

Paramterization of splines

Dynamic model p(x t |x t-1 ) Can learn from examples using ARMA or LDS Can be modeled with random walk Often factorize x t

Observation model p(y t |x t )

Observation model: 2D case

Particle filtering results

Issues Initialization Obtaining observation and dynamics model –Generative observation model: “render” the state on top of the image and compare –Discriminative observation model: classifier or detector score –Dynamics model: learn (very difficult) or specify using domain knowledge Prediction vs. correction –If the dynamics model is too strong, will end up ignoring the data –If the observation model is too strong, tracking is reduced to repeated detection Data association –What if we don’t know which measurements to associate with which tracks? Drift –Errors caused by dynamical model, observation model, and data association tend to accumulate over time Failure detection

Mean-Shift Object Tracking Search in the model’s neighborhood in next frame Start from the position of the model in the current frame Find best candidate by maximizing a similarity func. Repeat the same process in the next pair of frames Current frame …… ModelCandidate

Mean-Shift Object Tracking Target Representation Choose a reference target model Quantized Color Space Choose a feature space Represent the model by its PDF in the feature space Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer

Mean-Shift Object Tracking PDF Representation Similarity Function: Target Model (centered at 0) Target Candidate (centered at y)

Mean-Shift Object Tracking Smoothness of Similarity Function Similarity Function: Problem: Target is represented by color info only Spatial info is lost Solution: Mask the target with an isotropic kernel in the spatial domain f(y) becomes smooth in y f is not smooth Gradient-based optimizations are not robust Large similarity variations for adjacent locations

Mean-Shift Object Tracking Finding the PDF of the target model Target pixel locations A differentiable, isotropic, convex, monotonically decreasing kernel Peripheral pixels are affected by occlusion and background interference The color bin index (1..m) of pixel x Normalization factor Pixel weight Probability of feature u in model Probability of feature u in candidate Normalization factor Pixel weight 0 model y candidate

Mean-Shift Object Tracking Similarity Function Target model: Target candidate: Similarity function: 1 1 The Bhattacharyya Coefficient

Mean-Shift Object Tracking Target Localization Algorithm Start from the position of the model in the current frame Search in the model’s neighborhood in next frame Find best candidate by maximizing a similarity func.

Linear approx. (around y 0 ) Mean-Shift Object Tracking Approximating the Similarity Function Model location: Candidate location: Independent of y Density estimate! (as a function of y)

Mean-Shift Object Tracking Maximizing the Similarity Function The mode of = sought maximum Important Assumption: One mode in the searched neighborhood The target representation provides sufficient discrimination

Mean-Shift Object Tracking Applying Mean-Shift Original Mean- Shift: Find mode of using The mode of = sought maximum Extended Mean- Shift: Find mode of using

Mean-Shift Object Tracking About Kernels and Profiles A special class of radially symmetric kernels: The profile of kernel K Extended Mean- Shift: Find mode of using

Mean-Shift Object Tracking Choosing the Kernel Epanechnikov kernel: A special class of radially symmetric kernels: Extended Mean- Shift: Uniform kernel:

Mean-Shift Object Tracking Adaptive Scale Problem: The scale of the target changes in time The scale (h) of the kernel must be adapted Solution: Run localization 3 times with different h Choose h that achieves maximum similarity

Mean-Shift Object Tracking Results Feature space: 16  16  16 quantized RGB Target: manually selected on 1 st frame Average mean-shift iterations: 4

Mean-Shift Object Tracking Results Partial occlusionDistractionMotion blur