1. Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE

2. Abstract In this paper, by proposing an adaptive Rao- Blackwellized Particle Filter (RBPF) for tracking in surveillance, we show how to exploit the analytical relationship among state variables to improve the efficiency and accuracy of a regular particle filter (PF). In this paper, by proposing an adaptive Rao- Blackwellized Particle Filter (RBPF) for tracking in surveillance, we show how to exploit the analytical relationship among state variables to improve the efficiency and accuracy of a regular particle filter (PF).

3. Introduction Visual tracking is an important step in many practical applications. Visual tracking is an important step in many practical applications. Generally, suppose we have an estimator depending upon 2 variables R and L, the RB theorem reveals its variance satisfies: Generally, suppose we have an estimator depending upon 2 variables R and L, the RB theorem reveals its variance satisfies: Non-negative

4. For the visual tracking problem, let denote the state to be estimated and the observation, with subscript t the time index. For the visual tracking problem, let denote the state to be estimated and the observation, with subscript t the time index. The key idea of RBPF is to partition the original state-space into two parts and. The key idea of RBPF is to partition the original state-space into two parts and. The justification for this decomposition follows from the factorization of the posterior probability The justification for this decomposition follows from the factorization of the posterior probability

5. RBPF for tracking in surveillance a) Partition the state space a) Partition the state space

6. In this paper,using 8-D ellipse model to describe the target In this paper,using 8-D ellipse model to describe the target

7. The scale change is related to its position alone y axis, so the scale change can be estimated conditional on the location components. The 8-D state space can separate into 2 groups The scale change is related to its position alone y axis, so the scale change can be estimated conditional on the location components. The 8-D state space can separate into 2 groups Root variables containing the motion information. Leaf variables containing the scale parameters.

8. b) Overview of the method b) Overview of the method In this work, root variables are propagated by a first order system motion model defined by In this work, root variables are propagated by a first order system motion model defined by Conditional on the root variables, the leaf variables forms a linear-Gaussian substructure specified by Conditional on the root variables, the leaf variables forms a linear-Gaussian substructure specified by transition matrix random noise A function encoding the conditional relation of L Gaussian random noise

9. Since both color histogram and gradient cues do not follow a linear-Gaussian relationship with state variable, the observation model is given in a general form: Since both color histogram and gradient cues do not follow a linear-Gaussian relationship with state variable, the observation model is given in a general form: The observations form a linear relationship with state L The observations form a linear relationship with state L Image observation Nonlinear function Random noise Gaussian random noise

10. Relationship between variables

11. The RBPF algorithm

12. Just like regular PF, RBPF represents the posterior density by a set of weighted particles: Just like regular PF, RBPF represents the posterior density by a set of weighted particles: Each particle is represented by a triplet. Each particle is represented by a triplet. The proposed RBPF algorithm will sample the motion using PF, while apply Kalman filter to estimate the scale parameters andconditional on the motion state. The proposed RBPF algorithm will sample the motion using PF, while apply Kalman filter to estimate the scale parameters andconditional on the motion state.

13. (1)Propagate samples a) Sample the object motion according to a) Sample the object motion according to After this step, we have minus sign is denotes the corresponding variable is a priori estimate b) Kalman prediction for leaf states according to b) Kalman prediction for leaf states according to

14. According to the Kalman filter model (4)and(6), we project forward the state and error covariance using: According to the Kalman filter model (4)and(6), we project forward the state and error covariance using: After this step, we have After this step, we have Prediction for the mean of the leaf variables Covariance for leaves Observation prediction

15. a) Compute the color histogram for each sample ellipse Γ characterized by ellipse center and scale a) Compute the color histogram for each sample ellipse Γ characterized by ellipse center and scale Pixels that are closer to the region center are given higher weights specified by Pixels that are closer to the region center are given higher weights specified by (2)Evaluate weight for each particle Kronecker delta function

16. b) Compute the gradient for each sample ellipse Γ characterized by ellipse center and scale the gradient of a sample ellipse is computed as an average over gradients of all the pixels on the boundary where the gradient at pixel is set to the maximum gradient by a local search along the normal line of the ellipse at location b) Compute the gradient for each sample ellipse Γ characterized by ellipse center and scale the gradient of a sample ellipse is computed as an average over gradients of all the pixels on the boundary where the gradient at pixel is set to the maximum gradient by a local search along the normal line of the ellipse at location

17. A simple operator is used to compute the gradient in x and y axis for pixel finally, the gradient at point is computed as A simple operator is used to compute the gradient in x and y axis for pixel finally, the gradient at point is computed as

18. c) Compute the weight c) Compute the weight one is based on color histogram similarity between the hypothetical region and the target model p stands for the color histogram of a sample hypothesis in the newly observed image, and q represents the color histogram of target model. one is based on color histogram similarity between the hypothetical region and the target model p stands for the color histogram of a sample hypothesis in the newly observed image, and q represents the color histogram of target model.

19. Another is based on gradient Another is based on gradient Notice that all the sample is divided by the maximum gradient to normalize into range[0,1], the final weight for each sample is given by Notice that all the sample is divided by the maximum gradient to normalize into range[0,1], the final weight for each sample is given by

20. (3)Select samples Resampling with replacement the latest measurements will be used to modify the prediction PDF of not only the root variables but also the leaf variables. Resampling with replacement the latest measurements will be used to modify the prediction PDF of not only the root variables but also the leaf variables. After this step, After this step,

21. (4)Kalman update for leaf variables Kalman update is accomplished by Kalman update is accomplished by After this step, we have After this step, we have

22. (5)Compute the mean state at time t Since resampling has been done, the mean state can be simply computed as the average of the state particles Since resampling has been done, the mean state can be simply computed as the average of the state particles

23. (6)Compute the new noise variance We found that when velocity is small and constant, we only need a small noise variance to reach the smallest MSE, if velocity changes dramatically, we need a much larger noise variance to reach the lowest MSE. We found that when velocity is small and constant, we only need a small noise variance to reach the smallest MSE, if velocity changes dramatically, we need a much larger noise variance to reach the lowest MSE. The noise variance is computed by The noise variance is computed by

24. Evaluation of the RBPF algorithm Evaluate the performance between RBPF and PF. Evaluate the performance between RBPF and PF.

28. Real data experiment

31. Discussion Failure cases: when camera is not mounted higher than the target object … Failure cases: when camera is not mounted higher than the target object … Computation cost: the same level of estimation accuracy, RBPF needs far fewer particles than PF dose; hence, it is more efficient than PF. Computation cost: the same level of estimation accuracy, RBPF needs far fewer particles than PF dose; hence, it is more efficient than PF.

32. Conclusion Comparative studies using both simulated and real data have demonstrated the improved performance of the proposed RBPF over regular PF. Comparative studies using both simulated and real data have demonstrated the improved performance of the proposed RBPF over regular PF. Future working: to find a proper dependency model from a large number of state variables. Future working: to find a proper dependency model from a large number of state variables.