Supervised by Dr. Hau-San WONG Prepared by Kam-fung YU ( )

 Video Surveillance System are widely used for monitoring  Performance is good as the number of object for detection is small (Spatial variation is small) and ;  The change over time is small (Temporal variation is small) BUT  A challenge for crowd flow  The number of objects is in the order of 10 2 ~10 3  The change of the scene is very fast

 Based on Tracking of Individuals  Shape and Color Model of Individuals  Trajectories of Points  Boundary Contour  xt Slices of Spatio-temporal Video Volume  People Counting in the Crowd

 Models human shape by using 3D model  Data-driven Markov chain Monte Carlo (DDMCMC)  Iterate an optimized solution T. Zhao et. al., “Bayesian Human Segmentation in Crowded Situations”, IEEE CVPR03, D human model

 Similar method with Shape & Color Model  Use some simple feature, such as corner of an object, to extract points probabilistically  Clustering the points into independently moving entities, cluster Shape and Color Model Trajectories of Points G. Brostow et. al., “Unsupervised Bayesian Detetcion of Independent Motion in Crowds”, IEEE CVPR, 2006.

 Use of low-interest points to detect the object clustering  Select by the high temporal and spatial discontinuity  Outline the object by joining edges P. Tu et. al., “Crowd Segmentation through Emergent Labeling”, In ECCV Workshop SMVP, 2004 Clustered objectSample Scene

 Scan interesting lines over a certain frames, xt-slice  Use the Hough transform to detect movement in the xt slices P. Reisman, “Crowd Detection in Video Sequences”, IEEE Intelligent Vehicles Symposium, Sample Scene with 5 lines 5 corresponding xt slices Hough transform

 Clustering of some feature points by their motion  Estimate the number of people by the number of cluster A result of clustering on two video scene V. Rabaud et. al., “Counting Crowded Moving Objects”, IEEE CVPR, 2006

 Involes Iteration  Convergence Decease as Number of Objects Increase  Large Computational Time  High of Computational Power  Difficulty to implement on Real Time Monitoring System

 Proposed by Saad Ali & Mubarak Shah in 2007  Individual Flow → Global Optical Flow  Tracking Individuals → Measuring Global Quantities  Using Fluid Dynamics to treat the problem  Global Quantities:Finite Time Lyapunov Exponents Field (FTLE), Lagrangian Coherent Structures(LCS)  Expect a Higher & Faster Algorithm in Performance

 16x16 size block  Displacement vector x  p frames for 1 mean field  q mean field for 1 block mean field S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

 Launch a set of particles over the optical flow field  Solve a flow map for a time period T = p  q frames  Interpolation a cubic velocity equation by 4th order Runge-Kutta- Fehlberg algorithm (RK4)   x,  y are used to record the x and y coordinate at each initial position launched after time Flow map of x-particle Flow map of y-particle S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

 Compute the four spatial derivates  Plug into the Cauchy-Green deformation tensor  The largest finite time Lyapunov exponent with the maxmum eigenvalue max of the tensor and the period T=p  q frames S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007 FTLE Field Plot

S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007 FTLE Field Plot

 This process involves two stages  Cut spatially into different region by the ridges in FTLE  Use the Lyapunov divergence to decided two segment merge or not 1 st stage 2 nd stage S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

 Flow instability is defined as the change in the number of flow segments with respect to time S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007 New segmentCurrent segment

 Capable for monitoring thousands of objects simultaneously  Get rid of number of people constrain  Capable for monitoring flow in any orientation  Obtain same result under any rotation  Capable for new segment detection over time  Locate the increase or the decrease of segments over time

 Potential for flow control or city design  Making immediate decision for crowd flow  Facilitate on the planning of city streets, traffic flow, overhead, bridges and passageways  Potential for flow pattern recognition  Extraction of various flow pattern  Flow pattern solution space construction for a given static scenery  Flow pattern bases finding

 Limitation on crowd density  Degraded as crowd density is low  Worse at only a number of objects  Limitation on a large number of many-fold dynamics flow  Too many segments (too noisy) on the scene  Hard to merge segment  Limitation on a rapid unstable flow  Hard to retrieval information from rapid changing flow  Too slow to capture the information

 Find out the critical crowd density for an acceptable performance  Finding out a method that can undergo segmentation under a noisy domain  Designing a rapid flow capturing algorithm  Finding out the possible flow patterns on given static scenery  Find out the flow patterns solution space and bases

 In this guided study, we studied about various kinds of methods and the Lagrangian Dynamics in solving the crowd segmentation problem.  We also realized the capabilities, potentials and limitations.  We finally suggested some possible direction for future studies.

1. Z.N. Li, M.S. Drew, “Fundamentals of Multimedia”, NJ: Pearson Education Hall, P.E. Mattison, “Practical Digital Video with programming examples in C”, NY: John Wiley & Sons Inc, L. Perko, “Differential Equations and Dynamical Systems 3 rd Ed.”, NY: Springer, Intel Corporation, “Open Source Computer Vision Library, Reference Manual”, USA: Intel Corporation, S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, S. C. Shadden, “Lagrangian Coherent Structures – Analysis of time-dependent dynamical systems using finite-time Lyapunov exponents”, Available Online : Last update: 15 th April, 2005http:// 7. P. Reisman, “Crowd Detection in Video Sequences”, IEEE Intelligent Vehicles Symposium, T. Zhao et. al., “Bayesian Human Segmentation in Crowded Situations”, IEEE CVPR03, P. Tu et. al., “Crowd Segmentation through Emergent Labeling”, In ECCV Workshop SMVP, G. Brostow et. al., “Unsupervised Bayesian Detetcion of Independent Motion in Crowds”, IEEE CVPR, D. Yang et. al., “Counting People in Crowds with a Real-Time Network of Simple Image Sensors”, ICCV, V. Rabaud et. al., “Counting Crowded Moving Objects”, IEEE CVPR, E. Rosten and T. Drummond, “Machine learning for high-speed corner detection”, Europe Conference on Computer Vision, May C. Tomasi and T. Kanade, “Detection and tracking of point features”, Technical Report CMU-CS , Carnegie Mellon University, April S. Ali, “Crowd Flow Segmentation & Stability Analysis”, Available Online : Last visited: 30 th Nov,

Department of Computer Science City University of Hong Kong