Evaluating Reliability of Motion Features in Surveillance Videos Longin Jan Latecki and Roland Miezianko, Temple University Dragoljub Pokrajac, Delaware State University November 2004

Motion Detection Goals of motion detection Identify moving objects Detection of unusual activity patterns Computing trajectories of moving objects Benefits of reliability assessment Reduction of false detections (e.g., false alarms) 5/3/2019

Applications of Motion Detection Many intelligent video analysis systems are based on motion detection. Such systems can be used in Homeland security Real time crime detection Traffic monitoring … 5/3/2019

Motion Measure Computation We use spatial-temporal blocks to represent videos Each block consists of NBLOCK x NBLOCK pixels from 3 consecutive frames Those pixel values are reduced to K principal components using PCA (Kahrunen-Loeve trans.) In our application, NBLOCK=8, K=10 Thus, we project 192 gray level values to a texture vector with 10 PCA components 5/3/2019

Frame t-1 5/3/2019

Frame t 5/3/2019

Frame t+1 5/3/2019

48-component block vector (4*4*3) 4*4*3 spatial-temporal block Location I=7, J=7, time t 48-component block vector (4*4*3) -0.5221 -0.0624 -0.1734 3 principal components 5/3/2019

Why texture of spatiotemporal blocks can work better? More robust in comparison to pixel-based approach Integrates texture- and movement (temporal) information Faster 5/3/2019

499 624 863 1477 5/3/2019

Trajectory of block (24,8) (Campus 1 video) Moving blocks corresponds to regions of high local variance Space of spatiotemporal block vectors 5/3/2019

Trajectory of a pixel from block (24,8) Space of RGB pixel values 5/3/2019

Detection of Moving Objects Based on Local Variation For each location (x,y) of the frames Consider vectors of derived attribute values corresponding to a symmetric window of size 2W+1 around each time instant t Derived attribute vectors: RGB; first 10 PCA projections of spatial-temporal blocks, etc. Compute the covariance matrix for the vectors motion measure is defined as the largest eigenvalue of the covariance matrix 5/3/2019

Feature Vectors in Space Feature vectors 4.2000 3.5000 2.6000 4.2000 3.5000 2.6000 4.1000 3.7000 2.8000 3.9000 3.9000 2.9000 4.0000 4.0000 3.0000 4.1000 3.9000 2.8000 4.2000 3.8000 2.7000 4.3000 3.7000 2.6500 Covariance matrix Current time 0.0089 -0.0120 -0.0096 -0.0120 0.0299 0.0201 -0.0096 0.0201 0.0157 Motion Measure Eigenvalues 0.0499 0.0035 0.0011 0.0499 5/3/2019

Feature Vectors in Space Feature vectors 4.3000 3.7000 2.6500 4.3000 3.7000 2.6500 4.4191 3.5944 2.4329 4.1798 3.8415 2.6441 4.2980 3.6195 2.5489 4.2843 3.7529 2.7114 4.1396 3.7219 2.7008 4.3257 3.6078 2.8192 Covariance matrix 0.0087 -0.0063 -0.0051 -0.0063 0.0081 0.0031 -0.0051 0.0031 0.0154 Current time Motion Measure Eigenvalues 0.0209 0.0093 0.0020 0.0209 5/3/2019

In our system we divide video plane in disjoint blocks (8x8 blocks), and compute motion measure for each block. mm(x,y,t) for a given block location (x,y) is a function of t 5/3/2019

Graph of motion measure mm(24,8,:) for Campus 1 video 5/3/2019

Motion amount The feature called motion amount is defined as The system decision on alarm situation is based on ma. 5/3/2019

5/3/2019

ma(t) as function of frame number t for Temple 1 video 5/3/2019

5/3/2019