International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July Video Compression and Retrieval of Moving Object Location Applied to Surveillance William R. Schwartz, Hélio Pedrini, Larry S. Davis

International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July Introduction Proposed Method Experimental Results Conclusions and Remarks Organization

International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July Motivation:  Surveillance cameras capture hours of data that need to be store and analyzed. Introduction  Analysis is based mostly on moving objects (interaction between people in the scene).  Reduce redundancy + store objects.

Introduction Proposed method Experimental results Conclusions 4 Goals:  Develop a video compression technique that takes advantage of static cameras (surveillance).  Provide information regarding the location of moving objects (higher-level computer vision tasks). Key ideas:  Estimate eigenspaces for non-overlapping blocks.  Use a second encoding scheme to encode regions not well modeled by the eigenspaces. Introduction

Introduction Proposed method Experimental results Conclusions 5 Use of eigenspaces allow high compression ratio.  Store the projection vectors (obtained using PCA).  For a new frame, save only projection coefficients (in case of no changes or linear changes in the block). Advantages of Eigenspaces Block size Compression ratio (assuming float coefficients) 1 coefficient2 coefficients3 coefficients4 coefficients 8x x x x

Introduction Proposed method Experimental results Conclusions 6 Eigenspaces do not model non-linear changes (i.e. moving person). Use a second encoding method for these cases. Two-Stage Scheme

Introduction Proposed method Experimental results Conclusions 7 Segment the image area into non-overlapping blocks. Estimate eigenspaces for each block. For new frames, project blocks onto the eigenspaces and compute the reprojection error. If the reprojection error is acceptable, save the scores, otherwise set the image block to be encoded using MPEG-4. Proposed Method

Introduction Proposed method Experimental results Conclusions 8 For each block, sample frames free of non-linear transformations (≈ 200 frames). Estimate projection vectors P = {p 1,…, p k }. Estimate reconstruction error distribution δ p for each pixel in the block (used to locate moving objects). Learning the Eigenspaces

Introduction Proposed method Experimental results Conclusions 9 Compression Algorithm

Introduction Proposed method Experimental results Conclusions 10 The reprojection error is high when there are non- linear transformations within a block (moving objects / non-linear local illumination changes). Given that a block was compressed using MPEG- 4, check the pixels that do not satisfy the error distribution δ p to decide if there is a moving object on that block. Location of Moving Objects

Introduction Proposed method Experimental results Conclusions 11 Our method was tested on four video sequences. Compared to MPEG-4 and H.263 using MEncoder. Experiments:  Compression with a constant PSNR.  Moving object location accuracy. Experimental Results

Introduction Proposed method Experimental results Conclusions 12 Frames were converted to YCbCr. Using blocks of 16 × 16 pixels. Initial 200 frames were used to learn the eigenspaces. Number of PCA coefficients kept is estimated for each block (bounded by a maximum). Video Compression

Introduction Proposed method Experimental results Conclusions 13 Video Compression video sequence Size & # frames PSNR (dB) Compression Ratio MPEG-4H.263proposed camera 1768x288x camera 2720x528x station720x576x robbery720x480x camera 2 camera 1 robbery station

Introduction Proposed method Experimental results Conclusions 14 Easy to retrieve object location due to the encoding scheme used. Evaluation compares the location obtained by the method to the ground truth location. At a false positive rate of obtained a false negative rate of Moving Object Location

Introduction Proposed method Experimental results Conclusions 15 High compression rates:  Robust to linear transformations in illumination.  No need for saving key-frames. Fast to encode (≈ 5 fps in MATLAB). Useful for higher level computer vision tasks due to the storage of moving object locations. The use of multiple size blocks might increase the compression ratio. Conclusions and Remarks

Introduction Proposed method Experimental results Conclusions 16 Removal of Foreground Blocks Estimate initial eigenspace. Compute reprojection error for each frame. Compute the median and standard deviation. Remove frames with high error (∆ > m + cσ).