Human Detection in Surveillance Applications Ashish Desai EE392J Final Project

Problem and Motivation Had many petty vandals cause damage to cars in my parking garage Landlord added cameras, but did not deter vandals What if we could count and have photos of all the people in the garage at any given time?

Breakdown into Class Concepts Simply, this is a foreground / background segmentation problem Perhaps use color based segmentation Perhaps use motion based segmentation

Methodology Captured 25 second video at 10 frames per second of 1, 2, or 3 people walking around in my apartment.

Methodology Used Staufer and Grimson color based segmentation Used 5 gaussians in RGB space Used maximum of 1.5 std. dev for classification Used alpha = 0.7 Combined with connectivity requirements (each pixel must touch 4 others) Combined with size requirements (each group of pixels must be larger than 100)

Methodology Very simplified motion based segmentation Used block matching between current and next frame 16x16 block size, +16/-15 full search, SAD criterion Eliminate blocks with fewer than 8 pixel movement Combine with size requirements (must be greater than 4 blocks) Note: did not use K-means clustering (and should have used log search) to improve real-time capabilities

Results of Individual Segmentations Color based segmentation Worked pretty well (I spent a lot of time tweaking this) Had problems with large areas of occlusion, shadows Motion based segmentation Purposely set to allow “false positives” Simple methodology could not handle global motion from person bumping the camera Blinds (possible interlace artifact) caused problems, aspect ratio (block-based)

Combinational Methodology Take both of the initial methodologies and create a confidence weighting (those pixels in the center of a group have higher weighting) Combine the two weights with more preference to color weights (color had better initial performance) Apply threshold, connectivity and size constraints.

Overall Block Diagram Input Sequence (RGB) RGB -> YUV Motion Estimation Color Gaussian Segmentation Connectivity/ Size Requirements Connectivity/ Size Requirements Weighting Weighting Threshold Output

Overall Results Performed pretty well especially with lateral movement and objects further from camera Still had problems with large occlusions (from color) and aspect ratio changes (from motion) but better than respective individuals Eliminated major shadow and global motion issues by combining the two

Results - Videos

Future Enhancements Improvements (if time allows) Use affine parameters for motion based Use log search (if still use block) for more real time Use K means clustering to segment, rather than simple threshold Use temporal knowledge (i.e. use object identification from previous frame or frames)