1. Human Detection Mikel Rodriguez

2. Organization 1. Moving Target Indicator (MTI) Background models Background models Moving region detection Moving region detection Target chip generation Target chip generation Results Results Input Frame Object Detection Target Chips Wavelet Features SVM Classifier MTIClassification 2. Target Classification (Human Detection) Target features Target features Support vector machines Support vector machines Results Results

3. Moving Target Indicator Moving target indicator (MTI) identifies moving objects which can be potential targets

4. MTI Motivation Becoming increasingly important in military and civilian applications Becoming increasingly important in military and civilian applications To minimize human involvement To minimize human involvement Expensive Expensive Short attention spans Short attention spans Computerized monitoring system Computerized monitoring system Real-time capability Real-time capability 24/7 24/7

5. MTI Challenges Different sensor modalities Different sensor modalities LADAR, IR, EO LADAR, IR, EO Targets with different dynamics Targets with different dynamics Small targets Small targets Weather conditions Weather conditions Illumination changes, shadows… Illumination changes, shadows…

6. Input Video MTI Background Modeling Intensity models Gradient models Moving Target Detection Background Subtraction dynamic update Targets ChipsPosition

7. Hierarchical Approach to Background Modeling Pixel level Pixel level Region level Region level Frame level Frame level

8. Pixel Level Background Features Intensity, heat index Intensity, heat index Gradient Gradient 2D: magnitude, orientation 2D: magnitude, orientation IREO Magnitude Orientation

9. Pixel Level Background Features Intensity, heat index Intensity, heat index Per-pixel mixture of Gaussians. Per-pixel mixture of Gaussians. Gradient based subtraction Gradient based subtraction Gradient feature vector  =[  m,  d d ] Gradient feature vector  =[  m,  d d ]

10. Pixel Level Moving Region Detection Mark pixels that are different from the background intensity model Mark pixels that are different from the background intensity model Mark pixels that are different from the background gradient model Mark pixels that are different from the background gradient model

11. Color basedImage Gradient Region Level Fusion of Intensity & Gradient Results For each color based region, presence of“edge difference” pixels at the boundaries is checked. Regions with small number of edge difference pixel are removed, color model is updated. Final

12. Frame Level Model Update Performs a high level analysis of the scene components Performs a high level analysis of the scene components If more > 50% of the intensity based background subtracted image becomes foreground. Frame level processing issues an alert Intensity based subtraction results are ignored

13. Structure of the MTI Class MTI Background ConnectedComponents() BoundaryEdges() SetNumGaussians() SetAlpha() SetRhoMean() SetWeightThresh() SetActiveRegion() GetNumGaussians() GetAlpha() GetRhoMean() GetWeightThresh() GetActiveRegion() Object SetBoundingBox() SetRhoLocation() SetCentroid() GetBoundingBox() GetRhoLocation() GetCentroid() IsFalseDetection() Chips Centroid() ObjectArea() Height() Width()

14. Results

15. Target Classification Classification of objects into two classes: humans and others, from target chips generated by MTI

16. Challenges Small size Small size Obscured targets Obscured targets Background clutter Background clutter Weather conditions Weather conditions

17. Classifier Flow Feature Extraction Wavelet Testing MTI Chips NegativePositive Training SVM Support Vectors Decision

18. Wavelet Based Target Features Blurred Vertical Horizontal Diagonal

19. Feature Extraction Apply 2D Wavelet Transform Apply 2D Wavelet Transform Daubechies wavelets Daubechies wavelets Apply Inverse 2D Wavelet Transform to each of the coefficient matrices individually Apply Inverse 2D Wavelet Transform to each of the coefficient matrices individually Rescale and vectorize output matrices Rescale and vectorize output matrices

20. Why Wavelets? Separability among samples Separability among samples can be separated from Humans can be separated from cars and background Correlation using gray levelsCorrelation using gradient mag.

21. Why Wavelets? Person 11 - DB3 Wavelet Correlation

22. Support Vector Machines (SVM) Classification of data into two classes Classification of data into two classes N dimensional data. N dimensional data. Linearly separable Linearly separable If not transform data into a higher dimensional space If not transform data into a higher dimensional space Find separating N dimensional hyperplane Find separating N dimensional hyperplane

23. SVM Linear Classifier hyperplane equation N dimensional data point x i Sample distance to hyperplane

24. SVM Best Hyperplane? Infinite number of hyperplanes. Infinite number of hyperplanes. Minimize r i over sample set x i Minimize r i over sample set x i Maximize margin  around hyperplane Maximize margin  around hyperplane Samples inside the margin are the support vectors Samples inside the margin are the support vectors

25. SVM Training Set Let  =1,A training set is a set of tuples: {(x 1,y 1 ),(x 2,y 2 ),…(x m,y m )}. Let  =1,A training set is a set of tuples: {(x 1,y 1 ),(x 2,y 2 ),…(x m,y m )}. For support vectors inequality becomes equality For support vectors inequality becomes equality Unknowns are w and b Unknowns are w and b

26. SVM Linear Separability Linear programming, Linear programming, Separator line in 2D w 1 x i,1 +w 2 x i,2 +b=0. Separator line in 2D w 1 x i,1 +w 2 x i,2 +b=0. Find w 1, w 2, b such that  is maximized Find w 1, w 2, b such that  is maximized Find w 1, w 2, b such that  (w)=w T w is minimized Find w 1, w 2, b such that  (w)=w T w is minimized

27. SVM Solution Has the following form: Has the following form: Non-zero  i indicates x i is support vector Non-zero  i indicates x i is support vector Classifying function is: Classifying function is:

28. Classification Class Human Classification TrainingFunction ReadPositiveImages() ReadNegativeImages() AssemblePositive() AssembleNegative() AssembleMatrices() TestingFunction LoadSVM() ReadImages() ExtractFeatures ConvertToGray() ApplyWaveletFilter() ApplyInverseTrans() ResizeInverse() VectorizeInverse() Concatenate() TrainSVM LIBSVM TestSVM LIBSVM

29. Classification Baseline Analysis Run time for 3.0GHz dualcore, 2GB RAM Run time for 3.0GHz dualcore, 2GB RAM Training: 276 training samples 8.015 seconds Training: 276 training samples 8.015 seconds Testing: 24.087 chips (25 by 25) per second Testing: 24.087 chips (25 by 25) per second Classifier size Classifier size Depends on diversity of images Depends on diversity of images For 276 training samples of 25x25, classifier size is 1.101 MB For 276 training samples of 25x25, classifier size is 1.101 MB

30. Classification Baseline Analysis Memory requirements Memory requirements Requires entire set of support vectors Requires entire set of support vectors Current classifier Current classifier

31. Experiments Vivid DatasetUCF Dataset Vivid DatasetUCF Dataset

32. Results Training set Training set 300 target chips 300 target chips Testing Testing 3872 human chips 3872 human chips 5605 vehicle and background chips 5605 vehicle and background chips Performance Performance 2.4% false positive (others classified as pedestrians) 2.4% false positive (others classified as pedestrians) 3.2% false negative (pedestrian classified as others) 3.2% false negative (pedestrian classified as others)

33. Future directions MTI MTI Detection by parts Detection by parts Motion clustering Motion clustering Classification Classification Various kernels for SVM Various kernels for SVM Better target features Better target features Motion, steerable pyramids, shape features (height, width) Motion, steerable pyramids, shape features (height, width) Local wavelet coefficients Local wavelet coefficients Adaboost Adaboost