1. Pedestrian Detection in Crowded Scenes Dhruv Batra ECE CMU

2. Pedestrian Detection in Crowded Scenes 1.Pedestrian Detection in Crowded Scenes. Bastian Leibe, Edgar Seemann, and Bernt Schiele. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, June 2005. 2.An Evaluation of Local Shape-Based Features for Pedestrian Detection. Edgar Seemann, Bastian Leibe, Krystian Mikolajczyk, and Bernt Schiele. In British Machine Vision Conference (BMVC'05) Oxford, UK, September 2005. 3.Combined Object Categorization and Segmentation with an Implicit Shape Model. Bastian Leibe, Ales Leonardis, and Bernt Schiele. In ECCV'04 Workshop on Statistical Learning in Computer Vision, Prague, May 2004.

3. Theme of the Paper  Probabilistic top-down/bottom-up formulation of segmentation/recognition  Basic Premise: “[Such a] problem is too difficult for any type of feature or model alone”

4. Theme of the Paper  Open Question: How would you do pedestrian detection/segmentation?  Solution: integrate as many cues as possible from many sources Original imageSupport of Segmentation from local featuresSegmentation from local featuresSupport of segmentation from global features (Chamfer Matching)Segmentation from global features (Chamfer Matching)

5. Theme of the Paper  Goal: Localize AND count pedestrians in a given image  Datasets Training Set: 35 people walking parallel to the image planeTesting Set (Much harder!): 209 images of 595 annotated pedestrians

6. Theme of the Paper

7. Evaluation Criteria  Criteria 1: Relative Distance Fixed aspect ratio- 11:15 Threshold d < 0.5

8. Evaluation Criteria  Criteria 2 & 3: Cover and Overlap Threshold cover >50% overlap >50%

9. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  Code book Approach (with spatial information)

10. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training: Lowe’s DoG Detector 3x 3 patches Resize to 25 x 25

11. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  Agglomerative Clustering

12. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  Agglomerative Clustering

13. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  Agglomerative Clustering

14. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  Agglomerative Clustering Codebook entries store figure-ground masks for these entries

15. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Training:  But wait! We just lost spatial information … Run again Lowe’s DoG Detector Resize to 25 x 25 3x 3 patches Find codebook patches Learn Spatial Distribution

16. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Initial Hypothesis: Overall

17. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Initial Hypothesis (Probabilistic Hough Voting Procedure) measuring similarity between patch and codebook entrylearnt from spatial distributions of codebook entries Search for maximum in probability spaceUsing a fixed size search window

18. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Initial Hypothesis: found as maxima in 3D voting space maxima computed using Mean Shift Mode Estimation over this balloon density estimator Uniform Cubicle Kernel

19. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Initial Hypothesis: Overall

20. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Initial Hypothesis: Overall

21. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Probabilistic top down segmentation Intermediate Goal: Find this start here Assumption: Uniform Priors Estimate from training data From similarity measure

22. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Probabilistic top down segmentation Marginalized over all patches in image Substitute this here to get this

23. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Probabilistic top down segmentation

24. Initial Recognition Approach  First Step: Generate hypotheses from local features (Intrinsic Shape Models)  Testing:  Probabilistic top down segmentation

25. Initial Recognition Approach  Second Step: Handling overlapping detections

26. Initial Recognition Approach  Second Step: Segmentation based Verification (Minimum Description Length)  Saving that can be achieved by explaining part of image by a particular hypothesis Number of pixels N explained by h Model complexityCost of describing the error made by hypothesis h Sum over all pixels hypothesized as figure Probability of being a background

27. Initial Recognition Approach  Second Step: Segmentation based Verification (Minimum Description Length)  With this framework we can resolve conflicts between overlapping hypothesis Relative importance assigned to support of hypothesis Bias term

28. Initial Recognition Approach  Second Step: Segmentation based Verification (Minimum Description Length)  Voila! It works

29. Initial Recognition Approach  Second Step: Segmentation based Verification (Minimum Description Length)  Caveat: it leads to another set of problems ISM doesn’t know a person doesn’t have three legs! Global Cues are needed Or four legs and three arms

30. Assimilation of Global Cues  Distance Transform, Chamfer Matching get Feature Image by an edge detectorget DT image by computing distance to nearest feature pointChamfer Distance between template and DT image

31. Assimilation of Global Cues (Attempt 1)  Distance Transform, Chamfer Matching Chamfer distance based matching Use scale estimate to cut out surrounding region Apply Canny detector and compute DT Yellow is highest Chamfer score Initial hypothesis generated by local features

32. Assimilation of Global Cues (Attempt 2)  Maximize Chamfer Score AND overlap with overlap with hypothesized segmentation instead of pure Chamfer Score Overlap expressed as Bhattacharya coeff. Joint score is linear combination of the two

33. Assimilation of Global Cues (Attempt 3)  Apply hypothesis saving MDL method again  Boolean quadratic formulation

34. Results