1. Student: Yao-Sheng Wang Advisor: Prof. Sheng-Jyh Wang ARTICULATED HUMAN DETECTION 1 Department of Electronics Engineering National Chiao Tung University Hsinchu, Taiwan 1

2.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 2

3.  Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 3

4.  Why we care about human detection?  We are human beings!  Wide range of applications:  Automotive safety  Surveillance system  Indoor care  Crime alert  Human-Computer Interface … etc. 4 MOTIVATION

5.  Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 5

6.  What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 6 CHALLENGE

7.  What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 7 CHALLENGE

8.  What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 8 CHALLENGE

9.  What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 9 CHALLENGE

10.  Progress on “Machine Learning” technology  Handle more general and complicate cases.  Definition:  “Articulated Human Detection”. 10 CHALLENGE

11.  Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 11

12.  Deformable Part Model  Root filter (mask).  Part filter (mask).  Penalty function. 12 REPRESENTATIVE WORKS (I) [P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable part model. In CVPR, 2008.]

13.  Pose-let: 13 REPRESENTATIVE WORKS (II) [Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. In ICCV, 2009.]..

14.  Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 14

15.  Problems:  System complexity increased with the complexity of human poses.  More detectors needed.  Exhaustive search.  Sliding window method + Image pyramid.  Both problems leads to unacceptable speed for applications in real life. 15 POTENTIAL PROBLEMS

16.  Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 16

17.  Target in the thesis:  Propose a detection scheme with acceptable detection speed in dealing with highly intra- class variation from the change of pose and viewpoint. 17 TARGET

18.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 18

19.  Better features:  Cheap to compute and capture crucial information at the same time. Ex: HOG.  Better classifiers:  Linear classifiers.  Ex: Adaboost, Linear-SVM and Random-forests.  Better prior knowledge:  Ex: Information about ground plane. 19 RELATED WORKS

20.  Cascades:  Cascade the part filters to reduce the searching regions. 20 RELATED WORKS [P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with Deformable Part Models. In CVPR, 2010.]

21.  Discard non-promising hypotheses.  Class-dependent:  Branch and bound. (CVPR, 2008)  Class-independent:  What is an object? (CVPR, 2010)  Closure boundary, different appearance or salience.  Segmentation as selective search. (ICCV, 2011) 21 RELATED WORKS

22.  Feature response approximation:  Feature approximation in testing step.  Feature approximation in training step. 22 RELATED WORKS [R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at 100 frames per second. In CVPR, 2012.] [P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]

23.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 23

24.  Recall the memory of the first problem:  System complexity increased with the complexity of human poses (include variation of viewpoints).  How can we break the relation between the complexity of system and the one of human poses?  Choose stable features or body parts for detection. 24 IDEA

25.  Better prior knowledge: 25 IDEA

26.  Recall the memory of the second problem:  Exhaustive search.  “Sliding Window” + “Image Pyramid”.  How can we reduce the searching region?  Detect the common feature among these parts.  Use the cumulative characteristic of the feature to handle the variation of scale. 26 IDEA

27.  Common feature  Body parts consist of combination of two edge segments.  Cumulative characteristic  Edge detector with fixed size + Combination. 27 IDEA

28.  The previous works focus on reducing the searching regions.  Specifically against “Exhaustive Search”.  Our method starts from breaking the relation between complexity of system and that of poses. Then, use the common feature and cumulative characteristic to cut down the searching space. 28 COMPARISON

29.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 29

30. 30 SYSTEM BLOCK  Bottom-up system:

31. 31 SYSTEM BLOCK  Bottom-up system:

32.  Steps:  Detection of edge candidates.  Production of part candidates.  Refinement of part candidates. 32 FAST PART DETECTION

33.  Detection and combination of segments (9 orientations). 33 DETECTION OF PART CANDIDATES

34.  Constraints on combination of edges.  Orientation, length ratio and color symmetry. 34 PRODUCTION OF PART CANDIDATES Neighbor orientation consideration

35.  HOG feature + Random forest training 35 REFINEMENT OF PART CANDIDATES Feature = [Length Orientation HOG_features] feature134 feature33feature2 ? ? feature400

36. 36 SYSTEM BLOCK  Bottom-up system:

37.  Problem:  No information about the classes of the limbs due to the low resolution of images or variation from hand gestures or appearance of shoes...etc.  Need another step to refine the combinations.  What information left?  Head-shoulder or head-torso. 37 PART COMBINATION

38.  Any possibility for us to estimate the position and orientation of head-torso based on the architecture of current combinations? 38 PART COMBINATION

39. 39 PART COMBINATION

40. 40 PART COMBINATION

41.  Conclusion for the clues mentioned in the previous slide.  Too complicate to combine the parts for the whole body.  Start from low-level combination of parts to reveal the benefits of physical constraints.  Break the problems into two levels.  Low-level combination.  High-level combination. 41 PART COMBINATION

42.  How far can we reach for low-level combination?  4-parts combination = lower body. 42 LOW-LEVEL COMBINATION

43.  False alarm exists.  Joints relative position + Random Forest 43 LOW-LEVEL COMBINATION feature134 feature33feature2 ? ? feature400

44. 44 HIGH-LEVEL COMBINATION

45. 45 SYSTEM BLOCK  Bottom-up system:

46.  Pose prediction.  Detection with DPM detector. 46 COMBINATION REFINEMENT

47.  Feature:  Relative size ratio and positions between low- level combinations and architecture of each low-level combination.  Random Forest. 47 POSE PREDICTION

48.  Use DPM detector to cover the intra-class variation.  Model: 48 DETECTION WITH DPM DETECTOR

49.  Much stronger than information of limbs.  Head-shoulder to head-torso.  Start from head-torso to combine limbs back. 49 USAGE OF HEAD-SHOULDER INFORMATION

50. 50 SYSTEM ILLUSTRATION

51.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 51

52.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 52

53.  Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 53