Student: Yao-Sheng Wang Advisor: Prof. Sheng-Jyh Wang ARTICULATED HUMAN DETECTION 1 Department of Electronics Engineering National Chiao Tung University Hsinchu, Taiwan 1

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 2

 Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 3

 Why we care about human detection?  We are human beings!  Wide range of applications:  Automotive safety  Surveillance system  Indoor care  Crime alert  Human-Computer Interface … etc. 4 MOTIVATION

 Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 5

 What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 6 CHALLENGE

 What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 7 CHALLENGE

 What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 8 CHALLENGE

 What makes human detection so difficult?  Illumination condition  Cluttered background  Change of viewpoints  Occlusion  Wearing difference  Diversity of human  Pose variation 9 CHALLENGE

 Progress on “Machine Learning” technology  Handle more general and complicate cases.  Definition:  “Articulated Human Detection”. 10 CHALLENGE

 Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 11

 Deformable Part Model  Root filter (mask).  Part filter (mask).  Penalty function. 12 REPRESENTATIVE WORKS (I) [P. Felzenszwalb, D. McAllester, and D. Ramanan. A discriminatively trained, multi-scale, deformable part model. In CVPR, 2008.]

 Pose-let: 13 REPRESENTATIVE WORKS (II) [Lubomir Bourdev, Jitendra Malik. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. In ICCV, 2009.]..

 Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 14

 Problems:  System complexity increased with the complexity of human poses.  More detectors needed.  Exhaustive search.  Sliding window method + Image pyramid.  Both problems leads to unacceptable speed for applications in real life. 15 POTENTIAL PROBLEMS

 Introduction  Motivation  Challenge  Representative Works  Potential Problems  Target  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 16

 Target in the thesis:  Propose a detection scheme with acceptable detection speed in dealing with highly intra- class variation from the change of pose and viewpoint. 17 TARGET

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 18

 Better features:  Cheap to compute and capture crucial information at the same time. Ex: HOG.  Better classifiers:  Linear classifiers.  Ex: Adaboost, Linear-SVM and Random-forests.  Better prior knowledge:  Ex: Information about ground plane. 19 RELATED WORKS

 Cascades:  Cascade the part filters to reduce the searching regions. 20 RELATED WORKS [P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with Deformable Part Models. In CVPR, 2010.]

 Discard non-promising hypotheses.  Class-dependent:  Branch and bound. (CVPR, 2008)  Class-independent:  What is an object? (CVPR, 2010)  Closure boundary, different appearance or salience.  Segmentation as selective search. (ICCV, 2011) 21 RELATED WORKS

 Feature response approximation:  Feature approximation in testing step.  Feature approximation in training step. 22 RELATED WORKS [R. Benenson, M. Mathias, R. Timofte, and L. Van Gool. Pedestrian detection at 100 frames per second. In CVPR, 2012.] [P. Dollár, S. Belongie, P. Perona. The fastest pedestrian detector in the west. In BMVC, 2010.]

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 23

 Recall the memory of the first problem:  System complexity increased with the complexity of human poses (include variation of viewpoints).  How can we break the relation between the complexity of system and the one of human poses?  Choose stable features or body parts for detection. 24 IDEA

 Better prior knowledge: 25 IDEA

 Recall the memory of the second problem:  Exhaustive search.  “Sliding Window” + “Image Pyramid”.  How can we reduce the searching region?  Detect the common feature among these parts.  Use the cumulative characteristic of the feature to handle the variation of scale. 26 IDEA

 Common feature  Body parts consist of combination of two edge segments.  Cumulative characteristic  Edge detector with fixed size + Combination. 27 IDEA

 The previous works focus on reducing the searching regions.  Specifically against “Exhaustive Search”.  Our method starts from breaking the relation between complexity of system and that of poses. Then, use the common feature and cumulative characteristic to cut down the searching space. 28 COMPARISON

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 29

30 SYSTEM BLOCK  Bottom-up system:

31 SYSTEM BLOCK  Bottom-up system:

 Steps:  Detection of edge candidates.  Production of part candidates.  Refinement of part candidates. 32 FAST PART DETECTION

 Detection and combination of segments (9 orientations). 33 DETECTION OF PART CANDIDATES

 Constraints on combination of edges.  Orientation, length ratio and color symmetry. 34 PRODUCTION OF PART CANDIDATES Neighbor orientation consideration

 HOG feature + Random forest training 35 REFINEMENT OF PART CANDIDATES Feature = [Length Orientation HOG_features] feature134 feature33feature2 ? ? feature400

36 SYSTEM BLOCK  Bottom-up system:

 Problem:  No information about the classes of the limbs due to the low resolution of images or variation from hand gestures or appearance of shoes...etc.  Need another step to refine the combinations.  What information left?  Head-shoulder or head-torso. 37 PART COMBINATION

 Any possibility for us to estimate the position and orientation of head-torso based on the architecture of current combinations? 38 PART COMBINATION

39 PART COMBINATION

40 PART COMBINATION

 Conclusion for the clues mentioned in the previous slide.  Too complicate to combine the parts for the whole body.  Start from low-level combination of parts to reveal the benefits of physical constraints.  Break the problems into two levels.  Low-level combination.  High-level combination. 41 PART COMBINATION

 How far can we reach for low-level combination?  4-parts combination = lower body. 42 LOW-LEVEL COMBINATION

 False alarm exists.  Joints relative position + Random Forest 43 LOW-LEVEL COMBINATION feature134 feature33feature2 ? ? feature400

44 HIGH-LEVEL COMBINATION

45 SYSTEM BLOCK  Bottom-up system:

 Pose prediction.  Detection with DPM detector. 46 COMBINATION REFINEMENT

 Feature:  Relative size ratio and positions between low- level combinations and architecture of each low-level combination.  Random Forest. 47 POSE PREDICTION

 Use DPM detector to cover the intra-class variation.  Model: 48 DETECTION WITH DPM DETECTOR

 Much stronger than information of limbs.  Head-shoulder to head-torso.  Start from head-torso to combine limbs back. 49 USAGE OF HEAD-SHOULDER INFORMATION

50 SYSTEM ILLUSTRATION

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 51

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 52

 Introduction  Related Works  Idea  Proposed Method  Experimental Results  Conclusion  Reference OUTLINE 53