1. Human Upper Body Pose Recognition Using Adaboost Template for Natural Human Robot Interaction
Liyuan Li, Jerry Kah Eng Hoe, Xinguo Yu, Li Dong, and Xinqi Chu
Institute for Infocomm Research (I2R), Singapore

2. Outline Introduction Related Works The Method
The Problem
Template Modeling
Adaboost Template
Recognition & Segmentation
Experiments and Evaluations
Conclusions

3. Introduction Motivations Difficulties for approaches on 2D images
Upper body pose is one of important clues of human social behavior in natural conversation, especially multiple persons are involved in the conversation;
A social robot has to be aware of various clues from human body for intelligent and natural human-robot-interaction;
An important clues in our social robots for attention estimation and engagement management (direction, distance, motion state, upper body pose, face pose, gaze, etc.)
Difficulties for approaches on 2D images
Pose ambiguity due to the lost depth information and self-occlusion;
Limited view of human objects when engaged in face-to-face interaction;
Variations of human shapes, scales, clothes, poses, etc.
Complexity of visual features due to lighting conditions, cluttered backgrounds, and crowded scenes.

4. Related Works Human Body Pose Recognition in Computer Vision
2D silhouette based approaches (e.g., Gavirla and Philomin, ICCV’09, Mittal, et al, IEEE AVSS’03, Dimitrijevic, et al, ICCV Workshop’05)
2D pictorial models (e.g., Ju, et al, FG’96, Felzenszwalb and Huttenlocher, IJCV 2005, Andriluka, et al, CVPR’09, Ferrari, et al, CVPR’09)
3D structure models (e.g., Taylor, CVIU 2000, Lee and Cohen, ECCV’04)
Template Matching
Deformable template matching (e.g., Cootes, Edwards, Taylor, Active Appearance Models)
Object tracking (Yilmaz, et al, ACM CS 2006 (Survey))
Face detection (Yang, et al, IEEE T-PAMI 2002 (Survey))
Image registration (Zitova and Flusser, IVC 2003 (Survey))
Adaboost Learning in Vision
Face detection (Viola & Jones, CVPR’01 (Cascade Classifiers))
Multi-view face detection (Huang, et al., IEEE T-PAMI 2007 (Vector Boosting Algorithm))
Multiclass object detection with shared features (Torralba, et al, CVPR’04 (Joint Boosting Algorithm))
Online tracking (e.g., Avidan, “Ensemble Tracking,” IEEE T-PAMI 2007)

5. Method: The Problem Problem formulation Challenges
Classify the upper body poses into seven categories: views of 0°, ±30°, ±60°, and ±90° to the camera.
Challenges
The depth measures from disparity images are not accurate;
Inter-class variations due to variations of human sizes, shapes, poses, and clothes;
Inter-class variations due to human positions to the camera;
Incompletion of disparity measures from body due to the lack of texture features.

6. Method: Template Modeling
Learning the basic templates
Learning the mean template for each category
Learning the variance template for each category
Learning the percentage template for each category

7. Method: Adaboost Template
Definition of positive and negative regions
Design of weak classifiers R+ R−

8. Method: Learning Adaboost learning algorithm
Given Nc training samples for category c.
Initialize:
For t=1,…,T
For each pixel x in the template
Compute the error with respect to the distribution Dt
Choose
Tune the template boundary
Update the distribution

9. Method: Recognition & Segmentation
Adaptive model-driven segmentation:
Quality level of disparity measurements
Adaptively compensate for the missing disparity measurements

10. Experiments and Evaluations
A New Benchmarking Data Set
Camera: Videre Design STOC stereo camera.
Data Set: 430 images from 19 individuals.
Training samples: Randomly select 93 images of 8 persons from the data set, among them, 28 for 0° view, 13 for +30° view, 10 for -30° view,
11 for +60° view, 10 for -60° view, 11 for +90° view, and 10 for -90° view.
Baseline Algorithm:
Template matching: 3D surface template matching (Breitenstein, et al, “Real-Time Face Pose Estimation from Single Range Images”, CVPR’08)
Distance: Let T(x) be a normalized input sample
Recognition

11. Experiments and Evaluations
Results on recognition: On average, the accuracy rate increased from 67.4% to 90.7%.
Template Matching
-90°
-60°
-30° 0°
+30°
+60°
+90°
100%
60%
37.1%
2.9%
2.6%
31.6%
65.8%
1.75%
14.3%
78.6%
3.6%
61.0%
9.7%
29.3%
27.8%
72.2%
Adaboost Template
-90°
-60°
-30° 0°
+30°
+60°
+90°
100%
3.4%
88%
6.9%
1.7%
3.3%
81.7%
15%
9.5%
87.3%
3.2%
5.2%
77.6%
17.2%
98.3%

12. Experiments and Evaluations
Results on segmentation:
Pose recognition
Quality estimation
Top-down segmentation

13. Application: Attention Estimation
Deployed in a robot receptionist for attention estimation

14. Conclusions
A new approach of human upper body pose recognition for human robot interaction
A new template model: Adaboost template
Easy for training (no need of negative samples)
Achieve good balance between generality and specialties of training samples
Both recognition and segmentation
Deployed and tested on a robot receptionist for attention estimation and the management of engagement in dialogs which may involve multiple participants.

15. Thank You!