3D Hand Pose Estimation by Finding Appearance-Based Matches in a Large Database of Training Views

outline Introduction Propose Framework Space Complexity Synthetic Versus Real Training Data Edge-Based View Matching Experimental Results Future Work Conclusion

Introduction Estimate 3D hand pose from a single image by matching the image with a large database. What are the storage requirement for an adequate database of training views? What are the similarity measures? How can the matching be done efficiently?

Introduction In the database contains more than 100,000 image, generated from 26 hand shape. In the real images use skin color dectection.

Proposed Framework Model the hand as an object, consisting 16 links : the palm and 15 links corresponding to finger parts.

Proposed Framework The five joints connecting fingers between finger links allow rotation with two degrees of freedom (DOFs). The 10 joints between finger links allow rotation with on DOF A total of 20 DOFs describes completely all degrees of freedom in the joint angles.

Proposed Framework Add the viewing parameter. Given a hand configuration vector and a viewing parameter vector, define the hand pose vector

Proposed Framework The generic framework that we propose for hand pose estimation is the following: 1. create a database containing a uniform of all possible views of all possible configuration. 2. for each novel image, find the database views that are the most similar. Use the parameters of those views estimates for the image.

Space Complexity Depend on the number of database images. In this paper, have 86 viewpoints and generated 48 images for each viewpoints Use PCA to reduce hand shape configuration

Synthetic Versus Real Training Data A big advantage of synthetic training sets is that the labeling of the data can be done automatically. Problem : hard to correct, need multicamera setup.

Edge-Based View Matching Have defined image similarity using chamfer distance. Given an input image, extract its edge pixels using an edge detector (canny) and store the coordinates in a set X.

Experimental Results DB have 26 different hand shapes, each shape rendered from 86 viewing direction, each direction have 48 images. Test have 28 real hand pose image.

Experimental Results Define the distance D between a point and a set of points X to be the Euclidean distance between and the point in X that is the closest to :

Experimental Results

Future work Database use real hand pose image Add finger detector

Conclusions Almost half of the test images the system retrieved correct views in the top ten matches.