1. IIIT Hyderabad Pose Invariant Palmprint Recognition Chhaya Methani and Anoop Namboodiri Centre for Visual Information Technology IIIT, Hyderabad, INDIA

2. IIIT Hyderabad Palmprint Aquisition Controlled pose, scale, and illumination High accuracy Fixed Scanner/Camera Restricted Palm position Palmprint-Specific Can we use a generic camera as the acquisition device?

3. IIIT Hyderabad Unrestricted Palmprint Imaging Minimal Constraints Intuitive, user friendly New applications Multibiometric sensor

4. IIIT Hyderabad Challenges Background Illumination Contrast Noise Pose Scale

5. IIIT Hyderabad Previous Work Background – Skin Color – Hand Shape Illumination – Normalize Noise Shadow,Wrinkles, Pixel Noise. – Good features Scale Stenger et al. “Model-Based Hand Tracking Using a Hierarchical Bayesian Filter”, TPAMI 28(9), Sept. 2006 JDoublet, et al. “Contactless hand Recognition Using Shape and Texture Features”, ICSP 2006

6. IIIT Hyderabad Variations in Pose Induce perspective line distortions Associated with scale changes Performance degradation EER: ~22% Dataset: 100 palms, 5 images per palm. Solution Directions: 1.Compute Pose-Invariant Features 2.Correct Pose variations Non-rigid transformations are difficult to model Assumption of planarity

7. IIIT Hyderabad Invariance to Perspective Projection Cross Ratio, defined by 5 coplanar points Assume a stretched out palm to be planar Sensitive to point position Need reliable point detection Zheng, Wang and Boult : “ Application of Projective Invariants in Hand Geometry Biometrics”, IEEE Transactions on Information Forensics and Security, 2007. Point matches found using SIFT

8. IIIT Hyderabad Finding Pose Transformation Parameters Palm considered a planar surface. Homography defines transformation parameters between 2 planes given 4 point correspondences are known. –Where x'/c and y'/c is the resulting point. 4 distinctive point correspondences needed.

9. IIIT Hyderabad Solution using Interest Points We use a combination of stable points and a set of interest points as candidate matches. Stable/Valley points are the consistent points. Valley Points 4 valley points available. Only 2 can be used. Rest of the points must be selected from the palm lines. Thus, we choose a bag of candidate interest points. These points are refined later to get reliable interest points.

10. IIIT Hyderabad Proposed Solution Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment

11. IIIT Hyderabad Image Acquisition Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment Fixed Camera and Background Flexible Palm pose and position Natural Illumination variations Sample Image

12. IIIT Hyderabad Image Preprocessing & Palm Extraction Finger valley points are used to extract ROI and correct in-plane rotations Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment

13. IIIT Hyderabad Proposed Solution – Image Alignment Assumption of planarity of the palm surface Homography can be used to estimate pose 4 distinct point correspondences needed. Back to the same problem! Use a combination of stable points and interest points Valley Points Other interest points? Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment

14. IIIT Hyderabad Proposed Solution – Image Alignment Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment Descriptors are made using 11x11 windows around each of the candidate interest points Correspondences found using correlationSimilar process is followed for the second palm Assuming equal probability of occurrence for all points on the line, a richly sampled point set is chosen on the palm line

15. IIIT Hyderabad Proposed Solution – Image Alignment Input to RANSAC based Homography: the 2 valley points and iterative selection of the other two from the interest points. Final set of inliers in both the template and the set image.The best set of parameters found by RANSAC are used for the final transformation. The final transformed image. Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment

16. IIIT Hyderabad Proposed Solution: Computing Features and Matching Thresholded Gabor responses D. Zhang, A. W. K. Kong, You, J., Wong M., “Online Palmprint Identification”, PAMI 2003. Image Preprocessing & Palm Extraction Image Preprocessing & Palm Extraction Image Acquisition Feature Extraction Matching Image Alignment Image Alignment dist(final) = min(dist(fixed), dist(corrected))‏

17. IIIT Hyderabad Datasets 100 palms, 5 images per palm Pose variations up to 45 degrees 50 palms, from PolyU dataset 10 synthetic poses per palm: 0 - 45 degrees

18. IIIT Hyderabad Results Comparison of EER values MethodSynthetic DataReal Data 0 ◦ - 20 ◦ 20 ◦ -30 ◦ 30 ◦ -35 ◦ 35 ◦ -40 ◦ 40 ◦ -45 ◦ Fixed Pose Approach 0.01%3.24%3.71%16.93%30.92%22.07% MethodSynthetic DataReal Data 0 ◦ - 20 ◦ 20 ◦ -30 ◦ 30 ◦ -35 ◦ 35 ◦ -40 ◦ 40 ◦ -45 ◦ Fixed Pose Approach 0.01%3.24%3.71%16.93%30.92%22.07% Blind Pose Approach 16.48%12.40%11.14%14.98%11.92%16.51% MethodSynthetic DataReal Data 0 ◦ - 20 ◦ 20 ◦ -30 ◦ 30 ◦ -35 ◦ 35 ◦ -40 ◦ 40 ◦ -45 ◦ Fixed Pose Approach 0.01%3.24%3.71%16.93%30.92%22.07% Blind Pose Approach 16.48%12.40%11.14%14.98%11.92%16.51% Proposed Approach 0.47%4.19%11.14%14.98%11.92%8.71%

19. IIIT Hyderabad Results: Synthetic Data

20. IIIT Hyderabad Results: Real Data

21. IIIT Hyderabad Results Semilog curve to observe the highlighted data. (p) : GAR low even with high FAR. Indicates genuine pairs with low similarity. Reasons: Blur, wrinkles, etc. (q) : There is a sharp drop in the GAR. Indicates imposter pairs with high similarity. Reasons: Pixel saturation, specular reflections of skin etc. (r) :The drop of GAR in case of proposed approach is earlier. Indicates imposter pairs with increased similarity. Reasons: Inherent in the algorithm.

22. IIIT Hyderabad Video Based Palmprint Recognition Base ImageBase Image + 2Base Image + 6Base image + 10 12.46%10.92%9.83%7.87% Successive addition of Gabor responses. Images shown after adding 2, 6 and 10 images respectively.

23. IIIT Hyderabad Conclusion/Observation Proposed view invariant recognition system for Palmprint. Very difficult to find point correspondences for palm. Solution using point correspondence of stable and interest points. RANSAC based Homography used to choose from approximate point correspondences. Major role played by illumination variations and noise. Video based palmprint recognition is a possible solution. Future Work: To study the effects of video based palmprint recognition in further in more detail.

24. IIIT Hyderabad Thank You