1. ONLINE FINGERPRINT VERIFICATION
Sharat Chikkerur
Center for Unified Biometrics and Sensors
University at Buffalo
Advisor: A. N. Cartwright
Committee: V. Govindaraju, A. H. Titus, L. Kondi

2. Abstract Background Challenges Contributions
Traditional password/token based authentication schemes are insecure and are being replaced by biometric authentication mechanisms
Fingerprints were one of the first biometrics to be widely used
Despite 40 years of research, fingerprint recognition is still an open problem.
Challenges
Feature extraction is very unreliable in poor quality prints
Matching fingerprints under non linear distortion is difficult
Contributions
New fingerprint image enhancement using STFT analysis.
New feature extraction algorithm based on chain code contours
New graph based matching algorithm robust to non linear distortion

3. Outline Introduction Biometrics Fingerprints 101
Fingerprint Image Enhancement
Minutia Feature Extraction
Matching Algorithm
Conclusion
Software Demos

4. Biometrics Definition Examples Physical Biometrics
Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits
Examples
Physical Biometrics
Fingerprint, Hand Geometry, Face
Measurement Biometric
Dependent on environment/interaction
Behavioral Biometrics
Handwriting, Signature, Speech, Gait
Performance/Temporal biometric
Dependent on state of mind
Chemical/Biological Biometrics
Skin spectroscopy
DNA, blood-glucose
Biometrics offers a promising solution for reliable and uniform identification and verification of an individual. Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits. Physical biometrics rely on physiological features such as fingerprints, hand geometry, iris pattern, facial features etc. for identity verification. Behavioral biometrics depends upon behavioral features such as speech patterns, handwriting, signature, walking gait etc. for authentication. These traits are unique to an individual and hence cannot be misused, lost or stolen.
Physiological biometrics are more stable and robust than behavioral biometrics and a single sample is enough to obtain identifying information. Biometrics such as signature and speech are acquiredand learnt over time and also subject to the user’s state of mind and disposition. Multiple samples are required to acquire a stable representation of user features.
Biometrics are based on established scientific principles as a basis for authentication.

5. Fingerprints as a Biometric
High Universality
A majority of the population (>96%) have legible fingerprints
More than the number of people who possess passports, license and IDs
High Distinctiveness
Even identical twins have different fingerprints (most biometrics fail)
Individuality of fingerprints established through empirical evidence
High Permanence
Fingerprints are formed in the fetal stage and remain structurally unchanged through out life.
High Performance
One of the most accurate forms of biometrics available
Best trade off between convenience and security
High Acceptability
Fingerprint acquisition is non intrusive. Requires no training.
The skin of the fingers is corrugated by a pattern of contours. These contour ridges are formed during foetal development and remain unchanged throughout a person’s life. It has been established that fingerprint patterns are unique to each individual. Of all the biometric techniques, fingerprint based authentication has been the most well established and well researched topic. It has been used in forensic science for authentication since the early 1900s.
Fingerprints have been an important tool used by for law enforcement and forensics for over a century. Automatic Fingerprint Identification Systems (AFIS) can provide absolute identification of an individual by processing the image of a fingerprint. Fingerprints are formed while the foetus is 4 months old and remain unchanged through out an individual’s life time

6. Fingerprints 101: Fingerprint Classes
A fingerprint is made up of system of oriented friction ridges
A fingerprint can be classified based on type the ridge flow pattern
The corrugated surface of the fingerprint is made up of ridges and valleys cover that the entire palmer surface of the hand. The flow pattern of these ridges and valleys are unique to each individual. These patterns that are used for identification and authentication. The image below shows the image of a fingerprint along with the distinguishing features on the print.
The flow of the ridges and patterns has been classified into 5 broad classes. This classification is used to catalog the fingerprints and also in authenticating two prints. Henry systems follow an elaborate classification scheme of cataloging and filing forensic prints. Fig1 shows the different classes of ridge flows. This methods cannot be used to distinguish between two fingerprints.
Fig 2 shows the distinguishing features on the fingerprint. These features are discontinuities or anomalies in the normal flow of ridges on the surface of the finger. These features are termed as minutiae (small details). There are eighteen different types of minutiae. Fig1 shows the most commonly encountered ones and their names. Fig 2 shows a thumbprint captured on paper. These are typically the kind of images that forensic AFIS (Automatic Fingerprint Identification Systems) have to deal with. The quality of the print not only deteriorates during capture but also during storage and hence AFIS systems are more sophisticated than their biometric counterparts.
Classification helps in narrowing down possible matches
In reality, the class distribution is skewed (>65% are loops)
Used only in law enforcement applications

7. Fingerprints 101: Ridge Characteristics
Fingerprints can be distinguished based on the ridge characteristics
Ridge characteristics mark local discontinuities in the ridge flow
No two individuals have the same pattern of ridge characteristics at the same relative locations
Local Features
Global Features

8. Prior Related Work: Matching Paradigms
Manual
Human experts use a combination of visual, textural, minutiae cues and experience for verification
Still used in the final stages of law enforcement applications
Image based
Utilizes only visual appearance.
Requires the complete image to be stored (large template sizes)
Texture based
Treats the fingerprint as an oriented texture image
Less accurate than minutiae based matchers since most regions in the fingerprints carry low textural content
Minutiae based
Uses the relative position of the minutiae points
The most popular and accurate approach for verification
Resembles manual approach very closely.

9. Image Based Matching: Optical Correlation
Advantages
Image itself is used as the template
Requires only low resolution images
Optical correlation makes it extremely fast (Choudary and Awwal ’99, Lee et al. 99, Roberge et al. 99, Baze et al.00)
Disadvantages
Image itself is used as the template (template size about 30 KB)
Requires accurate alignment of the two prints (unreliable in poor prints)
Not robust to changes in scale, orientation and position.

10. Texture Based Matching: Filterbanks
Advantages
Uses texture information (lost in optical and minutiae based schemes)
Performs well with poor quality prints
Features are statistically independent from minutiae and can be combined with minutiae matchers for higher accuracy (Jain et al. 00, Jain et al 01)
Disadvantages
Requires accurate alignment of the two prints (unreliable in poor prints)
Not invariant to translation, orientation and non-linear distortion.
Less Accurate than minutiae based matchers

11. Minutiae Based Matching
Advantages
Invariant to translation, rotation and scale changes
Very accurate (Ratha et al 96, Jain et al. 97, Jian Yau 00, Bazen and Garez 03)
Disadvantages
Minutiae extraction is error prone is low quality images
Not robust to non-linear distortion.
Does not use visual and textural cues

12. General Architecture

13. Outline Introduction Fingerprint Image Enhancement
Need for Enhancement
Prior Related Work
Proposed Algorithm: STFT Analysis
Experimental Evaluation
Minutia Feature Extraction
Matching Algorithm
Software Demos

14. Need for Enhancement
What you see
What you ‘think’ you see

15. Reality: What you usually get..
High contrast print
Typical dry print
Faint print
The performance of any fingerprint recognizer depends upon the quality of the fingerprint image processed. AFIS systems have achieved high performances in case of reasonably good prints. However there is not yet satisfactory methods to deal with bad quality fingerprints as often encountered in forensic and biometric applications. Effective methodologies for cleaning the valleys between the ridge contours are lacking in current fingerprint recognition systems. The figures show some prints acquired under different conditions. It can be seen that the minutiae features cannot be easily extracted from the original gray scale image. Therefore some form on enhancement is required before processing the fingerprint image further.
Low contrast print
Typical Wet Print
Creases

16. Challenges Challenges
Fingerprint image is non stationary (has dominant local orientation and frequency)
General purpose image processing algorithms are not useful
Traditional operators and filters assume Gaussian noise model
‘Noise’ in fingerprint images consists mostly of ridge breaks
Contextual Filters
Existing techniques are based on ‘contextual’ filtering
Filter parameters are adapted to each local neighborhood
Filter parameters in ‘unrecoverable’ regions can be interpolated based on its neighbors

17. Prior Related Work: Spatial Filtering
(Yang et.al 1996, Greenberg et. Al 1999) proposed local anisotropic filtering
Filter kernel adapts at each pixel location
Hong et al, 96/98 proposed the use of Gabor filters for enhancement
Gabor filter has the best joint space-frequency localization
Does not handle high curvature regions well due to block wise approach.
Even Symmetric Kernel
Fourier spectrum showing the localization

18. Prior Related Work: Fourier Domain Filtering
Sherlock et al 94, proposed the use of Fourier domain filtering
The image is convolved with a filter bank of directionally selective filters
Image enhanced by selecting a linear combination of filter responses
Watson et al. 94, proposed the use or ‘root filtering’ for enhancement.(Pseudo matched filter)
Does not require the computation of orientation images
Root Filtering
Fourier Domain Filtering

19. Traditional Approaches
Local Orientation
(x,y)
Gradient Method
Enhancement
Frequency/Spatial
The fingerprint can be seen as an oriented texture. This property of the fingeprints can be used while enhancing the fingerprint image. Traditional image processing methods such as gaussian smoothening, or low pass filtering cannot be used as they tend to bridge the gaps between the ridges. The ridges have to be enhanced only in a direction parallel to their orientation. There are spatial and frequency domain methods that perform such kind of filtering. Spatial domain methods are based on anisotropic filters aligned in the ridge direction(??) or are based on Gabor filtering(Anil Jain et al). Frequency domain methods utilizing FFT (Grother et al, Monroe et al) are also present and are more successful. The enhancement depends upon the accurate estimation of the local ridge orientation and local ridge spacing within the image. Presently the local orientation and the local ridge spacing are obtained through separate algorithms. The enhancement is performed using this information. This approach requires multiple passes and is computationally expensive. We propose a unified approach to fingerprint image enhancement using FFT analysis. The proposed method extracts the local orientation and ridge spacing information in one pass and at the same time performs the enhancement.
Local Ridge Spacing
F(x,y)
Projection Based Method
[Ratha et al 95]

20. Proposed Approach: Overview
Region
Mask
STFT
Analysis
Frequency
Image
Fourier domain
Enhancement
The proposed methods uses FFT based frequency domain analysis to estimate the ridge spacing and orientation in the image. The enhancement is also done in the frequency domain. The analysis yields the energy map, orientation map and ridge spacing map corresponding to the image. The energy map indicates the presence of ridges and their contrast within the image. The energy map can be successfully used to segment the fingerprint image from the background. The orientation map provides the orientation of the ridges in a local neighborhood. This information is used in the enhancement procedure. The ridge spacing map provides the inter ridge distance variation in the image. The ridge spacing can be used in the enhancement procedure to design a suitable band pass filter that allows ridges and eliminates noise and gray level gradients across the image.
The following slides show the results of the FFT analysis algorithm.
Orientation
Image
Coherence
Image

21. STFT Analysis
Fingerprint image is non stationary, so we require both space and frequency resolution: time frequency analysis
STFT in 1D
STFT in 2D

22. Surface Wave Model
Fingerprint ridges can be modeled as an oriented wave
Local ridge orientation
Local ridge frequency
Surface wave
Local Neighborhoods
Validity of the model

23. Parameter Estimation
Paradigm: The Fourier domain response can be viewed as a distribution of surface waves. Each term F(r, θ) corresponds to a surface wave of frequency 1/r and orientation θ
We seek to find the most likely surface wave and hence estimate the dominant direction and frequency
We can represent the Fourier spectrum in polar form as F(r,θ) The power spectrum is reduced to a joint probability density function using
The angular and frequency densities are given by marginal density functions
The enhancement techniques can be divided into two distinct classes (I) Spatial domain techniques and (ii) Frequency domain based techniques
Spatial domain techniques: This technique involves convolving a filter kernel(mask) with the entire image to obtain the filtered image. Standard spatial domain filters such as isotropic gaussian filters and other low pass filters cannot be applied to fingerprint images as they blur the image in all directions. Fingerprint image is a highly oriented structure. The spatial filter that is used for enhancement should blur or average the pixels in the direction of the ridge at the same time increasing the contrast between the ridges and valleys in a direction perpendicular to the ridge. In general, spatial domain filters are computationally less expensive compared to frequency domain based filters.
Frequency domain filters are based on enhancement in the Fourier domain. The most common approaches include the block level filtering as proposed by Grother, Candela et al. In this method the Fourier transform of the block is multiplied by its power spectrum raised to a fractional power. This is similar to matched filtering approach found in communication and signal processing systems. Other approaches such as Morno et al. are based on directionally selective filters. The image is passed through a filter bank where each filter is oriented in a given direction. The resulting image is obtained by combining the images obtained from each of the directional filter. In general, Fourier based methods are computationally expensive but yield better results when compared to spatial filters.
Most of the existing methods in fingerprint image enhancement are based on ‘Contextual filtering’, where the filter parameters adapt to the local ridge orientation and frequency.
Anisotropic filters are a special case of spatial contextual filters. Unlike standard gaussian kernel, the anisotropic filter kernel is not uniform in all directions. It is designed to smooth selectively in the direction of the ridge. The shape of the kernel adapts by using the local ridge orientation and ridge frequency. The advantage of this approach is that it creates far fewer spurious ridges and artifacts compared to Fourier based methods and is more robust to noise.
Gabor filters can be applied in both spatial and frequency domain. Unlike other directional filters, Gabor filters are both directionally and frequency selective filters. It has been shown that the Gabor kernel has the optimal space frequency resolution. In this approach, the image is divided into blocks and the orientation and ridge frequency is estimated in each block. The block is then filtered using a Gabor kernel that has the corresponding orientation and frequency. The final image is obtained by combining the filtered block images.

24. Ridge Orientation Image

25. Region Mask
The surface wave approximation does not hold in the background region
The region mask is obtained by simple thresholding of the block energy image

26. Frequency Image
[Jain et al 00]

27. Coherence Image
Block processing is unreliable in regions of high curvature
Sherlock and Monro 94, relax filter parameters near the singular locations
Estimation of singular point is difficult in poor images!
We use an angular coherence measure proposed by Rao and Jain 90

28. Enhancement
The slide shows the results of the frequency domain enhancement. Presently the enhancement approach suggested by Grother et al is used to perform the enhancement. The frequency and orientation map obtained in the previous section is not included in the filter design. The results will only improve with their inclusion.

29. Additional Enhancement Results

30. Qualitative Comparison
Root Filtering
Original Image

31. Qualitative Comparison(cont.)
Gabor Filter based Enhancement
Proposed Approach

32. Objective Evaluation
We evaluated the effect of enhancement on 800 images from FVC2002 DB3
The evaluation consists of 2800 genuine test and 4950 impostor tests
It can be seen that the matcher performance improves with enhancement

33. Outline Introduction Fingerprint Image Enhancement
Minutia Feature Extraction
Prior Related Work
Chain code contour
Experimental Evaluation
Matching Algorithm
Conclusion
Software Demos

34. Background Minutiae represent local discontinuities in ridge flow
Minutiae features are the most widely used fingerprint representation
There are several standards such as CBEFF (file format) and ANSI-NIST (interchange format) standards for minutiae based fingerprint representation
Minutiae extraction approaches may be broadly categorized into
Binarization based approaches
Direct gray scale extraction

35. Prior Related Work Binarization Approaches
MINDTCT,NIST NFIS, (Garris et. Al, 02)
Directionally adaptive binarization
Template matching is used to detect minutiae
Adaptive Flow Orientation technique (Ratha et. al., 95)
Binarization is performed by peak detection
Peak detection leads to false positives in regions of poor ridge constrast.
Direct Gray Scale Ridge Following
Ridge Following (Maio and Maltoni 97, Jiang and Yau 01)
Based on ridge pursuit
Has low computational complexity.
Cannot handle poor contrast prints and images with poor ridge structure.
Relies on a good orientation map for ridge pursuit
The enhancement techniques can be divided into two distinct classes (I) Spatial domain techniques and (ii) Frequency domain based techniques
Spatial domain techniques: This technique involves convolving a filter kernel(mask) with the entire image to obtain the filtered image. Standard spatial domain filters such as isotropic gaussian filters and other low pass filters cannot be applied to fingerprint images as they blur the image in all directions. Fingerprint image is a highly oriented structure. The spatial filter that is used for enhancement should blur or average the pixels in the direction of the ridge at the same time increasing the contrast between the ridges and valleys in a direction perpendicular to the ridge. In general, spatial domain filters are computationally less expensive compared to frequency domain based filters.
Frequency domain filters are based on enhancement in the Fourier domain. The most common approaches include the block level filtering as proposed by Grother, Candela et al. In this method the Fourier transform of the block is multiplied by its power spectrum raised to a fractional power. This is similar to matched filtering approach found in communication and signal processing systems. Other approaches such as Morno et al. are based on directionally selective filters. The image is passed through a filter bank where each filter is oriented in a given direction. The resulting image is obtained by combining the images obtained from each of the directional filter. In general, Fourier based methods are computationally expensive but yield better results when compared to spatial filters.
Most of the existing methods in fingerprint image enhancement are based on ‘Contextual filtering’, where the filter parameters adapt to the local ridge orientation and frequency.
Anisotropic filters are a special case of spatial contextual filters. Unlike standard gaussian kernel, the anisotropic filter kernel is not uniform in all directions. It is designed to smooth selectively in the direction of the ridge. The shape of the kernel adapts by using the local ridge orientation and ridge frequency. The advantage of this approach is that it creates far fewer spurious ridges and artifacts compared to Fourier based methods and is more robust to noise.
Gabor filters can be applied in both spatial and frequency domain. Unlike other directional filters, Gabor filters are both directionally and frequency selective filters. It has been shown that the Gabor kernel has the optimal space frequency resolution. In this approach, the image is divided into blocks and the orientation and ridge frequency is estimated in each block. The block is then filtered using a Gabor kernel that has the corresponding orientation and frequency. The final image is obtained by combining the filtered block images.

36. Binarization Method Acquisition Binarization Thinning
Minutia Detection

37. Proposed Approach: Chain Code Contours
Provides a lossless description of the contour and also gives direction and curvature information.
Translation and rotation invariant
Used in computer vision for encoding object boundaries
Used for character recognition (Madhavanth et. al 99)

38. Minutiae Detection using Chain Codes
Minutiae are encountered as points of ‘significant’ turn on the contour
Left turn: Ridge ending
Right turn: Bifurcation

39. Determining Turn Points

40. Results

41. Results (cont.)

42. Experimental Evaluation
Test Data
150 prints from FVC2002(DB1) were randomly selected for evaluation.
Ground truth was established using a semi automated truthing tool.
Results compared using NIST NFIS open source software.
Metrics
Proposed by Sherlock et. Al 94
Sensitivity: Ability of the algorithm to detect true minutiae
Specificity : Ability of the algorithm to avoid false positives
Flipped : Minutiae whose type has been exchanged

43. Quantitative Analysis : Results
Examples
File Name
NIST
Proposed method
Actual TP FP M F
10_8.tif 18 16 8 2 1 17
11_6.tif 50 40 4 10 41 9
12_8.tif 29 22 5 7 3
13_6.tif 35 28
14_6.tif 44 34 12 6 37 13
15_7.tif 38
16_7.tif 36
17_6.tif 43 11
18_8.tif 31 32
19_7.tif 26

44. Sensitivity distribution
Results
Summary results
Count TP(NIST) > proposed : 40 of 150
Count E(NIST) < proposed : 40 of 150
Metric
NIST
Proposed
Sensitivity(%)
82.8
83.5
Specificity(%)
77.2
76.8
Flipped(%)
12.0
10.9
Sensitivity distribution
Overall statistics

45. Outline Introduction Fingerprint Image Enhancement
Minutia Feature Extraction
Matching Algorithm
Prior Related Work
New Representation: K-plet
Local Matching: Dynamic Programming
Consolidation: Coupled BFS
Experimental Evaluation
Conclusion
Software Demos

46. Minutiae Based Matching
Challenges
Minutiae extraction is error prone is low quality images
Not robust to non-linear distortion.
Intra-user variation

47. Challenges: Non-linear Distortion

48. Challenges: Quality and Intra-user variance
Variation in quality
Intra-user variation

49. Prior Related Work :Global Matching
Point correspondences not known : combinatorial problem
Relaxation approach (Ranade and Rosenfield 93)
Likelihood of each match is either decreased or increased at each iteration based on compatibility of rest of the points
Iterative approach makes it too slow to be practical
Generalized Hough Transform (Ratha et al. 96)
All possible transformation represented as a quantized search space
Searches for the most optimal transform in the search space
Very fast
Ridge Alignment (Jain et al. 97)
Performs explicit alignment before matching
Each minutiae is associated with its ridge (represented by a curve)
The alignment is based on ridge correspondence
Global matching is then performed using string edit distance

50. Prior Related Work: Local Matching
(Jiang and Yau 00)
11 dimensional local features derived from reference minutiae and two closest neighbors
Best match is used only for explicit alignment
(Jea and Govindaraju 04)
5 dimesional features Si (ri0, ri1, φi0, φi1, δi) derived from two closest neighbors
Alignment is still required
(Ratha et al. 00)
‘Star’ representation derived from all minutiae within a particular radius
Consolidation by checking consistency
(Garris et. al 03: BOZORTH3)
Line features
Consolidation by linking consisting matches
Instead of the 3-elements minutiae representation, we use this 5-elements secondary feature representation.
For a given feature point or called central minutia, we find the two nearest neighboring minutiae of it. The secondary feature contains the information of the distances between the central minutia to the neighboring minutiae, the orientation between them and the angle between this two line segments.
The secondary feature is derived only from the widely-used minutiae representation, thus our algorithm can be easily adapted other systems.
We do not use the ridge count information, because it is not required information in minutiae representation.
We do not use the minutiae types as the feature, because they are not reliable.
Here is an example, these two feature points are extracted from the same finger but in different impressions. One is interpreted as a bifurcation while the other one is seen as a ridge ending.
Secondary feature also has some other advantages.
It is a pure localized feature. It doesn’t rely on any global landmarks.
It is orientation invariant. There is no pre-alignment stage needed.

51. Proposed Algorithm Representation K-Plet
Features invariant to rotation and translation
Local relationship formally represented by a directed graph
Local Matching
Posed as a string alignment problem and solved by dynamic programming
Matches all neighbors simultaneously
Consolidation
Coupled Breadth First Search
Breadth first search is used to propagate the matches
Similar to human verification
No explicit alignment required at any stage

52. Neighborhood Representation: K-plet

53. K-plet Θ r Φ

54. Local Matching
All local neighbors have to be matched simultaneously. Greedy approach does not work when conflicts occur
These can solved by finding the alignment through optimization process such as by solving a string alignment problem
Example of alignment:
S (acbcdb) – (ac__bcdb)
T (cadbd) - (_cadb_d_)
Trivial solution requires exponential time
Each match is given a cost. Alignment solved through recurrence relation

55. The ‘Graphical’ View

56. Graphical Matching: Coupled BFS

57. Coupled BFS

58. Graphical Matching: Coupled BFS

59. Graphical Matching: Coupled BFS

60. Graphical Matching: Coupled BFS

61. Important Differences
Traditional Breadth First Search
Traversal Defined only over a single graph
All neighbors are considered for expanding the path
Coupled Breadth First Search
Traversal proceeds in two directed graphs simultaneously
Only ‘matched’ neighbors are considered for expanding the path
Constant number of neighbors provides a bound for the traversal complexity

62. Experimental Evaluation
800 prints from FVC2002(DB1)
2800 genuine tests,4950 impostor tests
Compared with BOZORTH 3
Error Rates
BOZORTH3: 3.6% EER, 5.0% FMR100
Proposed: 1.5% EER, 1.65% FMR100

63. Software CUBS Truthing Tool CUBS Minutiae Truthing Tool
CUBS Fingerprint Verification Demo
Matlab code for Fingerprint Enhancement
Matlab Toolbox for Fingerprint Verification
1179 downloads since 01/30
637 downloads since 03/22

64. Conclusion Contributions
New Fingerprint Image Enhancement using STFT Analysis.
Simultaneously estimates all intrinsic images
Increases recognition rate of existing matchers
New Feature Extraction Algorithm using Chain code Contour
Obviates need for thinning
Performs favorably with NIST feature extractor
New Graph based matching algorithm
Robust to non linear distortion
Formal technique for propagating local matches
Performs better than NIST BOZORTH3 matcher over FVC DB1 database

65. Acknowledgements Tsai Yang Jea (Alan) Chaohang Wu Sergey Tulyakov
Faisal Farooq
Amit Mhatre
Karthik Sridharan
Sankalp Nayak
Rest of the research group at CUBS

66. Thank You