ONLINE FINGERPRINT VERIFICATION Sharat Chikkerur Center for Unified Biometrics and Sensors University at Buffalo www.cubs.buffalo.edu Advisor: A. N. Cartwright Committee: V. Govindaraju, A. H. Titus, L. Kondi

Abstract Background Challenges Contributions Traditional password/token based authentication schemes are insecure and are being replaced by biometric authentication mechanisms Fingerprints were one of the first biometrics to be widely used Despite 40 years of research, fingerprint recognition is still an open problem. Challenges Feature extraction is very unreliable in poor quality prints Matching fingerprints under non linear distortion is difficult Contributions New fingerprint image enhancement using STFT analysis. New feature extraction algorithm based on chain code contours New graph based matching algorithm robust to non linear distortion

Outline Introduction Biometrics Fingerprints 101 Fingerprint Image Enhancement Minutia Feature Extraction Matching Algorithm Conclusion Software Demos

Biometrics Definition Examples Physical Biometrics Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits Examples Physical Biometrics Fingerprint, Hand Geometry, Face Measurement Biometric Dependent on environment/interaction Behavioral Biometrics Handwriting, Signature, Speech, Gait Performance/Temporal biometric Dependent on state of mind Chemical/Biological Biometrics Skin spectroscopy DNA, blood-glucose Biometrics offers a promising solution for reliable and uniform identification and verification of an individual. Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits. Physical biometrics rely on physiological features such as fingerprints, hand geometry, iris pattern, facial features etc. for identity verification. Behavioral biometrics depends upon behavioral features such as speech patterns, handwriting, signature, walking gait etc. for authentication. These traits are unique to an individual and hence cannot be misused, lost or stolen. Physiological biometrics are more stable and robust than behavioral biometrics and a single sample is enough to obtain identifying information. Biometrics such as signature and speech are acquiredand learnt over time and also subject to the user’s state of mind and disposition. Multiple samples are required to acquire a stable representation of user features. Biometrics are based on established scientific principles as a basis for authentication.

Fingerprints as a Biometric High Universality A majority of the population (>96%) have legible fingerprints More than the number of people who possess passports, license and IDs High Distinctiveness Even identical twins have different fingerprints (most biometrics fail) Individuality of fingerprints established through empirical evidence High Permanence Fingerprints are formed in the fetal stage and remain structurally unchanged through out life. High Performance One of the most accurate forms of biometrics available Best trade off between convenience and security High Acceptability Fingerprint acquisition is non intrusive. Requires no training. The skin of the fingers is corrugated by a pattern of contours. These contour ridges are formed during foetal development and remain unchanged throughout a person’s life. It has been established that fingerprint patterns are unique to each individual. Of all the biometric techniques, fingerprint based authentication has been the most well established and well researched topic. It has been used in forensic science for authentication since the early 1900s. Fingerprints have been an important tool used by for law enforcement and forensics for over a century. Automatic Fingerprint Identification Systems (AFIS) can provide absolute identification of an individual by processing the image of a fingerprint. Fingerprints are formed while the foetus is 4 months old and remain unchanged through out an individual’s life time

Fingerprints 101: Fingerprint Classes A fingerprint is made up of system of oriented friction ridges A fingerprint can be classified based on type the ridge flow pattern The corrugated surface of the fingerprint is made up of ridges and valleys cover that the entire palmer surface of the hand. The flow pattern of these ridges and valleys are unique to each individual. These patterns that are used for identification and authentication. The image below shows the image of a fingerprint along with the distinguishing features on the print.   The flow of the ridges and patterns has been classified into 5 broad classes. This classification is used to catalog the fingerprints and also in authenticating two prints. Henry systems follow an elaborate classification scheme of cataloging and filing forensic prints. Fig1 shows the different classes of ridge flows. This methods cannot be used to distinguish between two fingerprints. Fig 2 shows the distinguishing features on the fingerprint. These features are discontinuities or anomalies in the normal flow of ridges on the surface of the finger. These features are termed as minutiae (small details). There are eighteen different types of minutiae. Fig1 shows the most commonly encountered ones and their names. Fig 2 shows a thumbprint captured on paper. These are typically the kind of images that forensic AFIS (Automatic Fingerprint Identification Systems) have to deal with. The quality of the print not only deteriorates during capture but also during storage and hence AFIS systems are more sophisticated than their biometric counterparts. Classification helps in narrowing down possible matches In reality, the class distribution is skewed (>65% are loops) Used only in law enforcement applications

Fingerprints 101: Ridge Characteristics Fingerprints can be distinguished based on the ridge characteristics Ridge characteristics mark local discontinuities in the ridge flow No two individuals have the same pattern of ridge characteristics at the same relative locations Local Features Global Features

Prior Related Work: Matching Paradigms Manual Human experts use a combination of visual, textural, minutiae cues and experience for verification Still used in the final stages of law enforcement applications Image based Utilizes only visual appearance. Requires the complete image to be stored (large template sizes) Texture based Treats the fingerprint as an oriented texture image Less accurate than minutiae based matchers since most regions in the fingerprints carry low textural content Minutiae based Uses the relative position of the minutiae points The most popular and accurate approach for verification Resembles manual approach very closely.

Image Based Matching: Optical Correlation Advantages Image itself is used as the template Requires only low resolution images Optical correlation makes it extremely fast (Choudary and Awwal ’99, Lee et al. 99, Roberge et al. 99, Baze et al.00) Disadvantages Image itself is used as the template (template size about 30 KB) Requires accurate alignment of the two prints (unreliable in poor prints) Not robust to changes in scale, orientation and position.

Texture Based Matching: Filterbanks Advantages Uses texture information (lost in optical and minutiae based schemes) Performs well with poor quality prints Features are statistically independent from minutiae and can be combined with minutiae matchers for higher accuracy (Jain et al. 00, Jain et al 01) Disadvantages Requires accurate alignment of the two prints (unreliable in poor prints) Not invariant to translation, orientation and non-linear distortion. Less Accurate than minutiae based matchers

Minutiae Based Matching Advantages Invariant to translation, rotation and scale changes Very accurate (Ratha et al 96, Jain et al. 97, Jian Yau 00, Bazen and Garez 03) Disadvantages Minutiae extraction is error prone is low quality images Not robust to non-linear distortion. Does not use visual and textural cues

General Architecture

Outline Introduction Fingerprint Image Enhancement Need for Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental Evaluation Minutia Feature Extraction Matching Algorithm Software Demos

Need for Enhancement What you see What you ‘think’ you see

Reality: What you usually get.. High contrast print Typical dry print Faint print The performance of any fingerprint recognizer depends upon the quality of the fingerprint image processed. AFIS systems have achieved high performances in case of reasonably good prints. However there is not yet satisfactory methods to deal with bad quality fingerprints as often encountered in forensic and biometric applications. Effective methodologies for cleaning the valleys between the ridge contours are lacking in current fingerprint recognition systems. The figures show some prints acquired under different conditions. It can be seen that the minutiae features cannot be easily extracted from the original gray scale image. Therefore some form on enhancement is required before processing the fingerprint image further. Low contrast print Typical Wet Print Creases

Challenges Challenges Fingerprint image is non stationary (has dominant local orientation and frequency) General purpose image processing algorithms are not useful Traditional operators and filters assume Gaussian noise model ‘Noise’ in fingerprint images consists mostly of ridge breaks Contextual Filters Existing techniques are based on ‘contextual’ filtering Filter parameters are adapted to each local neighborhood Filter parameters in ‘unrecoverable’ regions can be interpolated based on its neighbors

Prior Related Work: Spatial Filtering (Yang et.al 1996, Greenberg et. Al 1999) proposed local anisotropic filtering Filter kernel adapts at each pixel location Hong et al, 96/98 proposed the use of Gabor filters for enhancement Gabor filter has the best joint space-frequency localization Does not handle high curvature regions well due to block wise approach. Even Symmetric Kernel Fourier spectrum showing the localization

Prior Related Work: Fourier Domain Filtering Sherlock et al 94, proposed the use of Fourier domain filtering The image is convolved with a filter bank of directionally selective filters Image enhanced by selecting a linear combination of filter responses Watson et al. 94, proposed the use or ‘root filtering’ for enhancement.(Pseudo matched filter) Does not require the computation of orientation images Root Filtering Fourier Domain Filtering

Traditional Approaches Local Orientation (x,y) Gradient Method Enhancement Frequency/Spatial The fingerprint can be seen as an oriented texture. This property of the fingeprints can be used while enhancing the fingerprint image. Traditional image processing methods such as gaussian smoothening, or low pass filtering cannot be used as they tend to bridge the gaps between the ridges. The ridges have to be enhanced only in a direction parallel to their orientation. There are spatial and frequency domain methods that perform such kind of filtering. Spatial domain methods are based on anisotropic filters aligned in the ridge direction(??) or are based on Gabor filtering(Anil Jain et al). Frequency domain methods utilizing FFT (Grother et al, Monroe et al) are also present and are more successful. The enhancement depends upon the accurate estimation of the local ridge orientation and local ridge spacing within the image. Presently the local orientation and the local ridge spacing are obtained through separate algorithms. The enhancement is performed using this information. This approach requires multiple passes and is computationally expensive. We propose a unified approach to fingerprint image enhancement using FFT analysis. The proposed method extracts the local orientation and ridge spacing information in one pass and at the same time performs the enhancement. Local Ridge Spacing F(x,y) Projection Based Method [Ratha et al 95]

Proposed Approach: Overview Region Mask STFT Analysis Frequency Image Fourier domain Enhancement The proposed methods uses FFT based frequency domain analysis to estimate the ridge spacing and orientation in the image. The enhancement is also done in the frequency domain. The analysis yields the energy map, orientation map and ridge spacing map corresponding to the image. The energy map indicates the presence of ridges and their contrast within the image. The energy map can be successfully used to segment the fingerprint image from the background. The orientation map provides the orientation of the ridges in a local neighborhood. This information is used in the enhancement procedure. The ridge spacing map provides the inter ridge distance variation in the image. The ridge spacing can be used in the enhancement procedure to design a suitable band pass filter that allows ridges and eliminates noise and gray level gradients across the image. The following slides show the results of the FFT analysis algorithm. Orientation Image Coherence Image

STFT Analysis Fingerprint image is non stationary, so we require both space and frequency resolution: time frequency analysis STFT in 1D STFT in 2D

Surface Wave Model Fingerprint ridges can be modeled as an oriented wave Local ridge orientation Local ridge frequency Surface wave Local Neighborhoods Validity of the model

Parameter Estimation Paradigm: The Fourier domain response can be viewed as a distribution of surface waves. Each term F(r, θ) corresponds to a surface wave of frequency 1/r and orientation θ We seek to find the most likely surface wave and hence estimate the dominant direction and frequency We can represent the Fourier spectrum in polar form as F(r,θ) The power spectrum is reduced to a joint probability density function using The angular and frequency densities are given by marginal density functions The enhancement techniques can be divided into two distinct classes (I) Spatial domain techniques and (ii) Frequency domain based techniques Spatial domain techniques: This technique involves convolving a filter kernel(mask) with the entire image to obtain the filtered image. Standard spatial domain filters such as isotropic gaussian filters and other low pass filters cannot be applied to fingerprint images as they blur the image in all directions. Fingerprint image is a highly oriented structure. The spatial filter that is used for enhancement should blur or average the pixels in the direction of the ridge at the same time increasing the contrast between the ridges and valleys in a direction perpendicular to the ridge. In general, spatial domain filters are computationally less expensive compared to frequency domain based filters. Frequency domain filters are based on enhancement in the Fourier domain. The most common approaches include the block level filtering as proposed by Grother, Candela et al. In this method the Fourier transform of the block is multiplied by its power spectrum raised to a fractional power. This is similar to matched filtering approach found in communication and signal processing systems. Other approaches such as Morno et al. are based on directionally selective filters. The image is passed through a filter bank where each filter is oriented in a given direction. The resulting image is obtained by combining the images obtained from each of the directional filter. In general, Fourier based methods are computationally expensive but yield better results when compared to spatial filters. Most of the existing methods in fingerprint image enhancement are based on ‘Contextual filtering’, where the filter parameters adapt to the local ridge orientation and frequency. Anisotropic filters are a special case of spatial contextual filters. Unlike standard gaussian kernel, the anisotropic filter kernel is not uniform in all directions. It is designed to smooth selectively in the direction of the ridge. The shape of the kernel adapts by using the local ridge orientation and ridge frequency. The advantage of this approach is that it creates far fewer spurious ridges and artifacts compared to Fourier based methods and is more robust to noise. Gabor filters can be applied in both spatial and frequency domain. Unlike other directional filters, Gabor filters are both directionally and frequency selective filters. It has been shown that the Gabor kernel has the optimal space frequency resolution. In this approach, the image is divided into blocks and the orientation and ridge frequency is estimated in each block. The block is then filtered using a Gabor kernel that has the corresponding orientation and frequency. The final image is obtained by combining the filtered block images.

Ridge Orientation Image

Region Mask The surface wave approximation does not hold in the background region The region mask is obtained by simple thresholding of the block energy image

Frequency Image [Jain et al 00]

Coherence Image Block processing is unreliable in regions of high curvature Sherlock and Monro 94, relax filter parameters near the singular locations Estimation of singular point is difficult in poor images! We use an angular coherence measure proposed by Rao and Jain 90

Enhancement The slide shows the results of the frequency domain enhancement. Presently the enhancement approach suggested by Grother et al is used to perform the enhancement. The frequency and orientation map obtained in the previous section is not included in the filter design. The results will only improve with their inclusion.

Additional Enhancement Results

Qualitative Comparison Root Filtering Original Image

Qualitative Comparison(cont.) Gabor Filter based Enhancement Proposed Approach

Objective Evaluation We evaluated the effect of enhancement on 800 images from FVC2002 DB3 The evaluation consists of 2800 genuine test and 4950 impostor tests It can be seen that the matcher performance improves with enhancement

Outline Introduction Fingerprint Image Enhancement Minutia Feature Extraction Prior Related Work Chain code contour Experimental Evaluation Matching Algorithm Conclusion Software Demos

Background Minutiae represent local discontinuities in ridge flow Minutiae features are the most widely used fingerprint representation There are several standards such as CBEFF (file format) and ANSI-NIST (interchange format) standards for minutiae based fingerprint representation Minutiae extraction approaches may be broadly categorized into Binarization based approaches Direct gray scale extraction

Prior Related Work Binarization Approaches MINDTCT,NIST NFIS, (Garris et. Al, 02) Directionally adaptive binarization Template matching is used to detect minutiae Adaptive Flow Orientation technique (Ratha et. al., 95) Binarization is performed by peak detection Peak detection leads to false positives in regions of poor ridge constrast. Direct Gray Scale Ridge Following Ridge Following (Maio and Maltoni 97, Jiang and Yau 01) Based on ridge pursuit Has low computational complexity. Cannot handle poor contrast prints and images with poor ridge structure. Relies on a good orientation map for ridge pursuit The enhancement techniques can be divided into two distinct classes (I) Spatial domain techniques and (ii) Frequency domain based techniques Spatial domain techniques: This technique involves convolving a filter kernel(mask) with the entire image to obtain the filtered image. Standard spatial domain filters such as isotropic gaussian filters and other low pass filters cannot be applied to fingerprint images as they blur the image in all directions. Fingerprint image is a highly oriented structure. The spatial filter that is used for enhancement should blur or average the pixels in the direction of the ridge at the same time increasing the contrast between the ridges and valleys in a direction perpendicular to the ridge. In general, spatial domain filters are computationally less expensive compared to frequency domain based filters. Frequency domain filters are based on enhancement in the Fourier domain. The most common approaches include the block level filtering as proposed by Grother, Candela et al. In this method the Fourier transform of the block is multiplied by its power spectrum raised to a fractional power. This is similar to matched filtering approach found in communication and signal processing systems. Other approaches such as Morno et al. are based on directionally selective filters. The image is passed through a filter bank where each filter is oriented in a given direction. The resulting image is obtained by combining the images obtained from each of the directional filter. In general, Fourier based methods are computationally expensive but yield better results when compared to spatial filters. Most of the existing methods in fingerprint image enhancement are based on ‘Contextual filtering’, where the filter parameters adapt to the local ridge orientation and frequency. Anisotropic filters are a special case of spatial contextual filters. Unlike standard gaussian kernel, the anisotropic filter kernel is not uniform in all directions. It is designed to smooth selectively in the direction of the ridge. The shape of the kernel adapts by using the local ridge orientation and ridge frequency. The advantage of this approach is that it creates far fewer spurious ridges and artifacts compared to Fourier based methods and is more robust to noise. Gabor filters can be applied in both spatial and frequency domain. Unlike other directional filters, Gabor filters are both directionally and frequency selective filters. It has been shown that the Gabor kernel has the optimal space frequency resolution. In this approach, the image is divided into blocks and the orientation and ridge frequency is estimated in each block. The block is then filtered using a Gabor kernel that has the corresponding orientation and frequency. The final image is obtained by combining the filtered block images.

Binarization Method Acquisition Binarization Thinning Minutia Detection

Proposed Approach: Chain Code Contours Provides a lossless description of the contour and also gives direction and curvature information. Translation and rotation invariant Used in computer vision for encoding object boundaries Used for character recognition (Madhavanth et. al 99)

Minutiae Detection using Chain Codes Minutiae are encountered as points of ‘significant’ turn on the contour Left turn: Ridge ending Right turn: Bifurcation

Determining Turn Points

Results

Results (cont.)

Experimental Evaluation Test Data 150 prints from FVC2002(DB1) were randomly selected for evaluation. Ground truth was established using a semi automated truthing tool. Results compared using NIST NFIS open source software. Metrics Proposed by Sherlock et. Al 94 Sensitivity: Ability of the algorithm to detect true minutiae Specificity : Ability of the algorithm to avoid false positives Flipped : Minutiae whose type has been exchanged

Quantitative Analysis : Results Examples File Name NIST Proposed method Actual TP FP M F 10_8.tif 18 16 8 2 1 17 11_6.tif 50 40 4 10 41 9 12_8.tif 29 22 5 7 3 13_6.tif 35 28 14_6.tif 44 34 12 6 37 13 15_7.tif 38 16_7.tif 36 17_6.tif 43 11 18_8.tif 31 32 19_7.tif 26

Sensitivity distribution Results Summary results Count TP(NIST) > proposed : 40 of 150 Count E(NIST) < proposed : 40 of 150 Metric NIST Proposed Sensitivity(%) 82.8 83.5 Specificity(%) 77.2 76.8 Flipped(%) 12.0 10.9 Sensitivity distribution Overall statistics

Outline Introduction Fingerprint Image Enhancement Minutia Feature Extraction Matching Algorithm Prior Related Work New Representation: K-plet Local Matching: Dynamic Programming Consolidation: Coupled BFS Experimental Evaluation Conclusion Software Demos

Minutiae Based Matching Challenges Minutiae extraction is error prone is low quality images Not robust to non-linear distortion. Intra-user variation

Challenges: Non-linear Distortion

Challenges: Quality and Intra-user variance Variation in quality Intra-user variation

Prior Related Work :Global Matching Point correspondences not known : combinatorial problem Relaxation approach (Ranade and Rosenfield 93) Likelihood of each match is either decreased or increased at each iteration based on compatibility of rest of the points Iterative approach makes it too slow to be practical Generalized Hough Transform (Ratha et al. 96) All possible transformation represented as a quantized search space Searches for the most optimal transform in the search space Very fast Ridge Alignment (Jain et al. 97) Performs explicit alignment before matching Each minutiae is associated with its ridge (represented by a curve) The alignment is based on ridge correspondence Global matching is then performed using string edit distance

Prior Related Work: Local Matching (Jiang and Yau 00) 11 dimensional local features derived from reference minutiae and two closest neighbors Best match is used only for explicit alignment (Jea and Govindaraju 04) 5 dimesional features Si (ri0, ri1, φi0, φi1, δi) derived from two closest neighbors Alignment is still required (Ratha et al. 00) ‘Star’ representation derived from all minutiae within a particular radius Consolidation by checking consistency (Garris et. al 03: BOZORTH3) Line features Consolidation by linking consisting matches Instead of the 3-elements minutiae representation, we use this 5-elements secondary feature representation. For a given feature point or called central minutia, we find the two nearest neighboring minutiae of it. The secondary feature contains the information of the distances between the central minutia to the neighboring minutiae, the orientation between them and the angle between this two line segments. The secondary feature is derived only from the widely-used minutiae representation, thus our algorithm can be easily adapted other systems. We do not use the ridge count information, because it is not required information in minutiae representation. We do not use the minutiae types as the feature, because they are not reliable. Here is an example, these two feature points are extracted from the same finger but in different impressions. One is interpreted as a bifurcation while the other one is seen as a ridge ending. Secondary feature also has some other advantages. It is a pure localized feature. It doesn’t rely on any global landmarks. It is orientation invariant. There is no pre-alignment stage needed.

Proposed Algorithm Representation K-Plet Features invariant to rotation and translation Local relationship formally represented by a directed graph Local Matching Posed as a string alignment problem and solved by dynamic programming Matches all neighbors simultaneously Consolidation Coupled Breadth First Search Breadth first search is used to propagate the matches Similar to human verification No explicit alignment required at any stage

Neighborhood Representation: K-plet

K-plet Θ r Φ

Local Matching All local neighbors have to be matched simultaneously. Greedy approach does not work when conflicts occur These can solved by finding the alignment through optimization process such as by solving a string alignment problem Example of alignment: S (acbcdb) – (ac__bcdb) T (cadbd) - (_cadb_d_) Trivial solution requires exponential time Each match is given a cost. Alignment solved through recurrence relation

The ‘Graphical’ View

Graphical Matching: Coupled BFS

Coupled BFS

Graphical Matching: Coupled BFS

Graphical Matching: Coupled BFS

Graphical Matching: Coupled BFS

Important Differences Traditional Breadth First Search Traversal Defined only over a single graph All neighbors are considered for expanding the path Coupled Breadth First Search Traversal proceeds in two directed graphs simultaneously Only ‘matched’ neighbors are considered for expanding the path Constant number of neighbors provides a bound for the traversal complexity

Experimental Evaluation 800 prints from FVC2002(DB1) 2800 genuine tests,4950 impostor tests Compared with BOZORTH 3 Error Rates BOZORTH3: 3.6% EER, 5.0% FMR100 Proposed: 1.5% EER, 1.65% FMR100

Software CUBS Truthing Tool CUBS Minutiae Truthing Tool CUBS Fingerprint Verification Demo Matlab code for Fingerprint Enhancement Matlab Toolbox for Fingerprint Verification http://www.mathworks.com/matlabcentral 1179 downloads since 01/30 637 downloads since 03/22

Conclusion Contributions New Fingerprint Image Enhancement using STFT Analysis. Simultaneously estimates all intrinsic images Increases recognition rate of existing matchers New Feature Extraction Algorithm using Chain code Contour Obviates need for thinning Performs favorably with NIST feature extractor New Graph based matching algorithm Robust to non linear distortion Formal technique for propagating local matches Performs better than NIST BOZORTH3 matcher over FVC DB1 database

Acknowledgements Tsai Yang Jea (Alan) Chaohang Wu Sergey Tulyakov Faisal Farooq Amit Mhatre Karthik Sridharan Sankalp Nayak Rest of the research group at CUBS

Thank You http://www.cubs.buffalo.edu