Information Fusion in Multibiometric Systems Md. Maruf Monwar Computer Science University of Calgary 2008

Outline Biometric and Multibiometric Systems Issues Involved in Multibiometric Systems Design Information Fusion A Sample Multibiometric System Conclusion 4/16/2017

Biometric System Biometric system An automatic pattern recognition system that recognizes a person by determining the authenticity of a specific biological and/or behavioral characteristic (biometric) possessed by that person Physiological biometric identifiers: - Fingerprints, - Hand geometry, - Ear patterns - Eye patterns (iris and retina), - Facial features and other physical characteristics. Behavioral identifiers: - Voice, - Signature - Typing patterns and others. 4/16/2017

Biometric System Application of Biometric Systems Physical access control of, for example, an airport. Here the airport infrastructure, or travel infrastructure in general, is the application. Logical access control of, for example, a bank account; i.e., the application is the access to and the handling of money. Ensuring uniqueness of individuals. Here the focus is typically on preventing double enrollment in some application, for example, a social benefits program. 4/16/2017

Operational Modes of a Biometric System Verification Mode One-to-One transaction The user effectively claims an identity by providing some information which is typically used to call up a reference number from a database. Identification Mode One-to-Many transaction The user’s information is compared against a database of reference templates and the user’s identity determined as a match against one of these templates 4/16/2017

Multibiometric System Most biometric systems deployed in real-world applications are monomodal, i.e. only one source of information is used for authentication. These systems often face numerous limitations, such as, susceptibility of the result to quality of the sample, its orientation/rotation and distortion, noise, intra-class variability, non-distinctiveness, non-universality, and others. Some of the limitations imposed by monomodal biometric systems can be overcome by including multiple sources of information for establishing identity. Such systems are known as Multibiometric Systems. 4/16/2017

Advantages of Multibiometric Systems Multibiometric systems can offer substantial improvement in the matching accuracy. Reduced FAR and FRR; Increase feature space and hence capacity of identification system Multibiometric addresses the issue of non- universality or insufficient population coverage. Achieved a certain degree of flexibility; If a dry finger prevent user from enrolling …….. Multibiometric system makes the life of any impostor harder. Facilitates challenge-response mechanism by asking the user to present a random subset of traits; Can be used in single biometric system also 4/16/2017

Advantages of Multibiometric Systems Multibiometric systems can effectively address the problem of noisy data. In case of poor voice, face can be used for verification Multibiometric systems can be effectively used in tracking or continuous monitoring system where only one trait is not sufficient. A person can be monitored by face and gait patterns, but in crowded place or for distance from camera, this is not possible simultaneously A multibiometric system may also be viewed as a fault tolerant system. Continues to operate even when certain source becomes unreliable 4/16/2017

Single and Multibiometric Systems 4/16/2017

Design Issues of Multibiometric Systems Cost benefits Tradeoff between the added cost and the improvement in matching Determining sources of biometric information Which of the sources are relevant to the application at hand? Acquisition and processing sequence Simultaneously or one-by-one; Processing can also be done one after another or simultaneously Type of information used What type of information is to be fused? True or virtual multimodal database 4/16/2017

Design Issues of Multibiometric Systems Fusion methodologies adopted Sum rule, maximum, minimum, Borda count Operational Mode Identification or verification Assigning weights to Biometric Weights need to be assigned or not? Multimodal Database True or virtual multimodal database 4/16/2017

Sources of Multiple Evidences - Multi-sensor systems (single biometric trait-multiple sensors) 2D texture content using CCD and 3D surface shape using range camera 4/16/2017

Sources of Multiple Evidences - Multi-algorithm systems (single biometric trait-multiple algorithm) 4/16/2017

Sources of Multiple Evidences - Multi-instance systems (multiple instances of same body traits) For dry skin, one fingerprint might not work 4/16/2017

Sources of Multiple Evidences - Multi-sample systems (multiple samples of same biometric traits) A small size sensor may acquire multiple dab prints of an individual’s finger and later done mosaicing 4/16/2017

Sources of Multiple Evidences - Multimodal systems (multiple biometric traits) 4/16/2017

Sources of Multiple Evidences - Hybrid systems Two speaker recognition algorithm combined with three face recognition algorithm 4/16/2017

Fusion The goal of Fusion is to determine the best set of experts in a given problem domain and devise an appropriate function that can optimally combine the decisions rendered by the individual experts. Two subcategories: Prior to matching - Sensor level fusion - Feature level fusion After Matching - Rank level fusion - Match score level fusion - Decision level fusion 4/16/2017

Prior to Matching Fusion Sensor level The raw data acquired from multiple sensors can be processed and integrated to generate new data from which features can be extracted. For example, in the case of face biometrics, both 2D texture information and 3D depth (range) information (obtained using two different sensors) may be fused to generate a 3D texture image of the face which could then be subjected to feature extraction and matching. Feature level The feature sets extracted from multiple data sources can be fused to create a new feature set to represent the individual. The geometric features of the hand, for example, may be augmented with the eigen-coefficients of the face in order to construct a new high-dimension feature vector. A feature selection/transformation procedure may be adopted to elicit a minimal feature set from the high-dimensional feature vector . 4/16/2017

After Matching Fusion Match Score Level Rank level Multiple classifiers output a set of match scores which are fused to generate a single scalar score. As an example, the match scores generated by the face and hand modalities of a user may be combined via the simple sum rule in order to obtain a new match score which is then used to make the final decision. Rank level This type of fusion is relevant in identification systems where each classifier associates a rank with every enrolled identity (a higher rank indicating a good match). Thus, fusion entails consolidating the multiple ranks associated with an identity and determining a new rank that would aid in establishing the final decision. Techniques such as the Borda count may be used to make the final decision 4/16/2017

After Matching Fusion Decision level When each matcher outputs its own class label (i.e., accept or reject in a verification system, or the identity of a user in an identification system), a single class label can be obtained by employing techniques such as majority voting or AND/OR rules. 4/16/2017

Various Fusion Level Possibilities 4/16/2017

Some Multibiometric Systems 4/16/2017

Example Multibiometric System Developed by Ross and Jain in 2003 Used face, fingerprint and hand geometry. Match score level fusion method is used to consolidate the information from these three monomodal experts. 4/16/2017

Example Multibiometric System 4/16/2017

Example Multibiometric System Face Verification: Eigenface technique is used. Fingerprint Verification: Minutiae matching technique is used. Hand Geometry Verification: Simple distance measure (between test and enrolled hand geometry feature vectors) technique is used. 4/16/2017

Minutiae Matching 4/16/2017

Example Multibiometric System Used virtual multimodal database – face and fingerprint from the same subjects, but hand geometry from different users. Mutual non-independence of the biometric indicators allows to assign the biometric data of one user to another. 4/16/2017

Example Multibiometric System Match Score level Fusion: The database consisted of matching scores obtained from three different modalities––face, fingerprint and hand geometry. This data was used to generate 1000 ( 50x20 - 5 scores from 5 faces, 5 scores from 5 fingerprints and 10 scores from hand geometry) genuine scores and 24500 (50x10x49 - each feature set of a user was compared against one feature set each of all other users, 5 match score selected from either face and fingerprint scores and 5 match scores selected from 10 hand geometry scores) impostor scores. 4/16/2017

Example Multibiometric System Match Score level Fusion: All scores were mapped to the range [0, 100]. Since the face and hand scores were not similarity scores (they were distance scores), they were converted to similarity scores by simply subtracting them from 100. A score vector represents the scores of multiple classifiers. The vector (x1, x2, x3) is a score vector, where x1, x2 and x3 correspond to the (similarity) scores obtained from the classifiers corresponding to the face, fingerprint and hand geometry systems, respectively. 4/16/2017

Example Multibiometric System Sum Rule: The simplest form of combination would be to take the weighted average of the scores from the multiple modalities. This strategy was applied to all possible combinations of the three modalities. Equal weights were assigned to each modality as the bias of each classifier was not computed. 4/16/2017

Match Score Fusion Example 4/16/2017

Example Multibiometric System Experiment Results 4/16/2017

Example Multibiometric System Experiment Results 4/16/2017

Example Multibiometric System Experiment Results 4/16/2017

Example Multibiometric System Experiment Results Their experiments suggest that the sum rule performs better than the decision tree. The FAR of the tree classifier is 0.036% and the FRR is 9.63%. The sum rule that combines all three scores has a corresponding FAR of 0.03% and a FRR of 1.78%. 4/16/2017

Conclusion Advantages of multibiometric systems over single biometric systems and the issues that need to be considered during designing such systems have been discussed. Information fusion plays a key role in a multibiometric system and the success of such system depends heavily on fusion methods. Initial results obtained on a match score level fusion based multibiometric system proposed by Ross and Jain in 2003 that uses face, fingerprint and hand geometry features for biometric verification purposes. 4/16/2017

One Other Important Issue Score Normalization Table. Genuine acceptance rate (GAR) (%) of different normalization and fusion techniques at the 0.1% false acceptance rate (FAR) 4/16/2017

Some References [1] Arun A. Ross, K. Nandakumar and Anil K. Jain, “Handbook of Multibiometrics”, Springer, NY, USA, 2006. [2] L. Hong and A. K. Jain, “Integrating faces and fingerprints for personal identification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1295–1307, 1998. [3] A. K. Jain, S. Prabhakar, and S. Chen, “Combining multiple matchers for high security fingerprint verification system, ”Pattern Recognition Letters, vol. 20, no. 11-13, pp. 1371–1379, 1999. [4] R. Frischholz, U. Dieckmann, “BioID: A Multimodal Biometric Identification System”, In IEEE Computer, vol. 33, no. 2, 2000, 64-68. [5] J. Fierrez-Aguilar, J. Ortega-Garcia, D. Garcia-Romero, and J. Gonzalez-Rodriguez, “A comparative evaluation of fusion strategies for multimodal biometric verification,” in Proceedings of 4th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 830–837. [6] A. Kumar, D. C. Wong, H. C. Shen, and A. K. Jain, “Personal verification using palmprint and hand geometry biometric,” in Proceedings of 4th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 668 – 678.

Thank You