1. Information Fusion in Multibiometric Systems
Md. Maruf Monwar
Computer Science
University of Calgary
2008

2. Outline Biometric and Multibiometric Systems
Issues Involved in Multibiometric Systems Design
Information Fusion
A Sample Multibiometric System
Conclusion
4/16/2017

3. Biometric System Biometric system
An automatic pattern recognition system that recognizes a person by determining the authenticity of a specific biological and/or behavioral characteristic (biometric) possessed by that person
Physiological biometric identifiers:
- Fingerprints,
- Hand geometry,
- Ear patterns
- Eye patterns (iris and retina),
- Facial features
and other physical characteristics.
Behavioral identifiers:
- Voice,
- Signature
- Typing patterns
and others.
4/16/2017

4. Biometric System Application of Biometric Systems
Physical access control of, for example, an airport. Here the airport infrastructure, or travel infrastructure in general, is the application.
Logical access control of, for example, a bank account; i.e., the application is the access to and the handling of money.
Ensuring uniqueness of individuals. Here the focus is typically on preventing double enrollment in some application, for example, a social benefits program.
4/16/2017

5. Operational Modes of a Biometric System
Verification Mode
One-to-One transaction
The user effectively claims an identity by providing
some information which is typically used to call up a
reference number from a database.
Identification Mode
One-to-Many transaction
The user’s information is compared against a database
of reference templates and the user’s identity determined
as a match against one of these templates
4/16/2017

6. Multibiometric System
Most biometric systems deployed in real-world applications are monomodal, i.e. only one source of information is used for authentication.
These systems often face numerous limitations, such as, susceptibility of the result to quality of the sample, its orientation/rotation and distortion, noise, intra-class variability, non-distinctiveness, non-universality, and others.
Some of the limitations imposed by monomodal biometric systems can be overcome by including multiple sources of information for establishing identity. Such systems are known as Multibiometric Systems.
4/16/2017

7. Advantages of Multibiometric Systems
Multibiometric systems can offer substantial
improvement in the matching accuracy.
Reduced FAR and FRR; Increase feature space and hence capacity of identification system
Multibiometric addresses the issue of non-
universality or insufficient population coverage.
Achieved a certain degree of flexibility; If a dry finger prevent user from enrolling ……..
Multibiometric system makes the life of any
impostor harder.
Facilitates challenge-response mechanism by asking the user to present a random subset of traits;
Can be used in single biometric system also
4/16/2017

8. Advantages of Multibiometric Systems
Multibiometric systems can effectively address the problem of noisy data.
In case of poor voice, face can be used for verification
Multibiometric systems can be effectively used in
tracking or continuous monitoring system where
only one trait is not sufficient.
A person can be monitored by face and gait patterns, but in crowded place or for distance from camera, this is not possible simultaneously
A multibiometric system may also be viewed as a
fault tolerant system.
Continues to operate even when certain source becomes unreliable
4/16/2017

9. Single and Multibiometric Systems
4/16/2017

10. Design Issues of Multibiometric Systems
Cost benefits
Tradeoff between the added cost and the improvement in matching
Determining sources of biometric information
Which of the sources are relevant to the application at hand?
Acquisition and processing sequence
Simultaneously or one-by-one; Processing can also be done one after another or simultaneously
Type of information used
What type of information is to be fused?
True or virtual multimodal database
4/16/2017

11. Design Issues of Multibiometric Systems
Fusion methodologies adopted
Sum rule, maximum, minimum, Borda count
Operational Mode
Identification or verification
Assigning weights to Biometric
Weights need to be assigned or not?
Multimodal Database
True or virtual multimodal database
4/16/2017

12. Sources of Multiple Evidences
- Multi-sensor systems (single biometric trait-multiple sensors)
2D texture content using CCD and 3D surface shape using range camera
4/16/2017

13. Sources of Multiple Evidences
- Multi-algorithm systems (single biometric trait-multiple algorithm)
4/16/2017

14. Sources of Multiple Evidences
- Multi-instance systems (multiple instances of same body traits)
For dry skin, one fingerprint might not work
4/16/2017

15. Sources of Multiple Evidences
- Multi-sample systems (multiple samples of same biometric traits)
A small size sensor may acquire multiple dab prints of an individual’s finger and later done mosaicing
4/16/2017

16. Sources of Multiple Evidences
- Multimodal systems (multiple biometric traits)
4/16/2017

17. Sources of Multiple Evidences
- Hybrid systems
Two speaker recognition algorithm combined with three face recognition algorithm
4/16/2017

18. Fusion
The goal of Fusion is to determine the best set of experts in a given problem domain and devise an appropriate function that can optimally combine the decisions rendered by the individual experts.
Two subcategories:
Prior to matching
- Sensor level fusion
- Feature level fusion
After Matching
- Rank level fusion
- Match score level fusion
- Decision level fusion
4/16/2017

19. Prior to Matching Fusion
Sensor level
The raw data acquired from multiple sensors can be processed and integrated to generate new data from which features can be extracted.
For example, in the case of face biometrics, both 2D texture information and 3D depth (range) information (obtained using two different sensors) may be fused to generate a 3D texture image of the face which could then be subjected to feature extraction and matching.
Feature level
The feature sets extracted from multiple data sources can be fused to create a new feature set to represent the individual.
The geometric features of the hand, for example, may be augmented with the eigen-coefficients of the face in order to construct a new high-dimension feature vector. A feature selection/transformation procedure may be adopted to elicit a minimal feature set from the high-dimensional feature vector .
4/16/2017

20. After Matching Fusion Match Score Level Rank level
Multiple classifiers output a set of match scores which are fused to generate a single scalar score.
As an example, the match scores generated by the face and hand modalities of a user may be combined via the simple sum rule in order to obtain a new match score which is then used to make the final decision.
Rank level
This type of fusion is relevant in identification systems where each classifier associates a rank with every enrolled identity (a higher rank indicating a good match).
Thus, fusion entails consolidating the multiple ranks associated with an identity and determining a new rank that would aid in establishing the final decision. Techniques such as the Borda count may be used to make the final decision
4/16/2017

21. After Matching Fusion Decision level
When each matcher outputs its own class label (i.e., accept or reject in a verification system, or the identity of a user in an identification system), a single class label can be obtained by employing techniques such as majority voting or AND/OR rules.
4/16/2017

22. Various Fusion Level Possibilities
4/16/2017

23. Some Multibiometric Systems
4/16/2017

24. Example Multibiometric System
Developed by Ross and Jain in 2003
Used face, fingerprint and hand geometry.
Match score level fusion method is used to consolidate the information from these three monomodal experts.
4/16/2017

25. Example Multibiometric System
4/16/2017

26. Example Multibiometric System
Face Verification: Eigenface technique is used.
Fingerprint Verification: Minutiae matching technique is used.
Hand Geometry Verification: Simple distance measure (between test and enrolled hand geometry feature vectors) technique is used.
4/16/2017

27. Minutiae Matching
4/16/2017

28. Example Multibiometric System
Used virtual multimodal database – face and fingerprint from the same subjects, but hand geometry from different users.
Mutual non-independence of the biometric indicators allows to assign the biometric data of one user to another.
4/16/2017

29. Example Multibiometric System
Match Score level Fusion:
The database consisted of matching scores obtained from three different modalities––face, fingerprint and hand geometry.
This data was used to generate 1000 ( 50x scores from 5 faces, 5 scores from 5 fingerprints and 10 scores from hand geometry) genuine scores and (50x10x49 - each feature set of a user was compared against one feature set each of all other users, 5 match score selected from either face and fingerprint scores and 5 match scores selected from 10 hand geometry scores) impostor scores.
4/16/2017

30. Example Multibiometric System
Match Score level Fusion:
All scores were mapped to the range [0, 100].
Since the face and hand scores were not similarity scores (they were distance scores), they were converted to similarity scores by simply subtracting them from 100.
A score vector represents the scores of multiple classifiers.
The vector (x1, x2, x3) is a score vector, where x1, x2 and x3 correspond to the (similarity) scores obtained from the classifiers corresponding to the face, fingerprint and hand geometry systems, respectively.
4/16/2017

31. Example Multibiometric System
Sum Rule:
The simplest form of combination would be to take the weighted average of the scores from the
multiple modalities. This strategy was applied to all
possible combinations of the three modalities.
Equal weights were assigned to each modality as
the bias of each classifier was not computed.
4/16/2017

32. Match Score Fusion Example
4/16/2017

33. Example Multibiometric System
Experiment Results
4/16/2017

34. Example Multibiometric System
Experiment Results
4/16/2017

35. Example Multibiometric System
Experiment Results
4/16/2017

36. Example Multibiometric System
Experiment Results
Their experiments suggest that the sum rule
performs better than the decision tree.
The FAR of the tree classifier is 0.036% and the
FRR is 9.63%. The sum rule that combines all
three scores has a corresponding FAR of 0.03%
and a FRR of 1.78%.
4/16/2017

37. Conclusion Advantages of multibiometric systems over single
biometric systems and the issues that need to be
considered during designing such systems have been
discussed.
Information fusion plays a key role in a multibiometric
system and the success of such system depends heavily
on fusion methods.
Initial results obtained on a match score level fusion based
multibiometric system proposed by Ross and Jain
in 2003 that uses face, fingerprint and hand geometry
features for biometric verification purposes.
4/16/2017

38. One Other Important Issue
Score Normalization
Table. Genuine acceptance rate (GAR) (%) of different normalization
and fusion techniques at the 0.1% false acceptance rate (FAR)
4/16/2017

39. Some References
[1] Arun A. Ross, K. Nandakumar and Anil K. Jain, “Handbook of Multibiometrics”, Springer, NY, USA,
[2] L. Hong and A. K. Jain, “Integrating faces and fingerprints for personal identification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp –1307, 1998.
[3] A. K. Jain, S. Prabhakar, and S. Chen, “Combining multiple matchers for high security fingerprint verification system, ”Pattern Recognition Letters, vol. 20, no , pp –1379, 1999.
[4] R. Frischholz, U. Dieckmann, “BioID: A Multimodal Biometric Identification System”, In IEEE Computer, vol. 33, no. 2, 2000,
[5] J. Fierrez-Aguilar, J. Ortega-Garcia, D. Garcia-Romero, and J. Gonzalez-Rodriguez, “A comparative evaluation of fusion strategies for multimodal biometric verification,” in Proceedings of 4th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 830–837.
[6] A. Kumar, D. C. Wong, H. C. Shen, and A. K. Jain, “Personal verification using palmprint and hand geometry biometric,” in Proceedings of 4th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS 2688, 2003, pp. 668 – 678.

40. Thank You