1. Fusing Binary Templates for Multi-biometric Cryptosystems
Guangcan Mai, Meng-Hui Lim, Pong C. Yuen
Hong Kong Baptist University
BTAS 2015
Sept. 10, 2015
Good afternoon, every one. I am MAI Guangcan from Hong Kong Baptist University, a Ph.D student supervised by Dr. Meng Hui Lim and Prof. PC Yuen.
Today I am glad to introduce our research titled “Fusing Binary Templates for Multi-biometric Cryptosystems”.
September 16, 2018
Hong Kong Baptist university

2. Biometric System Security
There is no doubt that the security of a biometric system is very critical because what they manage for us is essential of our daily life, such as banking, passport information.
Typically, the biometric data is stored/processed in plaintext in the system.
Once the system storage is compromised by some hackers, the biometric data is exposed.
Then, the spoofs such as face mask, fake fingerprint could be created to access the biometric systems as a target user.
Eventually, our money, privacy data and even the id information will be stolen.
Therefore, we have to do something to protect the biometric data that stored/processed in the system.
So that we can prevent the creation of the fake biometric and protect our money, privacy data and id information.
money
privacy data
id information
September 16, 2018
Hong Kong Baptist university

3. Template Protection Scheme
Feature transformation
One-way transformation of enrollment template
Biometric cryptosystem
Generate a secure sketch from enrollment template for recognition
Hybrid approach
Feature transformation followed by biometric cryptosystem
To protect the biometric data in the system, several template protection schemes have been proposed.
First, the feature transformation perform a one-way transformation on the enrollment template.
Second, the biometric cryptosystem generates a secure sketch from enrollment template for recognition.
Third, the hybrid approach adopt a biometric cryptosystem to secure the transformation of the enrollment template.
All of these three approaches are promising and we choose the biometric cryptosystem to secure the biometric data in our system.
September 16, 2018
Hong Kong Baptist university

4. Multi-biometric Cryptosystem
The most arguably famous:
fuzzy extractor, fuzzy commitment
Accepts binary only
Unified feature representation
Embedding Algorithm 1
Embedding Algorithm 2
Embedding Algorithm 3
Real-valued
Binary
Higher accuracy,
Biometric Cryptosystem
Binary
Feature Fusion
Larger population,
Point-set
Harder to be spoofed,
Real-valued
Point-set
Binary …
Convert to binary is required if not in binary
Multi-biometric leverages information from multiple traits and generally have advantages such as higher matching accuracy, larger population coverage and much more harder to be spoofed, which have been widely deployed in many critical applications, such as immigration clearance, e-business.
To secure the multi-biometric data, we could first perform a feature fusion and then adopt the biometric cryptosystem.
As the features of different traits typically have different representations, for example, the real-valued for face, point-set for fingerprint and binary for iris, embedding algorithm need to be employed to convert features of different representation to a united representation before fusion.
Considering that we convert the binary feature to other types, let say, real-valued or point-set, however, when features are extracted using a commercial black-box feature extractor, there is no binarization parameters available and hence they cannot be converted to other types properly.
Furthermore, the most arguably famous biometric cryptosystems such as fuzzy commitment, fuzzy extractor accept binary input only.
When adopting them, the fused features have to be converted to binary when they are not.
All in all, it is better to convert features of different types into binary and then perform a binary feature fusion.
Therefore, in this work, we focus on binary feature fusion for multi-biometric cryptosystem.
Commercial “black-box” feature extractor
No binalization parameters
September 16, 2018
Hong Kong Baptist University

5. Criteria for Binary Feature Fusion
Discriminability
Small intra-user variations of feature bits
Large inter-user variations of feature bits
Security (high-entropy)
Low dependency among bits
High uniformity of feature bits
Privacy
No information leakage from helper data
To specify the research problem here, we have defined some of the criteria for (fused) binary feature in a feature-level fusion based system.
First, discriminability, the essential requirement for a recognition system, which require each feature bit to have small intra-user variations and large inter-user variations.
Second, security, one of the critical requirement for a biometric system, which require each feature bit to have high uniformity and low dependency among feature bits.
Third, privacy, the helper data that is used for binary feature fusion should have no information leakage on the biometric templates.
In this research, we propose a binary feature fusion method that produce a discriminative fused binary feature with high entropy, where the helper data we use is non-user-specific and have no information leakage for a target user in the system that is based on the proposed binary feature fusion method.
September 16, 2018
Hong Kong Baptist university

6. Hong Kong Baptist university
Related Work
Concatenation
Concatenate features of multiple traits together
Discriminability could be limited
Treat features from different modalities equally
Multimodal features have different discrimination power
Security is typically not considered
Iris + face (Sutcu et al. CVPR2007, Kanade et al. CVPRW2010)
Left iris + right iris (Kanade et al. BTAS2009)
Bit selection
Select features bit by some measures
Difficult to select discriminative bits with high entropy
Highly discriminative bits are very likely to be mutual dependent
Z-score (Kelkboom et al. BTAS2009)
Discriminability (Nagar et al. TIFS2012)
State of the art research on binary feature fusion can be categorized in concatenation and bit selection.
Concatenated features of multiple traits might have a limited discriminability when multimodal features have different discrimination power.
This is because it treats features of different modalities equally.
Furthermore, there is typically no consideration on security.
%Examples can be found in CVPR2007, CVPRW2010 and BTAS2009.
Bit selection select feature bits by some measures.
It is difficult to select discriminative bits with high entropy, because highly discriminative bits are very likely to be mutual dependent.
This will lead to a serve tradeoff between discriminability and security.
%Examples of bit selection can be found in BTAS2009 and TIFS2012.
September 16, 2018
Hong Kong Baptist university

7. The Proposed Binary Feature Fusion
Stage one: dependency-reductive bit grouping
Dependency among bits (security)
Stage two: discriminative within-group fusion
Bit-uniformity (security), intra-user variations (discriminability), inter-user variations (discriminability)
dependency reductive bit-grouping
Grouping information
Fusion function
The proposed binary feature level fusion approach consist of two stages, that is, stage one, dependency-reductive bit grouping and stage two, discriminative within-group fusion.
In stage one, we try to reduce the dependency among the fused bits, which is one of the sub-criteria for security.
In stage two, we try to maximize the bit-uniformity, another sub-criteria for security, and optimize the discriminability, that is, minimize the intra-user variations and maximize the inter-user variations.
Given a multimodal sample, we will first extract its binary features, where features of different modalities are initially concatenated together.
Then we use the bit-grouping to group the strongly dependent bits together and separate the weakly/non-dependent bits
After that, we will fuse each bit-group to a bit with maximizing the bit-uniformity and discriminability.
Eventually, we concatenate the bits fused from each group and produce the fused binary feature, which is expected to be discriminative and high-entropy.
To achieve dependency reductive bit-grouping and discriminative within-group fusion, the optimal grouping information and the within-group fusion have to been sought in the training phase.
discriminative within-group fusion
Discriminative
binary feature
with high entropy
September 16, 2018
Hong Kong Baptist university

8. The Proposed Binary Feature Fusion
Training
Testing
grouping information
𝐶 = 𝑐 1 , ⋯ 𝑐 𝐿
grouping information
𝐶 = 𝑐 1 , ⋯ 𝑐 𝐿
dependency reductive bit-grouping 1
𝑓 1
Let me tell you how we actually do.
We first extract binary features from all of the multimodal training samples.
Let each circle in the red rectangle denotes a feature bit, to obtain the grouping information, we then compute dependency between any two feature bits and use our proposed average mutual information based hierarchical clustering to perform a dependency reductive bit-grouping.
The grouping information hat C is given after grouping.
After that, we use the grouping information and the extracted binary features to perform the grouping and then obtain a set of training samples for each group.
Let each of the triangle in the big circle denotes a possible pattern in a group, taking this three-bit group as an example, we first find out all of its eight possible pattern and then search for a discriminative fusion function that could optimize the genuine bit-error probability, impostor bit-error probability and the bit uniformity of the fused bit, where the patterns over the function will be fused to ‘0’ and the remains are fused ‘1’.
All of the groups should be processed accordingly and are skipped here.
Eventually, after the training phase, we have already obtained the grouping information and the fusion function for each group.
In the testing phase, given a multimodal sample, we first extract the multimodal features, then use the grouping information to perform the dependency reductive bit-grouping.
Finally, we fuse each group to a bit and then concatenate the fused bits to produce the output binary feature.
𝑓 𝐿
within-group
fusion function
discriminative within-group fusion
September 16, 2018
Hong Kong Baptist university

9. Experiments Evaluation Experimental setting
Discriminability ( Area under ROC curve)
Security (average Renyi entropy, Hidano et al. BIOSIG2012)
Experimental setting
Multimodal Database
WVU
Chimeric A (FVC2000DB2 + FERET + CASIA)
Chimeric B (FVC2002DB2 + FRGC + ICE2006)
Subjects
106
100
Training Sample 3 4
Testing Sample 2
We have evaluated the discriminability and the security of the system that using the proposed binary feature fusion on three multimodal database.
The first one is WVU multimodal dataset consist of 106 subjects, each of which have three training samples and two testing samples.
The remaining two are chimeric dataset that consist of 100 subjects , respectively, where each subject have four training samples and four testing samples.
September 16, 2018
Hong Kong Baptist university

10. Hong Kong Baptist university
Experimental Results
Discriminability Security
Area Under ROC Curve
XOR
(a) WVU DISC
(b) Chimeric A DISC
(c) Chimeric B DISC
(d) WVU SEC
(e) Chimeric A SEC
(f) Chimeric B SEC
Average Renyi Entropy
The experiments on discriminability and security are shown on figures (a) – (c) and figures (d) – (f), respectively.
All of the six figures are obtained by varying the feature length from 150 to 600, where the length of concatenation is 3000 in our experiment and shown as a horizontal line.
In the results of discriminability, figures (a) – (c), the y-axis denotes the area under ROC curve.
The higher the curve, the more discriminative the feature/scheme.
Our approach is plotted in black with cross.
The comparison includes bit-selection and concatenation.
The uni-biometric feature and the bit-wise operation based approaches are plotted as baseline.
We find that the proposed method is comparable with bit-selection and concatenation.
This is because the discriminability of these two methods and the best uni-biometric feature are too high to have space to be improved.
In the results of security, figures (d) to (f), the y-axis denotes the average Renyi entropy, which measures entropy per feature bit, with maximum one.
The higher the curve, the higher the security that the feature/scheme can provide.
We find that the proposed method outperform all other methods excluding the XOR-fusion rule.
This is because that the fused bit of the XOR-fusion rule is optimized for the bit-uniformity but without any guarantee on discriminability.
Connecting to the results of discriminability, we can find that the discriminability of the XOR-fusion rule is almost the lowest!!!
September 16, 2018
Hong Kong Baptist university

11. Conclusion and Future Work
A binary feature fusion approach has been proposed
Dependency-reductive bit grouping
Discriminative within-group fusion
Experimental results on three multi-modal databases
Discriminative fused features with high entropy are produced
(Future work) tradeoff between discriminability and security of the cryptosystem that adopt the proposed fusion approach need to be further studied.
September 16, 2018
Hong Kong Baptist university

12. Hong Kong Baptist university
Thank You
September 16, 2018
Hong Kong Baptist university