Fusing Binary Templates for Multi-biometric Cryptosystems Guangcan Mai, Meng-Hui Lim, Pong C. Yuen Hong Kong Baptist University BTAS 2015 Sept. 10, 2015 Good afternoon, every one. I am MAI Guangcan from Hong Kong Baptist University, a Ph.D student supervised by Dr. Meng Hui Lim and Prof. PC Yuen. Today I am glad to introduce our research titled “Fusing Binary Templates for Multi-biometric Cryptosystems”. September 16, 2018 Hong Kong Baptist university

Biometric System Security There is no doubt that the security of a biometric system is very critical because what they manage for us is essential of our daily life, such as banking, passport information. Typically, the biometric data is stored/processed in plaintext in the system. Once the system storage is compromised by some hackers, the biometric data is exposed. Then, the spoofs such as face mask, fake fingerprint could be created to access the biometric systems as a target user. Eventually, our money, privacy data and even the id information will be stolen. Therefore, we have to do something to protect the biometric data that stored/processed in the system. So that we can prevent the creation of the fake biometric and protect our money, privacy data and id information. money privacy data id information September 16, 2018 Hong Kong Baptist university

Template Protection Scheme Feature transformation One-way transformation of enrollment template Biometric cryptosystem Generate a secure sketch from enrollment template for recognition Hybrid approach Feature transformation followed by biometric cryptosystem To protect the biometric data in the system, several template protection schemes have been proposed. First, the feature transformation perform a one-way transformation on the enrollment template. Second, the biometric cryptosystem generates a secure sketch from enrollment template for recognition. Third, the hybrid approach adopt a biometric cryptosystem to secure the transformation of the enrollment template. All of these three approaches are promising and we choose the biometric cryptosystem to secure the biometric data in our system. September 16, 2018 Hong Kong Baptist university

Multi-biometric Cryptosystem The most arguably famous: fuzzy extractor, fuzzy commitment Accepts binary only Unified feature representation Embedding Algorithm 1 Embedding Algorithm 2 Embedding Algorithm 3 Real-valued Binary Higher accuracy, Biometric Cryptosystem Binary Feature Fusion Larger population, Point-set Harder to be spoofed, Real-valued Point-set Binary … Convert to binary is required if not in binary Multi-biometric leverages information from multiple traits and generally have advantages such as higher matching accuracy, larger population coverage and much more harder to be spoofed, which have been widely deployed in many critical applications, such as immigration clearance, e-business. To secure the multi-biometric data, we could first perform a feature fusion and then adopt the biometric cryptosystem. As the features of different traits typically have different representations, for example, the real-valued for face, point-set for fingerprint and binary for iris, embedding algorithm need to be employed to convert features of different representation to a united representation before fusion. Considering that we convert the binary feature to other types, let say, real-valued or point-set, however, when features are extracted using a commercial black-box feature extractor, there is no binarization parameters available and hence they cannot be converted to other types properly. Furthermore, the most arguably famous biometric cryptosystems such as fuzzy commitment, fuzzy extractor accept binary input only. When adopting them, the fused features have to be converted to binary when they are not. All in all, it is better to convert features of different types into binary and then perform a binary feature fusion. Therefore, in this work, we focus on binary feature fusion for multi-biometric cryptosystem. Commercial “black-box” feature extractor No binalization parameters September 16, 2018 Hong Kong Baptist University

Criteria for Binary Feature Fusion Discriminability Small intra-user variations of feature bits Large inter-user variations of feature bits Security (high-entropy) Low dependency among bits High uniformity of feature bits Privacy No information leakage from helper data To specify the research problem here, we have defined some of the criteria for (fused) binary feature in a feature-level fusion based system. First, discriminability, the essential requirement for a recognition system, which require each feature bit to have small intra-user variations and large inter-user variations. Second, security, one of the critical requirement for a biometric system, which require each feature bit to have high uniformity and low dependency among feature bits. Third, privacy, the helper data that is used for binary feature fusion should have no information leakage on the biometric templates. In this research, we propose a binary feature fusion method that produce a discriminative fused binary feature with high entropy, where the helper data we use is non-user-specific and have no information leakage for a target user in the system that is based on the proposed binary feature fusion method. September 16, 2018 Hong Kong Baptist university

Hong Kong Baptist university Related Work Concatenation Concatenate features of multiple traits together Discriminability could be limited Treat features from different modalities equally Multimodal features have different discrimination power Security is typically not considered Iris + face (Sutcu et al. CVPR2007, Kanade et al. CVPRW2010) Left iris + right iris (Kanade et al. BTAS2009) Bit selection Select features bit by some measures Difficult to select discriminative bits with high entropy Highly discriminative bits are very likely to be mutual dependent Z-score (Kelkboom et al. BTAS2009) Discriminability (Nagar et al. TIFS2012) State of the art research on binary feature fusion can be categorized in concatenation and bit selection. Concatenated features of multiple traits might have a limited discriminability when multimodal features have different discrimination power. This is because it treats features of different modalities equally. Furthermore, there is typically no consideration on security. %Examples can be found in CVPR2007, CVPRW2010 and BTAS2009. Bit selection select feature bits by some measures. It is difficult to select discriminative bits with high entropy, because highly discriminative bits are very likely to be mutual dependent. This will lead to a serve tradeoff between discriminability and security. %Examples of bit selection can be found in BTAS2009 and TIFS2012. September 16, 2018 Hong Kong Baptist university

The Proposed Binary Feature Fusion Stage one: dependency-reductive bit grouping Dependency among bits (security) Stage two: discriminative within-group fusion Bit-uniformity (security), intra-user variations (discriminability), inter-user variations (discriminability) dependency reductive bit-grouping Grouping information Fusion function The proposed binary feature level fusion approach consist of two stages, that is, stage one, dependency-reductive bit grouping and stage two, discriminative within-group fusion. In stage one, we try to reduce the dependency among the fused bits, which is one of the sub-criteria for security. In stage two, we try to maximize the bit-uniformity, another sub-criteria for security, and optimize the discriminability, that is, minimize the intra-user variations and maximize the inter-user variations. Given a multimodal sample, we will first extract its binary features, where features of different modalities are initially concatenated together. Then we use the bit-grouping to group the strongly dependent bits together and separate the weakly/non-dependent bits After that, we will fuse each bit-group to a bit with maximizing the bit-uniformity and discriminability. Eventually, we concatenate the bits fused from each group and produce the fused binary feature, which is expected to be discriminative and high-entropy. To achieve dependency reductive bit-grouping and discriminative within-group fusion, the optimal grouping information and the within-group fusion have to been sought in the training phase. discriminative within-group fusion Discriminative binary feature with high entropy September 16, 2018 Hong Kong Baptist university

The Proposed Binary Feature Fusion Training Testing grouping information 𝐶 = 𝑐 1 , ⋯ 𝑐 𝐿 grouping information 𝐶 = 𝑐 1 , ⋯ 𝑐 𝐿 dependency reductive bit-grouping 1 𝑓 1 Let me tell you how we actually do. We first extract binary features from all of the multimodal training samples. Let each circle in the red rectangle denotes a feature bit, to obtain the grouping information, we then compute dependency between any two feature bits and use our proposed average mutual information based hierarchical clustering to perform a dependency reductive bit-grouping. The grouping information hat C is given after grouping. After that, we use the grouping information and the extracted binary features to perform the grouping and then obtain a set of training samples for each group. Let each of the triangle in the big circle denotes a possible pattern in a group, taking this three-bit group as an example, we first find out all of its eight possible pattern and then search for a discriminative fusion function that could optimize the genuine bit-error probability, impostor bit-error probability and the bit uniformity of the fused bit, where the patterns over the function will be fused to ‘0’ and the remains are fused ‘1’. All of the groups should be processed accordingly and are skipped here. Eventually, after the training phase, we have already obtained the grouping information and the fusion function for each group. In the testing phase, given a multimodal sample, we first extract the multimodal features, then use the grouping information to perform the dependency reductive bit-grouping. Finally, we fuse each group to a bit and then concatenate the fused bits to produce the output binary feature. 𝑓 𝐿 within-group fusion function discriminative within-group fusion September 16, 2018 Hong Kong Baptist university

Experiments Evaluation Experimental setting Discriminability ( Area under ROC curve) Security (average Renyi entropy, Hidano et al. BIOSIG2012) Experimental setting Multimodal Database WVU Chimeric A (FVC2000DB2 + FERET + CASIA) Chimeric B (FVC2002DB2 + FRGC + ICE2006) Subjects 106 100 Training Sample 3 4 Testing Sample 2 We have evaluated the discriminability and the security of the system that using the proposed binary feature fusion on three multimodal database. The first one is WVU multimodal dataset consist of 106 subjects, each of which have three training samples and two testing samples. The remaining two are chimeric dataset that consist of 100 subjects , respectively, where each subject have four training samples and four testing samples. September 16, 2018 Hong Kong Baptist university

Hong Kong Baptist university Experimental Results Discriminability Security Area Under ROC Curve XOR (a) WVU DISC (b) Chimeric A DISC (c) Chimeric B DISC (d) WVU SEC (e) Chimeric A SEC (f) Chimeric B SEC Average Renyi Entropy The experiments on discriminability and security are shown on figures (a) – (c) and figures (d) – (f), respectively. All of the six figures are obtained by varying the feature length from 150 to 600, where the length of concatenation is 3000 in our experiment and shown as a horizontal line. In the results of discriminability, figures (a) – (c), the y-axis denotes the area under ROC curve. The higher the curve, the more discriminative the feature/scheme. Our approach is plotted in black with cross. The comparison includes bit-selection and concatenation. The uni-biometric feature and the bit-wise operation based approaches are plotted as baseline. We find that the proposed method is comparable with bit-selection and concatenation. This is because the discriminability of these two methods and the best uni-biometric feature are too high to have space to be improved. In the results of security, figures (d) to (f), the y-axis denotes the average Renyi entropy, which measures entropy per feature bit, with maximum one. The higher the curve, the higher the security that the feature/scheme can provide. We find that the proposed method outperform all other methods excluding the XOR-fusion rule. This is because that the fused bit of the XOR-fusion rule is optimized for the bit-uniformity but without any guarantee on discriminability. Connecting to the results of discriminability, we can find that the discriminability of the XOR-fusion rule is almost the lowest!!! September 16, 2018 Hong Kong Baptist university

Conclusion and Future Work A binary feature fusion approach has been proposed Dependency-reductive bit grouping Discriminative within-group fusion Experimental results on three multi-modal databases Discriminative fused features with high entropy are produced (Future work) tradeoff between discriminability and security of the cryptosystem that adopt the proposed fusion approach need to be further studied. September 16, 2018 Hong Kong Baptist university

Hong Kong Baptist university Thank You September 16, 2018 Hong Kong Baptist university