Deeply learned face representations are sparse, selective, and robust Authors: Yi Sun1 Xiaogang Wang2;3 Xiaoou Tang 1;3 1Department of Information Engineering, The Chinese University of Hong Kong 2Department of Electronic Engineering, The Chinese University of Hong Kong 3Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Course:CSC8860 Seminar on Computer Vision and Pattern Recognition Instructor: Ming Dong Presenter: Lu Wang

Contents Introduction DeepID2 Methods Comparison of DeepID2 and DeepID2+ Performance Conclusion

Introduction-LDA Each face is represented by a large number of pixel values. Linear discriminant analysis is primarily used here to reduce the number of features to a more manageable number before classification. Each of the new dimensions is a linear combination of pixel values, which form a template. The linear combinations obtained using Fisher's linear discriminant are called Fisher faces, while those obtained using the related principal component analysis are called eigenfaces.

Introduction- Bayesian face Direct visual matching of images for the purposes of face recognition and image retrieval, using a probabilistic measure of similarity, based primarily on a Bayesian (MAP) analysis of image differences. The performance advantage of this probabilistic matching technique over standard Euclidean nearest 􏱒neighbor eigenface matching.

Introduction- Unified Subspace Decomposes face difference into three components, intrinsic difference, transformation difference, and noise. Use the face difference model and a detailed subspace analysis on the three components to develop a unified framework for subspace analysis. Construct of a 3D parameter space that uses three subspace dimensions as axis.

Introduction- DeepID2(Deep IDentification-verification features) Problem/Challenge Develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. Solution Deep learning with using both face identification and verification signals as supervision.

Introduction- DeepID2 Source data LFW (Labeled Faces in the Wild):a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13,000 images of faces collected from the web. Each face has been labeled with the name of the person pictured. 1680 of the people pictured have two or more distinct photos in the data set. The only constraint on these faces is that they were detected by the Viola-Jones face detector.

Introduction- DeepID2 Identification-verification guided deep feature learning

Introduction- DeepID2

Introduction- DeepID2

Introduction- DeepID2 Face Verification SDM algorithm to detect 21 facial landmarks Joint Bayesian model for face verification based on the extracted DeepID2:

Introduction- DeepID2 Forward-backward greedy algorithm to select a small number of effective and complementary DeepID2 vectors (25 in experiment), which saves most of the feature extraction time during test.

Introduction- DeepID2 Balancing the identification and verification signals

Introduction- DeepID2 Balancing the identification and verification signals

Introduction- DeepID2 Rich identity information improves feature learning

Introduction- DeepID2 Investigating the verification signals

Introduction- DeepID2 Final system and comparison with other methods

Methods-Comparison of DeepID2 and DeepID2+ DeepID2+ nets are inherited from DeepID2 nets, which have four convolutional layers, with 20 , 40 , 60 ,and 80 feature maps, followed by a 160 -dimensional feature layer fully-connected to both the third and fourth convolutional layers. The 160 -dimensional feature layer (DeepID2 feature layer) is supervised by both face identification and verification tasks.

Methods-Comparison of DeepID2 and DeepID2+ Compared to DeepID2, DeepID2+ makes three improvements. First, it is larger with 128 feature maps in each of the four convolutional layers. The final feature representation is increased from 160 to 512 dimensions. Second, our training data is enlarged by merging the CelebFaces+ dataset,the WDRef dataset [6], and some newly collected identities exclusive from LFW. The larger DeepID2+ net is trained with around 290; 000 face images from 12; 000 identities compared to 160; 000 images from 8; 000 identities used to train the DeepID2 net. Third, in the DeepID2 net, supervisory signals are only added to one fully-connected layer connected to the third and fourth convolutional layers, while the lower convolutional layers can only get supervision with gradients back-propagated from higher layers.

Performance High-performance of DeepID2+ nets

Performance High-performance of DeepID2+ nets

Performance High-performance of DeepID2+ nets

Performance Moderate sparsity of neural activations

Performance Neural activation distribution

Performance Neural activation distribution

Performance Neural activation distribution

Performance Neural activation distribution

Performance Robustness of DeepID2+ features

Performance Robustness of DeepID2+ features

Performance Robustness of DeepID2+ features

Conclusion This paper designs a high-performance DeepID2+ net which sets new sate-of-the-art on LFW and YouTube Faces for both face identification and verification. This work not only significantly advances the face recognition performance, but also provides valuable insight to help people to understand deep learning and its connection with many existing computer vision researches such as sparse representation, attribute learning and occlusion handling.

Thank you for listening!