1. Deeply learned face representations are sparse, selective, and robust
Authors: Yi Sun1 Xiaogang Wang2;3 Xiaoou Tang 1;3
1Department of Information Engineering, The Chinese University of Hong Kong
2Department of Electronic Engineering, The Chinese University of Hong Kong
3Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
Course:CSC8860 Seminar on Computer Vision and Pattern Recognition
Instructor: Ming Dong
Presenter: Lu Wang

2. Contents Introduction DeepID2 Methods
Comparison of DeepID2 and DeepID2+
Performance
Conclusion

3. Introduction-LDA
Each face is represented by a large number of pixel values. Linear discriminant analysis is primarily used here to reduce the number of features to a more manageable number before classification. Each of the new dimensions is a linear combination of pixel values, which form a template. The linear combinations obtained using Fisher's linear discriminant are called Fisher faces, while those obtained using the related principal component analysis are called eigenfaces.

4. Introduction- Bayesian face
Direct visual matching of images for the purposes of face recognition and image retrieval, using a probabilistic measure of similarity, based primarily on a Bayesian (MAP) analysis of image differences. The performance advantage of this probabilistic matching technique over standard Euclidean nearest 􏱒neighbor eigenface matching.

5. Introduction- Unified Subspace
Decomposes face difference into three components, intrinsic difference, transformation difference, and noise. Use the face difference model and a detailed subspace analysis on the three components to develop a unified framework for subspace analysis. Construct of a 3D parameter space that uses three subspace dimensions as axis.

6. Introduction- DeepID2(Deep IDentification-verification features)
Problem/Challenge
Develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences.
Solution
Deep learning with using both face identification and verification signals as supervision.

7. Introduction- DeepID2 Source data
LFW (Labeled Faces in the Wild):a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13,000 images of faces collected from the web.
Each face has been labeled with the name of the person pictured of the people pictured have two or more distinct photos in the data set. The only constraint on these faces is that they were detected by the Viola-Jones face detector.

8. Introduction- DeepID2
Identification-verification guided deep feature learning

9. Introduction- DeepID2

10. Introduction- DeepID2

11. Introduction- DeepID2 Face Verification
SDM algorithm to detect 21 facial landmarks
Joint Bayesian model for face verification based on the extracted DeepID2:

12. Introduction- DeepID2
Forward-backward greedy algorithm to select a small number of effective and complementary DeepID2 vectors (25 in experiment), which saves most of the feature extraction time during test.

13. Introduction- DeepID2
Balancing the identification and verification signals

14. Introduction- DeepID2
Balancing the identification and verification signals

15. Introduction- DeepID2
Rich identity information improves feature learning

16. Introduction- DeepID2
Investigating the verification signals

17. Introduction- DeepID2
Final system and comparison with other methods

18. Methods-Comparison of DeepID2 and DeepID2+
DeepID2+ nets are inherited from DeepID2 nets, which have four convolutional layers, with 20 , 40 , 60 ,and 80 feature maps, followed by a 160 -dimensional feature layer fully-connected to both the third and fourth convolutional layers. The 160 -dimensional feature layer (DeepID2 feature layer) is supervised by both face identification and verification tasks.

19. Methods-Comparison of DeepID2 and DeepID2+
Compared to DeepID2, DeepID2+ makes three improvements.
First, it is larger with 128 feature maps in each of the four convolutional layers. The final feature representation is increased from 160 to 512 dimensions.
Second, our training data is enlarged by merging the CelebFaces+ dataset,the WDRef dataset [6], and some newly collected identities exclusive from LFW. The larger DeepID2+ net is trained with around 290; 000 face images from 12; 000 identities compared to 160; 000 images from 8; 000 identities used to train the DeepID2 net.
Third, in the DeepID2 net, supervisory signals are only added to one fully-connected layer connected to the third and fourth convolutional layers, while the lower convolutional layers can only get supervision with gradients back-propagated from higher layers.

20. Performance
High-performance of DeepID2+ nets

21. Performance
High-performance of DeepID2+ nets

22. Performance
High-performance of DeepID2+ nets

23. Performance
Moderate sparsity of neural activations

24. Performance
Neural activation distribution

25. Performance
Neural activation distribution

26. Performance
Neural activation distribution

27. Performance
Neural activation distribution

28. Performance
Robustness of DeepID2+ features

29. Performance
Robustness of DeepID2+ features

30. Performance
Robustness of DeepID2+ features

31. Conclusion
This paper designs a high-performance DeepID2+ net which sets new sate-of-the-art on LFW and YouTube Faces for both face identification and verification.
This work not only significantly advances the face recognition performance, but also provides valuable insight to help people to understand deep learning and its connection with many existing computer vision researches such as sparse representation, attribute learning and occlusion handling.

32. Thank you for listening!