CS 534 Spring 2019 Machine Vision Showcase

Slides:



Advertisements
Similar presentations
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Advertisements

Convolutional Neural Networks
Face detection State-of-the-art face detection demo (Courtesy Boris Babenko)Boris Babenko.
ImageNet Classification with Deep Convolutional Neural Networks
Face Recognition and Feature Subspaces
Principal Component Analysis
Computer Vision I Instructor: Prof. Ko Nishino. Today How do we recognize objects in images?
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Methods in Leading Face Verification Algorithms
MACHINE LEARNING AND ARTIFICIAL NEURAL NETWORKS FOR FACE VERIFICATION
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
Recognition Part II Ali Farhadi CSE 455.
Face Recognition and Feature Subspaces
Face Recognition and Feature Subspaces
Deep face recognition Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman.
80 million tiny images: a large dataset for non-parametric object and scene recognition CS 4763 Multimedia Systems Spring 2008.
Object Recognition in Images Slides originally created by Bernd Heisele.
Face Image-Based Gender Recognition Using Complex-Valued Neural Network Instructor :Dr. Dong-Chul Kim Indrani Gorripati.
GENDER AND AGE RECOGNITION FOR VIDEO ANALYTICS SOLUTION PRESENTED BY: SUBHASH REDDY JOLAPURAM.
Learning Features and Parts for Fine-Grained Recognition Authors: Jonathan Krause, Timnit Gebru, Jia Deng, Li-Jia Li, Li Fei-Fei ICPR, 2014 Presented by:
2D-LDA: A statistical linear discriminant analysis for image matrix
Deep Learning Overview Sources: workshop-tutorial-final.pdf
Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces Speaker: Po-Kai Shen Advisor: Tsai-Rong Chang Date: 2010/6/14.
Things iPhoto thinks are faces
Recent developments in object detection
A Discriminative Feature Learning Approach for Deep Face Recognition
Deeply learned face representations are sparse, selective, and robust
an introduction to: Deep Learning
From Vision to Grasping: Adapting Visual Networks
Convolutional Neural Fabrics by Shreyas Saxena, Jakob Verbeek
Recognition with Expression Variations
Article Review Todd Hricik.
Regularizing Face Verification Nets To Discrete-Valued Pain Regression
Combining CNN with RNN for scene labeling (segmentation)
Face Recognition and Feature Subspaces
Hybrid Features based Gender Classification
Recognition: Face Recognition
Intelligent Information System Lab
Classification of Hand-Written Digits Using Scattering Convolutional Network Dongmian Zou Advisor: Professor Radu Balan.
Training Techniques for Deep Neural Networks
Final Year Project Presentation --- Magic Paint Face
Object Recognition in the Dynamic Link Architecture
Unsupervised Face Alignment by Robust Nonrigid Mapping
FaceNet A Unified Embedding for Face Recognition and Clustering
R-CNN region By Ilia Iofedov 11/11/2018 BGU, DNN course 2016.
State-of-the-art face recognition systems
Announcements Project 1 artifact winners
In summary C1={skin} C2={~skin} Given x=[R,G,B], is it skin or ~skin?
By: Kevin Yu Ph.D. in Computer Engineering
Deep Face Recognition Omkar M. Parkhi Andrea Vedaldi Andrew Zisserman
Introduction to Neural Networks
Face Recognition and Detection Using Eigenfaces
Outline Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no.
Domingo Mery Department of Computer Science
Introduction of MATRIX CAPSULES WITH EM ROUTING
Convolutional neural networks Abin - Roozgard.
[Figure taken from googleblog
A Proposal Defense On Deep Residual Network For Face Recognition Presented By SAGAR MISHRA MECE
ONE shot learning for recognition
Lecture: Deep Convolutional Neural Networks
CS4670: Intro to Computer Vision
Announcements Project 2 artifacts Project 3 due Thursday night
Announcements Project 4 out today Project 2 winners help session today
Where are we? We have covered: Project 1b was due today
An introduction to: Deep Learning aka or related to Deep Neural Networks Deep Structural Learning Deep Belief Networks etc,
Domingo Mery Department of Computer Science
CS295: Modern Systems: Application Case Study Neural Network Accelerator Sang-Woo Jun Spring 2019 Many slides adapted from Hyoukjun Kwon‘s Gatech “Designing.
Learning and Memorization
The “Margaret Thatcher Illusion”, by Peter Thompson
Deep learning: Recurrent Neural Networks CV192
Presentation transcript:

CS 534 Spring 2019 Machine Vision Showcase by Colleen Finnegan, Alec Kohlhoff, Sean McGovern, Yash Shukla, and Huaming Sun Showcasing work by: Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf on Facebook’s DeepFace Send description of presentation to Professor Include slide numbers

References Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014. G. B. Huang, M. Ramesh, T. Berg, and E. Learned-miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In ECCV Workshop on Faces in Real-life Images, 2008. L. Wolf, T. Hassner, and I. Maoz. Face recognition in unconstrained videos with matched background similarity. In CVPR, 2011. Browne, R. (2019). IBM hopes 1 million faces will help fight bias in facial recognition. [online] CNBC. Available at: https://www.cnbc.com/2019/01/29/ibm-releases-diverse-dataset-to-fight-facial-recognition-bias.html [Accessed 30 Mar. 2019]. J. Buolamwini and T. Gebru, “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification,” Proceedings of Machine Learning Research, vol. 81, pp. 1–15, 2018. F.A. Bhat & M.A. Wani, “Performance Comparison of Major Classical Face Recognition Techniques,” International Conference on Machine Learning and Applications, 2004. Alex K. , IIya Sutskever, Geoffrey H. , “ImageNet Classification with Deep Convolutional Neural Networks”, Communication of the ACM, Vol. 60, Issue 6, pp. 82-83

Agenda DeepFace Edge Detection 3-D Face Modeling Deep Neural Networks Deep Neural Networks in DeepFace Siamese Network Questions

Classical facial recognition methods Principal Component Analysis (PCA): Decompose face image into uncorrelated components called Eigenfaces. Extract features by thresholding. Linear Discriminant Analysis (LDA): Minimize Within Class Scatter Matrix and Maximize Between Class Scatter Matrix. Project (test image-mean image) onto face space and Calculate Eucledian distance. Discrete Cosine Transform (DCT): Compute DCT of image f(x,y) In most images, much signal energy lies at low frequencies. (X): Face image from ORL database, (Y):DCT transformed image from X , (Z) : The top left rectangle (low frequency) carries maximum information Discuss a few key previous methods used in facial recognition Image source: Performance Comparison of Major Classical Face Recognition Techniques [6]

Classical facial recognition methods Independent Component Analysis (ICA): Outputs independent vectors that are linear combinations of image. Less sensitive to pose variations and more to lighting conditions. Variation in: 1. Facial expressions 2. Lighting conditions 3. Poses From top to bottom: Variation in: Facial expressions, Lighting conditions, Poses Image source: Performance Comparison of Major Classical Face Recognition Techniques [6]

Brief History of Convolutional Neural Networks CNNs have been around since the 1980s (So why so new?) Vast recent enhancements in GPU processing Access to large, labeled visual datasets (ImageNet, Pascal, etc.) ImageNet Large Scale Visual Recognition Competition (2012) Team from University of Toronto used a breakthrough CNN to achieve a low error of 17% Nicknamed AlexNet (designed by Alex Krizhevsky) CNNs have been used ever since Source: https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

DeepFace Introduction Facebook’s facial recognition algorithm Introduced in 2014 Deep Learning Framework instead of well engineered features 9-layer deep neural network Uses a Siamese network Contributions added to face recognition: Affective facial alignment system based on explicit 3D modeling of faces Advanced the state of the art significantly Leverages a very large labeled dataset of faces to obtain a face representation

Datasets Social Face Classification (SFC) -> Training Collection of photos from Facebook 4.4 million labeled faces; 4030 people each with 800 to 1200 faces Most recent 5% images of each identity are left out for testing Identities do not intersect with any of the identities with below datasets Labeled Faces in the Wild (LFW) -> Testing 13,323 web photos of 5,749 celebrities Youtube Faces (YTF) -> Testing 3,425 YouTube videos of 1,595 subjects (a subset of the celebrities in the LFW) Source: Taigman, Y. et al. In Proc. Conference on Computer Vision and Pattern Recognition 1701–1708 (2014). Discuss data sets used in training and evaluating the deep face algorithm. Remove explanations of data sets from other slides

Face Alignment pipeline: 3D Modeling Detecting 6 fiducial points in the 2D detection cropped image (a). Scale, rotate and translate image to anchor locations to obtain a 2D aligned crop (b). Localizing additional 67 fiducial points in the 2D aligned crop (c). Manually place 67 anchor points on the 3D shape model (d). Source: Taigman, Y. et al. In Proc. Conference on Computer Vision and Pattern Recognition 1701–1708 (2014). Detect 6 fiducial points in the 2-D detection cropped image (a). Scale, rotate and translate image to anchor locations to obtain (b). To register for out-of-plane rotations, a 3D shape model is used to warp 2D images to 3D shape. A generic 3-D shape model obtained from USF Human ID database. Additional 67 fiducial points are detected on the 2-D aligned crop image (c). These additional fiducial points are then manually mapped on to corresponding 3D references. An affine 3D-to-2D camera is then fitted using least squares solution. To reduce corruption caused by non-rigid deformation and full perspective projection, a residual is added to each component of fiducial point. Frontalization is achieved by a piece-wise affine transformation T from source to target as directed by the Delaunay triangulation.

Face Alignment pipeline: 3D Modeling An affine 3D-to-2D camera P is then fitted using the generalized least squares solution to the linear system X2d = X3dP (e). The 67 fiducial points induced by the 3D model that are used to direct the piece-wise affine warpping (f) to achieve frontalization (g). A new view generated by the 3D model (h). Source: Taigman, Y. et al. In Proc. Conference on Computer Vision and Pattern Recognition 1701–1708 (2014). Detect 6 fiducial points in the 2-D detection cropped image (a). Scale, rotate and translate image to anchor locations to obtain (b). To register for out-of-plane rotations, a 3D shape model is used to warp 2D images to 3D shape. A generic 3-D shape model obtained from USF Human ID database. Additional 67 fiducial points are detected on the 2-D aligned crop image (c). These additional fiducial points are then manually mapped on to corresponding 3D references. An affine 3D-to-2D camera is then fitted using least squares solution. To reduce corruption caused by non-rigid deformation and full perspective projection, a residual is added to each component of fiducial point. Frontalization is achieved by a piece-wise affine transformation T from source to target as directed by the Delaunay triangulation.

Deep Neural Networks Deep neural networks are a type of artificial neural networks with multiple hidden layers. Input layer takes in a vector of features which are fed forward to either hidden or output layers. Hidden layers transform input data to intermediate representations. Can be fully-connected, convolutional, pooling, recurrent, etc. Output layer output predicted labels. Talk about pooling? Talk about locally connected layers?

Deep Neural Network in DeepFace Input is a 152x152 Image as 3 channels (red, green, blue) (C1) 2D convolutional layer 32x11x11x3 (M2) Max-pooling layer (C3) 2D convolutional layer 16x9x9x16 (L4-6) Locally connected layers (F7,8) Fully connected layers, F8 uses softmax Extract low-level features Local statistics Correlate parts of faces Taigman et. al.

Siamese Network Uses the previously trained neural network. The neural network without F8 is used in parallel with identical weights. Determine loss as absolute error between the two sets of labels. Output identifies if the two inputs represent a single logistic unit.

Results: Deep Neural Network in DeepFace DeepFace uses a combination of several networks, all of which are DeepFace-single networks with a certain input type: 3D aligned RGB Gray-level image plus gradient magnitude and orientation 2D aligned RGB These are combined using a non-linear SVM Uses a “kernel trick” to implicitly map data to higher-dimensional spaces Accuracy of the combined three networks is 97.25% Adding 4 more DeepFace-single networks to the set of networks increases the accuracy to 97.35% These accuracy values are based on the YTF dataset. Check usage of SVM Show figure of true positive vs. false positive (accuracy table) - can’t, this data isn’t present in the paper. Clean up wording/explainations - done.

Future work in facial recognition Create more diverse datasets in order to better recognize all types of faces MIT study showed lack of diversity in existing datasets IBM has created a dataset with over 1 million faces Darker-skinned females are most misclassified group with error rates up to 34.7% Lighter-skinned males are most well-classified, with error rates only reaching 0.8% Creating a facial recognizer that is invariant to: Pose Orientation Image quality Facial expression Discuss future work in the topic

Questions?