Download presentation
Presentation is loading. Please wait.
1
Recognition of Faces and Facial Attributes using Accumulative Local Sparse Representations
Domingo Mery Department of Computer Science Universidad Católica de Chile Sandipan Banerjee Department of Computer Science & Engineering University of Notre Dame
2
Agenda Motivation Proposed method Experiments Conclusions
3
Agenda Motivation Proposed method Experiments Conclusions
4
class 1 class 2 class 3 : : : LEARNING TESTING class description
classifier’s design LEARNING TESTING description classification class query image
5
class 3 class 2 class 1 : LEARNING TESTING class description
classification class classifier’s design query image
6
Are all parts of the face important?
Important for gender. This problem gives rise to some interesting questions This problem raises some interesting questions Some interesting questions arise out of this problem Important for race. Important for expression.
7
Are all parts of the face important?
This problem gives rise to some interesting questions This problem raises some interesting questions Some interesting questions arise out of this problem Important for Mary.
8
Are all parts of the face important?
This problem gives rise to some interesting questions This problem raises some interesting questions Some interesting questions arise out of this problem Important for Mary and Miguel. Not important at all!!! Important for Miguel.
9
Agenda Motivation Proposed method Experiments Conclusions
10
SRC Sparse Representation Classification
Our method is based on... SRC Sparse Representation Classification
11
Gallery Subject Subject Subject Subject Subject-k . . .
12
Gallery = Dictionary Subject Subject Subject Subject Subject-k . . .
13
. . . Gallery = Dictionary Sparse Representation Query
Subject Subject Subject Subject Subject-k . . . Sparse Representation Query
14
. . . Gallery = Dictionary 0.6 0.3 0.1 Sparse Representation Query
Subject Subject Subject Subject Subject-k . . . 0.6 0.3 0.1 Sparse Representation Query
15
. . . Gallery = Dictionary 0.6 0.3 0.1 Sparse Representation Query
Subject Subject Subject Subject Subject-k . . . 0.6 0.3 0.1 Sparse Representation Query
16
+ + . . . Gallery 0.6 0.3 0.1 Sparse Representation 0.3 0.6 0.1 Query
Subject Subject Subject Subject Subject-k . . . 0.6 0.3 0.1 Sparse Representation Query
17
Gallery Subject Subject Subject Subject Subject-k . . . The query image is represented as a linear combination of few images of the gallery. Query
18
Query
19
Query - Not similar: reconstruction error is high
20
Query - Very similar: reconstruction error is very low
21
Query Query is classified as this subject In SRC, the query is classified as the subject with the lowest reconstruction error
22
ALSR Accumulative Local Sparse Representation [ PROPOSED METHOD ]
23
Our approach uses Patches!
description selection classification ID query image
24
. . . . . . class i class 1 class k : : : LEARNING TESTING class
: : : description description description classifier’s design LEARNING TESTING description classification class query image
25
. . . . . . class i class 1 class k : : : LEARNING TESTING
: : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING Sparse representations have been widely used in many computer vision problems such as face recognition. We build a dictionary from the gallery images. We reconstruct the query image using a sparse combination of the dictionary. We recognize the class by searching the minimal reconstruction error. description sparse representation classification class query image
26
. . . . . . class i class 1 class k : : : LEARNING TESTING class
: : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING description SRC class query image
27
. . . . . . class 1 class i class k : : : : : : LEARNING TESTING class
: : : : : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING description SRC class query image
28
. . . . . . class 1 class i class k : : : : : : LEARNING TESTING class
: : : : : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING description SRC class query image
29
. . . . . . for each test patch : for all test patches class 1 class i
class k : : : : : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING Huge dictionary Sparse representation could be very time consuming for each test patch for all test patches majority vote : class description SRC query image
30
. . . . . . for each test patch : for all test patches class 1 class i
class k : : : : : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING for each test patch for all test patches majority vote : class description selection of best dictionaries SRC query image
31
. . . . . . for each test patch : for all test patches class 1 class i
class k : : : : : : description description description dictionary 1 dictionary i dictionary k LEARNING TESTING for each test patch for all test patches majority vote : class description selection of best dictionaries SRC score query image
32
Visual Vocabulary & Stop List
description : class k class i class 1 Visual Vocabulary & Stop List dictionary 1 dictionary i dictionary k LEARNING TESTING for each test patch for all test patches majority vote : class description selection of best dictionaries SRC face mask score query image
33
. . . . . . Gallery For each image of the gallery: Dictionary
Subject Subject Subject Subject Subject-k . . . For each image of the gallery: Dictionary . . . Patches of Subject-1 Patches of Subject-2 Patches of Subject-k + position (x,y) of each patch
34
. . . Gallery Patches of Class-1 Class-2 Class-k General dictionary D
Class Class Class Class Class-k Gallery Patches of Class-1 Class-2 Class-k General dictionary D
35
Dictionary Query
36
i For patch i 0. Original dictionary
1. Selection of the nearest patches (using (x,y) information) 2. Selection of the most similar patches (using intensity information) 3. Sparse Representation 0.5 0.3 0.2 Contribution si 4. 0.8 ... 0.2 i For patch i
37
i D Dn Ds For patch i 0. Original dictionary
1. Selection of the nearest patches (using (x,y) information) Dn 2. Selection of the most similar patches (using intensity information) Ds 3. Sparse Representation of yi using Ds 0.5 0.3 0.2 xi Contribution i 4. si 0.8 ... 0.2 i Patch yi Neighborhood of yi For patch i
38
Subject Contribution of each patch: 1 2 3 N 0.1 0.5 0.2 0.3 0.4 0.6 : 0.7 z 1.1 14.1 0.7 1.5 TOTAL Query classified as #2
39
Query Patches Contribution p 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 0.2
0.1 0.3 0.4 0.6 0.7 0.5 1 12 Query
40
x x x x Patches that are not discriminative can be removed
Patches Mask Contribution p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 0.2 0.1 0.3 0.4 0.6 0.7 0.5 Patches that are not discriminative can be removed x x Face masks used in our experiments x x Query
41
Patches that are not discriminative can be removed
Patches Mask Contribution p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.3 0.2 0.4 0.6 0.5 Patches that are not discriminative can be removed Query
42
SCI: Sparsity Concentration Index (score)
Patches Mask Contribution SCI p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.3 0.2 0.4 0.6 0.5 SCI - 0.2 0.3 0.4 0.8 0.5 SCI: Sparsity Concentration Index (score) Query
43
SCI: Sparsity Concentration Index > 0.25
Patches Mask Contribution SCI p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.3 0.2 0.4 0.6 0.5 SCI - 0.2 0.3 0.4 0.8 0.5 SCI: Sparsity Concentration Index > 0.25 Query
44
SCI: Sparsity Concentration Index > 0.25
Patches Mask Contribution SCI p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.2 0.3 0.4 0.8 0.5 SCI: Sparsity Concentration Index > 0.25 Query
45
SCI: Sparsity Concentration Index > 0.25
Patches Mask Contribution SCI p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 SCI: Sparsity Concentration Index > 0.25 Query
46
Maximal value of each contribution
Patches Mask Contribution SCI max p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 max - 0.4 0.6 0.3 0.5 Maximal value of each contribution Query
47
Each contribution is divided by its maximum
Patches Mask Contribution SCI max Normalization p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 max - 0.4 0.6 0.3 0.5 1 2 3 4 - 0.25 1.0 0.17 0.33 0.2 0.4 Each contribution is divided by its maximum Query
48
The normalized contributions must be greater than 0.2
Patches Mask Contribution SCI max Normalization p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 max - 0.4 0.6 0.3 0.5 1 2 3 4 - 0.25 1.0 0.17 0.33 0.2 0.4 The normalized contributions must be greater than 0.2 Query
49
The normalized contributions must be greater than 0.2
Patches Mask Contribution SCI max Normalization p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 max - 0.4 0.6 0.3 0.5 1 2 3 4 - 0.25 1.0 0.33 0.4 The normalized contributions must be greater than 0.2 Query
50
The query is classified according the maximal normalized contribution
Patches Mask Contribution SCI max Normalization p 1 2 3 4 5 6 7 8 9 10 11 12 q 1 1 2 3 4 - 0.1 0.4 0.6 0.2 0.3 0.5 SCI - 0.3 0.4 0.8 0.5 max - 0.4 0.6 0.3 0.5 1 2 3 4 - 0.25 1.0 0.33 0.4 0.25 3.33 1.4 0.33 max The query is classified according the maximal normalized contribution Query
51
The code of the MATLAB implementation is available on our webpage:
> Material > ALSR
52
Agenda Motivation Proposed method Experiments Conclusions
53
Experiments Face Recognition in LFW Gender Recognition in AR
Expression Recognition in Oulu-CASIA
54
Face Recognition in LFW [ PROTOCOL ]
The gallery has 143 subjects with at least 11 images per subject (10 for training, the rest for testing). There are 1430 images for training and 2744 for testing.
55
Example in LFW Contributions per class Maximum for #117 1 ... 117 143
Images of the same subject in the gallery (subject #117). Query image Contributions per class Maximum for #117
56
Example in LFW Images of the same subject in the gallery (subject #98).
57
80.8 In this table, we do not report deep learning methods that require millions of training images (for the sake of truth, VGG-Face in this experiment achieves 97.7%)
58
Gender Recognition in AR [ PROTOCOL ]
100 subjects (50 women and 50 men). For gender recognition, 14 non-occluded images per subject. In this experiment, the first 25 males and 25 females were used for training and the last 25 males and 25 females were used for testing.
60
Expression Recognition in Oulu-CASIA [ PROTOCOL ]
Six different facial expressions (surprise, happiness, sadness, anger, fear and dis- gust) under normal illumination from 80 subjects (59 males and 21 females) ranging from 23 to 58 years in age. The dataset contains 480 sequences: the first 9 images of each sequence are not considered, the first 40 individuals are taken as training subset and the rest as testing.
62
Agenda Motivation Proposed method Experiments Conclusions
63
We presented a new algorithm that is able to recognize faces and facial attributes automatically from face images captured under less constrained conditions including some variability in ambient lighting, pose, expression, size of the face and distance from the camera. The robustness of our algorithm is due to two reasons: The dictionary used in the recognition corresponds to a rich collection of representations of relevant parts which were selected using closeness and similarity criteria. The testing stage is based on accumulative sparse contributions according to location and relevance criteria. We believe that this new approach can be used to solve other kind of computer vision problems in which there are similar unconstrained conditions and a huge number of training images is not available. In the future, we will train our own deep learning network to obtain a better description of the patches, and we will learn the face image masks from training data, instead of manual selection.
64
Recognition of Faces and Facial Attributes using Accumulative Local Sparse Representations
Domingo Mery Department of Computer Science Universidad Católica de Chile Sandipan Banerjee Department of Computer Science & Engineering University of Notre Dame
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.