1. An Experimental Study of the Potential of Using Small
Size DNNs Collaboratively Solve a Complex Task
Boyu Zhang
Dec. 12, 2018

2. Executive Summary Objective: Tasks Performed: Self Assessment:
This project performs an experimental study to evaluate the potential of using small size DNN models collaboratively to solve a complex task. In particular, we studied how the search space of face ID can be narrowed by using predictions generated by age and gender models.
Tasks Performed:
Find the appropriate dataset and clean it up.
Trained DNN models for age and gender prediction. Then evaluate their individual performance as well as how much the search space is narrowed.
Aggressively simplified the trained models and evaluated them again.
Self Assessment:
Basically finished the tasks in the proposal but this is just a preliminary work towards the understanding of collaboration between small size DNNs.

3. Data Preparation Requirement of the dataset: Adience: Clean up:
For each sample in the dataset, there are multiple labels associated with it, and there exist a relationship between these labels. Thus, we can train some small DNN models to predict some labels and use the predictions to infer the information of the remaining labels.
Adience:
This is a dataset of face photos that was intended to facilitate the study of age and gender recognition. The images included in this dataset are captured under real-world imaging conditions.
26,580 images of 2,284 persons.
Each image has three labels: age (1 of 8 groups), gender (1 of 2 groups), and person ID.
Clean up:
The original dataset and the meta files associated with it are very dirty. They need to be cleaned before performing any training.
The downloaded dataset only contains 19,370 images. Moreover, are many age and gender labels are invalid and / or missing.
After cleaning up, 2,192 (output 2,284) persons have both valid age and gender labels, and this corresponds to 17,608 (output 19,370) images.

4. Model Training and Structural Simplification
Train age and gender models:
Model definitions are from the paper of the dataset.
We converted the images to TFRecord format for high performance training.
We implemented the data pipeline and training procedure in TensorFlow with Python.
The training is performed with GPU (1080 Ti) on my desktop. Each takes about less then one hour.
The trained models are then get evaluated with another Python script.
Structural Simplification (neuron elimination):
We extract the trained weights and intermediate outputs from the trained models.
We used Matlab to carry out the neuron elimination technique.
We evaluated the impact of the neuron elimination technique on model accuracies as well as the effectiveness of narrowing down the search space.

5. Results Training Loss Training Accuracy Validation Accuracy Age Model
Gender
Model

6. Results (simplify age model)
Metrics:
Gender accuracy, Age accuracy (exact / off-by-one), both accuracy (exact / off-by-one).
List of Candidate size (exact / off-by-one).
FC1
FC2
FC3

7. Results (simplify gender model)
Metrics:
Gender accuracy, Age accuracy (exact / off-by-one), both accuracy (exact / off-by-one).
List of Candidate size (exact / off-by-one).
FC1
FC2
FC3

8. Results (simplify both models)
Metrics:
Gender accuracy, Age accuracy (exact / off-by-one), both accuracy (exact / off-by-one).
List of Candidate size (exact / off-by-one).
FC1
FC2
FC3

9. Conclusions and Lesson Learned
Using current age and gender predictions to narrow down the search space of person ID is not very effective. The accuracies of the age and gender models need to be further improved to achieve more effective narrow down.
The age and gender models’ accuracy as well as the effectiveness of narrowing down the person ID search space are not very sensitive to the input neuron numbers. This may suggest there exist large redundancy in the original age and gender models.
Lesson learned:
Don’t trust the advertisements of dataset. There may exist many traps in the dataset.