Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION

Use location to narrow down search space Increasing number of photos taken with mobile devices Use the location information associated with the photo to narrow down the person in the photo SOLUTION Many facial expressions Changes in appearance Hair style Cosmetics With or without glasses Illumination Varying viewpoints CHALLENGES INTRODUCTION

GIVEN: Training set—set of face images labeled with the person’s identity Testing set—set of unlabeled photos from the same group of people GOAL: Identify each person in the testing photos FACE RECOGNITION PROBLEM

User will have different probabilities of appearing in a photo based on the location Example: Alice lives in Palo Alto, CA. Then pictures taken at Alice’s home have a high probability of belonging to Alice, Alice’s family, or Alice’s friends. The pictures have a low probability of belonging to someone in Norfolk, VA When trying to identify someone in the photo, only compare photos which are taken at places the person usually appears ASSUMPTIONS

Each face image is associated with a location The server creates clusters of locations from the training set Each location cluster contains a set of users who have photos in that location, their photos, and photos of their friends The client can take a photo and attach its location information, then send it to the server and query the person in the photo The server will answer the query and return the identification of the person in the photo HOW IT WORKS

HOW TO FORM THE LOCATION CLUSTERS AND THE GRANULARITY FOR THE LOCATIONS How to process the photo and extract useful features How to search smartly in order to recognize the face and identify the person How to accelerate the entire process and avoid long response time on client side CHALLENGES

Make use of the location information from mobile-taken photos and propose a face recognition algorithm which reduces the search space Build a hybrid face recognition algorithm FIRST SEARCH AND MATCH PHOTOS WITHIN THE GIVEN LOCATION; IF THIS FAILS, SEARCH OVER ALL PHOTOS MAIN CONTRIBUTIONS (1)

Take into account social network information When a user appears frequently in a locations, the user’s friends also have a high probability of showing up in that location FRIENDS PHOTOS ARE USED TO TRAIN THE FACE CLASSIFIER FOR THE LOCATION MAIN CONTRIBUTIONS (2)

Transmit the compressed face descriptor to the server for the query, rather than sending the original image SAVES THE NETWORK TRAFFIC AND REDUCES RESPONSE TIME MAIN CONTRIBUTIONS (3)

Client side: User takes a photo of people on a mobile phone and sends the recognition query to the server using wireless networks Face features and location information are transmitted to the server for recognition Server side: Organize the face database by locations Maintain a backup classifier which is obtained from all images in the database Sends back the identification result FRAMEWORK

LOCATION CLUSTERING

Training a classifier for each location: Convert photos associated with the location to feature descriptors Describing each face: Adopt a local descriptor based face feature pipeline Detecting faces: Viola-Jones face detector used to detect face patches Nested nose detector applied Face patches normalized to the same size FACE FEATURE

Use algorithm to detect facial landmarks Align each face patch Remove effects of illumination FROM EACH LANDMARK, TWO SIFT DESCRIPTORS OF DIFFERENT SCALES ARE EXTRACTED AND CONCATENATED TO FORM THE FACE FEATURE DESCRIPTOR FACE FEATURE

Face descriptor computed using the pipeline Compresses the descriptor and sends to the server MOBILE CLIENT Face descriptors are extracted from photos and trained by a Support Vector Machine (SVM) classifier Each location has its own SVM classifier and is represented by the coordinate of the cluster center When a query is received, it checks the location info and finds the nearest location in the database This location is used for face recognition Confidence score is defined If the confidence score is too low, use the backup database SERVER SYSTEM

Dataset: 2,001 images 60 people 6 locations Names and social network relations among the 60 people are known EVALUATIONS

Compared with a baseline method, i.e. method without location COMPARISON OF PERFORMANCE Accuracy of 5 tests at a particular location 80% of images used as training set; 20% used as testing set EVALUATION SUMMARY FACE RECOGNITION ACCURACY

Seems to improve accuracy Limitations: Only supports finding people already in the dataset Future work: Scalability—more locations will pose an issue Increase training set incrementally through social network or crowd wisdom Handling when people move from one location to another Predict which locations a person will appear since one person can be in many locations CONCLUSION