Partial Face Recognition

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.
Distinctive Image Features from Scale-Invariant Keypoints David Lowe.
Object Recognition from Local Scale-Invariant Features David G. Lowe Presented by Ashley L. Kapron.
Zhimin CaoThe Chinese University of Hong Kong Qi YinITCS, Tsinghua University Xiaoou TangShenzhen Institutes of Advanced Technology Chinese Academy of.
Detecting Faces in Images: A Survey
Face Description with Local Binary Patterns:
Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed.
Computer Vision for Human-Computer InteractionResearch Group, Universität Karlsruhe (TH) cv:hci Dr. Edgar Seemann 1 Computer Vision: Histograms of Oriented.
Fırat Onur Alsaran, Neşe Alyüz, Melike Esma İlter, Mürsel Taşgın, Ahmet Burak Yoldemir CmpE 58Z Term Project.
IIIT Hyderabad Pose Invariant Palmprint Recognition Chhaya Methani and Anoop Namboodiri Centre for Visual Information Technology IIIT, Hyderabad, INDIA.
Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction Atul Kanaujia, CBIM, Rutgers Cristian Sminchisescu, TTI-C Dimitris Metaxas,CBIM, Rutgers.
Instructor: Mircea Nicolescu Lecture 15 CS 485 / 685 Computer Vision.
Image alignment Image from
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Robust and large-scale alignment Image from
Patch Descriptors CSE P 576 Larry Zitnick
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
A Study of Approaches for Object Recognition
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
Distinctive Image Feature from Scale-Invariant KeyPoints
Object Recognition Using Distinctive Image Feature From Scale-Invariant Key point D. Lowe, IJCV 2004 Presenting – Anat Kaspi.
Object Recognition Using Geometric Hashing
Scale Invariant Feature Transform (SIFT)
Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics IEEE Trans on PAMI, VOL. 25, NO.9, 2003 Kyong Chang, Kevin W. Bowyer,
1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.
Lecture 6: Feature matching and alignment CS4670: Computer Vision Noah Snavely.
Scale-Invariant Feature Transform (SIFT) Jinxiang Chai.
Partial Face Recognition
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION.
Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.
Computer vision.
Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR) Presenter: Yao-Hung Tsai Dept. of Electrical Engineering, NTU.
Internet-scale Imagery for Graphics and Vision James Hays cs195g Computational Photography Brown University, Spring 2010.
EADS DS / SDC LTIS Page 1 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen.
Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.
Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.
Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Timo Ahonen, Abdenour Hadid, and Matti Pietikainen
A Tutorial on using SIFT Presented by Jimmy Huff (Slightly modified by Josiah Yoder for Winter )
By Pushpita Biswas Under the guidance of Prof. S.Mukhopadhyay and Prof. P.K.Biswas.
Distinctive Image Features from Scale-Invariant Keypoints
Presented by David Lee 3/20/2006
776 Computer Vision Jan-Michael Frahm Spring 2012.
Image features and properties. Image content representation The simplest representation of an image pattern is to list image pixels, one after the other.
Recognizing specific objects Matching with SIFT Original suggestion Lowe, 1999,2004.
Distinctive Image Features from Scale-Invariant Keypoints Presenter :JIA-HONG,DONG Advisor : Yen- Ting, Chen 1 David G. Lowe International Journal of Computer.
WLD: A Robust Local Image Descriptor Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikäinen, Xilin Chen, Wen Gao 报告人:蒲薇榄.
Face Detection & Recognition
776 Computer Vision Jan-Michael Frahm Spring 2012.
SIFT Scale-Invariant Feature Transform David Lowe
Guillaume-Alexandre Bilodeau
Presented by David Lee 3/20/2006
Lecture 07 13/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Scale Invariant Feature Transform (SIFT)
Recovery from Occlusion in Deep Feature Space for Face Recognition
Feature description and matching
Object Recognition in the Dynamic Link Architecture
Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science
Outline S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine.
From a presentation by Jimmy Huff Modified by Josiah Yoder
Aim of the project Take your image Submit it to the search engine
Domingo Mery Department of Computer Science
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
SIFT.
Feature descriptors and matching
Presented by Xu Miao April 20, 2005
Domingo Mery Department of Computer Science
Presentation transcript:

Partial Face Recognition Cite paper S. Liao, A. K. Jain, and S. Z. Li, "Partial Face Recognition: Alignment-Free Approach", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 5, pp. 1193-1205, May 2013, doi: 10.1109/TPAMI.2012.191

Cooperative Face Recognition People stand in front of a camera with good illumination conditions. Border pass, access control, attendance, etc. http://www.prlog.org/10391917-face-recognition-solution-associating-reliability-with-security-system.html http://english.china.com/zh_cn/Olympic/spotlight/11068320/20080813/15025578.html

Unconstrained Face Recognition Images are captured with less user cooperation, in more challenging conditions Video surveillance, hand held system, etc. http://www.howtovanish.com/2010/01/avoid-nosy-surveillance-cameras/ http://www.physorg.com/news3233.html http://www.cultofmac.com/police-use-facial-recognition-iphone-app-to-id-perps/47059

Partial Faces in Unconstrained Environments

Face Recognition and the London Riots Summer 2011 FR lead to many arrests: Widespread looting and rioting: Yet, many suspects still unable to be identified by COTS FRS: Extensive CCTV Network:

Face Detection in a Crowd PittPatt-5 Face Detector Normalized Pixel Difference (NPD) Face Detector OpenCV Viola-Jones Face Detector

Unconstrained Face Recognition Problem: Recognize an arbitrary face image captured in unconstrained environment Possible areas for improvement: Face detection? Alignment? Feature representation? Classification? Importance: Recognize a suspect in crowd Identify a face from its partial image

Alignment Free Partial Face Recognition (PFR) Proposed alignment-free method: MKD-SRC

Alignment Free Partial Face Recognition (PFR) Multi Keypoint Descriptors (MKD) Each image is described by a set of keypoints and descriptors (e.g. SIFT): Keypoints: p1, p2, …, pk Descriptors: d1, d2, …, dk The number of descriptors, k, may be different from image to image

Alignment Free Partial Face Recognition (PFR)

Sparse Representation Classification (SRC) based on MKD Descriptors from the same class c can be viewed as a sub-dictionary: Combining sub-dictionaries: For each descriptor yi of in a probe image, solve Determine the identity of the probe image by SRC:

Sparse Representation Classification (SRC) based on MKD

An Example Solution MKD-SRC is more discriminant for PFR Quincy Delight Jones Morgan Freeman Morgan Freeman is an American actor, film director, and narrator. Freeman has received Academy Award nominations for his performances in Street Smart, Driving Miss Daisy, The Shawshank Redemption and Invictus and won in 2005 for Million Dollar Baby. Quincy Delight Jones, Jr. is an American record producer, conductor, arranger, film composer, television producer, and trumpeter. MKD-SRC is more discriminant for PFR The horizontal axis represents the index of the gallery keypoint descriptors The vertical axis denotes the coefficient strength, as computed by

Large Scale Partial Face Recognition In the dictionary, the number of atoms, K, can be of the order of millions Fast atom filtering: (*) For each yi, we filter out only T (T<<K) atoms according to the top T largest values in ci, resulting in a small sub-dictionary. The computation of Eq. (*) is linear w.r.t. K, the selection of the largest T values can be done in O(K), thus the proposed fast atom filtering scales linearly w.r.t. K, while the remaining computation of l1 minimization takes a constant time.

Effects of the Fast Atom Filtering A subset of FRGCv2, with 1,398 gallery images and 466 probe images, resulting in K=111,643 for the dictionary.

Keypoint Descriptors Scale Invariant Feature Transform (SIFT) Advantage: promising results, efficient to compute Disadvantage: limited number of keypoints (~80), not affine invariant Gabor Ternary Pattern (GTP) descriptor Adopts edge based affine invariant keypoint detector called CanAff, which provides sufficient number of keypoints (~800) for PFR Robust to illumination variations and noises Even with fast atom filtering, run time is O(n2) with keypoints per image 10 times more keypoints, 100 times slower

Keypoint Descriptors SIFT (37) GTP (first 150 of 571)

GTP Descriptor

Keypoint Region Normalization Normalize the detected region to 40x40 pixels Clipped Z-Score normalization: Normalize the pixel values to [0,1] Reduce the influence of illumination variation Reduce the influence of extreme pixel values Add slide on MHD

Gabor Filters Odd Gabor filters with small scale, 4 orientations Imaginary part of Gabor filters, sensitive to edges and their locations. Scale 0, 5x5 support area, 0º, 45º, 90º, 135º

Local Ternary Pattern Encode the responses of the 4 Gabor filters Local structure about the responses of Gabor filters in 4 orientations Examples of some local structures encoded 4 orientations 2201 2011 0222

Building the descriptor Calculate the histogram of local ternary patterns (34 bins) over each grid cell, and concatenate them to form a 1,296 element vector Transform by a sigmoid function ( tanh(20x) ) Reduce the influence of extreme values Reduce the dimension to 128 by PCA

GTP Descriptor Local patch of 40x40 pixels 4x4 grid cells 34 bins for each cell 1296 bins in total PCA to 128 dims

Labeled Faces in the Wild (LFW)1 Real faces from the internet, most with non-frontal views or occlusion 13,233 images of 5,749 subjects 1 http://vis-www.cs.umass.edu/lfw/

Experiments on LFW MKD-SRC performs better than FaceVACS, but is not as good as PittPatt Fusion of MKD-SRC & PittPatt improves performance

Experiments on LFW Face image pairs that can be correctly recognized by MKD-SRC but not by PittPatt at FAR=1%

Experiment on PubFig Database2 Large-scale open-set identification Gallery: 5,083 full frontal faces Probe: 817 partial faces (belong to gallery) with large pose variation or occlusion 7,210 faces as impostors (do not belong to gallery) 2 http://www.cs.columbia.edu/CAVE/databases/pubfig/

Experiment on PubFig Database Proposed MKD-SRC method is better than two commercial SDKs, FaceVACS and PittPatt

Synthetic Partial Face Image Generation Rotate images; degree of rotation randomly drawn from a normal distribution (mean 0, std. dev. 10º) Sample width and height for the patch, drawn from a uniform distribution from 50-100% of original size Sample a starting position for the patch Randomly rescale the patch Original (size reduced for display) Rotated (size reduced for display) Rescaled patch Original size patch 5/28/2013

FRGC+ Dataset Open set recognition FRGC dataset Gallery: Probe 466 FRGC Images 10,000 PCSO Images Probe A. 15,562 FRGC partial faces (matching the FRGC subjects in gallery) B. 10,000 PCSO partial faces (not matching any gallery subjects) Average time per probe image ~1 second vs. 10,466 image gallery Pittpatt 5.2 fails to enroll ~50% of the partial faces

Experiment on MOBIO database3 Videos captured by mobile phone from six universities/institutes in Europe 4,880 videos of 61 subjects for verification Gallery (top) and probe (bottom) 3 http://www.idiap.ch/dataset/mobio

Experiment on MOBIO database A. Female B. Male

Experiment on the Mobile dataset Unconstrained face images with a mobile phone Pose, illumination, expression, occlusion or invisible parts Gallery images of 14 subjects plus additional 1,000 background subjects; one image/subject Probe: 168 mobile phone images of 14 subjects, with additional 1,000 impostors Open-set (watch-list) identification experiment

Experiment on the Mobile dataset Add summary, slide about MHD PittPatt cannot be applied because the probe faces cannot be aligned

Other Keypoint Matching Methods Keypoint based representations are naturally variable size The previously discussed method reconstructs each probe keypoint from the gallery using SRC Other options: Bag of words methods – fixed sized representation over a dictionary Modified Hausdorff Distance – apply a general distance metric to sets of points

Modified Hausdorff Distance Given a distance metric d, and 2 sets of keypoints A and B find: D(A,B) = mean(mina in A(d(a,B))) Compute the min distance from each keypoint in A to a keypoint in B, average the results over all keypoints in A D(A,B) ≠ D(B,A) MHD(A,B) = max(D(A,B), D(B,A)) We calculate all probe to gallery keypoint distances for the atom filtering step, so computing MHD is not costly

Summary Face recognition based on applying SRC to local keypoint descriptors Outperformed by other methods for mugshot style images, but can be used even when faces cannot be aligned E.g. only part of the face is available, or face/eye detection fail