Partial Face Recognition

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Object Recognition from Local Scale-Invariant Features David G. Lowe Presented by Ashley L. Kapron.
Zhimin CaoThe Chinese University of Hong Kong Qi YinITCS, Tsinghua University Xiaoou TangShenzhen Institutes of Advanced Technology Chinese Academy of.
Face Recognition. Introduction Why we are interested in face recognition? Why we are interested in face recognition? Passport control at terminals in.
Face Processing in Social Networking Services NTIA Privacy Multistakeholder Process: Commercial Facial Recognition Technology March 25, 2014 Olga Raskin.
Proposed concepts illustrated well on sets of face images extracted from video: Face texture and surface are smooth, constraining them to a manifold Recognition.
Computer Vision for Human-Computer InteractionResearch Group, Universität Karlsruhe (TH) cv:hci Dr. Edgar Seemann 1 Computer Vision: Histograms of Oriented.
Instructor: Mircea Nicolescu Lecture 15 CS 485 / 685 Computer Vision.
Partial Face Recognition
Robust Object Tracking via Sparsity-based Collaborative Model
Matching with Invariant Features
Recognition using Regions CVPR Outline Introduction Overview of the Approach Experimental Results Conclusion.
1 Interest Operators Find “interesting” pieces of the image –e.g. corners, salient regions –Focus attention of algorithms –Speed up computation Many possible.
A Study of Approaches for Object Recognition
UPM, Faculty of Computer Science & IT, A robust automated attendance system using face recognition techniques PhD proposal; May 2009 Gawed Nagi.
1 Interest Operator Lectures lecture topics –Interest points 1 (Linda) interest points, descriptors, Harris corners, correlation matching –Interest points.
Feature extraction: Corners and blobs
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman University of Oxford ICCV 2003.
Object Recognition Using Distinctive Image Feature From Scale-Invariant Key point D. Lowe, IJCV 2004 Presenting – Anat Kaspi.
Scale Invariant Feature Transform (SIFT)
Automatic Face Recognition for Film Character Retrieval in Feature-Length Films Ognjen Arandjelović Andrew Zisserman.
Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics IEEE Trans on PAMI, VOL. 25, NO.9, 2003 Kyong Chang, Kevin W. Bowyer,
1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Spatial Pyramid Pooling in Deep Convolutional
Scale-Invariant Feature Transform (SIFT) Jinxiang Chai.
Overview Introduction to local features
Learning to classify the visual dynamics of a scene Nicoletta Noceti Università degli Studi di Genova Corso di Dottorato.
Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.
Computer vision.
Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR) Presenter: Yao-Hung Tsai Dept. of Electrical Engineering, NTU.
EADS DS / SDC LTIS Page 1 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen.
Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.
Shape-Based Human Detection and Segmentation via Hierarchical Part- Template Matching Zhe Lin, Member, IEEE Larry S. Davis, Fellow, IEEE IEEE TRANSACTIONS.
Recognition and Matching based on local invariant features Cordelia Schmid INRIA, Grenoble David Lowe Univ. of British Columbia.
COMPUTER VISION: SOME CLASSICAL PROBLEMS ADWAY MITRA MACHINE LEARNING LABORATORY COMPUTER SCIENCE AND AUTOMATION INDIAN INSTITUTE OF SCIENCE June 24, 2013.
Overview Harris interest points Comparing interest points (SSD, ZNCC, SIFT) Scale & affine invariant interest points Evaluation and comparison of different.
Local invariant features Cordelia Schmid INRIA, Grenoble.
Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.
Video Based Palmprint Recognition Chhaya Methani and Anoop M. Namboodiri Center for Visual Information Technology International Institute of Information.
Face Recognition: An Introduction
Local invariant features Cordelia Schmid INRIA, Grenoble.
Human Detection Method Combining HOG and Cumulative Sum based Binary Pattern Jong Gook Ko', Jin Woo Choi', So Hee Park', Jang Hee You', ' Electronics and.
Jack Pinches INFO410 & INFO350 S INFORMATION SCIENCE Computer Vision I.
Overview Introduction to local features Harris interest points + SSD, ZNCC, SIFT Scale & affine invariant interest point detectors Evaluation and comparison.
776 Computer Vision Jan-Michael Frahm Spring 2012.
Image features and properties. Image content representation The simplest representation of an image pattern is to list image pixels, one after the other.
Recognizing specific objects Matching with SIFT Original suggestion Lowe, 1999,2004.
1 05 December 2008 Still Face Challenge Problem Multiple Biometric Grand Challenge Preliminary Results of Version 1.
WLD: A Robust Local Image Descriptor Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikäinen, Xilin Chen, Wen Gao 报告人:蒲薇榄.
ZAGAZIG UNIVERSITY-BENHA BRANCH SHOUBRA FACULTY OF ENGINEERING ELECTRICAL ENGINGEERING DEPT. COMPUTER SYSTEM DIVISION GRAUDATION PROJECT 2003.
SIFT Scale-Invariant Feature Transform David Lowe
Guillaume-Alexandre Bilodeau
Video Google: Text Retrieval Approach to Object Matching in Videos
Recovery from Occlusion in Deep Feature Space for Face Recognition
Object detection as supervised classification
Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science
State-of-the-art face recognition systems
Attributes and Simile Classifiers for Face Verification
CSE 455 – Guest Lectures 3 lectures Contact Interest points 1
Domingo Mery Department of Computer Science
Brief Review of Recognition + Context
KFC: Keypoints, Features and Correspondences
SIFT keypoint detection
SIFT.
Paper Reading Dalong Du April.08, 2011.
Video Google: Text Retrieval Approach to Object Matching in Videos
Presented by Xu Miao April 20, 2005
Recognition and Matching based on local invariant features
Domingo Mery Department of Computer Science
Presentation transcript:

Partial Face Recognition Shengcai Liao NLPR, CASIA April 29, 2015

Background Cooperated face recognition People are asked to stand in front of a camera with good illumination conditions Border pass, access control, attendance, etc. Mostly solved http://www.prlog.org/10391917-face-recognition-solution-associating-reliability-with-security-system.html http://english.china.com/zh_cn/Olympic/spotlight/11068320/20080813/15025578.html

Background Unconstrained face recognition Images are captured arbitrarily without or with little user cooperation Video surveillance, hand held system, etc. Difficult task http://www.howtovanish.com/2010/01/avoid-nosy-surveillance-cameras/ http://www.physorg.com/news3233.html http://www.cultofmac.com/police-use-facial-recognition-iphone-app-to-id-perps/47059

Background Partial face recognition in unconstrained environments

Background Partial faces in unconstrained environments

Face Recognition and the London Riots Summer 2011 Widespread looting and rioting: FR leads to many arrests: Yet, many suspects still unable to be identified by COTS FRS: Extensive CCTV Camera Network:

Partial Face Recognition (PFR) Problem Recognize an arbitrary partial face image captured in uncontrolled environment Importance Recognize a suspect in crowd Identify a face from its partial image Difference from traditional face recognition Alignment? Feature representation? Classification?

Alignment Free Partial Face Recognition: Application Scenario × × Recognition by COTS software Face can be detected? Yes Face can be aligned? Yes detected face aligned face No No image containing face Recognition by alignment-free PFR algorithm manually cropped face region

Alignment Free Partial Face Recognition: Overview

Interest Point Based Local Descriptor keypoint detection interest region description matching

Interest Point Based Local Descriptor Image retrieval Image matching Object recognition Texture recognition Robot localization …

Interest Point Based Local Descriptor Intensity histogram SIFT HOG GLOH PCA-SIFT SURF

Face Description with Interest Points SIFT detector Detects blobs Limited keypoints CanAff detector Canny edge based Plenty keypoints SIFT (37 keypoints) CanAff (571 keypoints) K. Mikolajczyk, A. Zisserman, and C. Schmid, “Shape recognition with edge-based features,” in Proceedings of the British Machine Vision Conference, 2003.

Face Description with Interest Points Advantages of interest point detectors Detections of local structures, not pre-defined components Good for partial faces Affine invariance Good for pose/viewpoint changes High repeatability Good for partial face matching

Face Description with Interest Points Gabor Ternary Pattern (GTP) based descriptor

Multi Keypoint Descriptors (MKD) Each image is described by a set of keypoints and descriptors: Keypoints: p1, p2, …, pk Descriptors: d1, d2, …, dk The number of descriptors, k, may be different from image to image

MKD based Sparse Representation Classification (MKD-SRC) Descriptors of the same class c can be viewed as a sub-dictionary: A gallery dictionary is built: For each descriptor yi in a test sample , solve Determine the identity of the test sample by SRC:

MKD based Sparse Representation Classification (MKD-SRC)

An Example of MKD-SRC Solution Genuine Impostor Lowe’s SIFT MKD-SRC is more discriminant in recognizing partial faces

Fast Atom Filtering In the dictionary, the number of atoms, K, can be of the order of millions Fast atom filtering For each yi, filter out T (T<<K) atoms, i.e. T largest values of ci, resulting in a small sub-dictionary The filtering scales linearly w.r.t. K, while the remaining MKD-SRC task takes a constant time

Effects of the Fast Atom Filtering A subset of FRGCv2, with 1,398 gallery images and 466 probe images, resulting in K=111,643 for the dictionary

Extension to Partial Face Verification Ori. Mirror Probe Virtual Gallery MKD-SRC Background faces

Differences with Previous Methods Lowe’s SIFT Wright’s SRC MKD-SRC Size of descriptor per image variable fixed Face image requirement alignment-free aligned and cropped Collaborative Representation × √ Holistic face Arbitrary partial face

Differences with Previous Methods

Experimental Settings Open-set face identification: FRGC 2.0+, AR+, PubFig+ Face verification: LFW

Open-set Face Identification Task: determine the identity of the probe, or reject the probe Practical scenarios: watch-list surveillance, attendance, forensic search, SNS photo tagging, etc. Genuine Probe PG Need to accept and identify, but large intra-class variations same persons Gallery different persons Impostor Probe PN Need to reject, but can be similar, e.g. similar frontal faces

Open-set Face Identification Performance measures: Detection and identification rate: percentage of images in PG that correctly accepted and identified False accept rate: percentage of images in PN that falsely accepted Detail and a recent benchmark can be seen in http://www.cbsr.ia.ac.cn/users/scliao/projects/blufr/

Experiments with Partial Faces FRGCv2.0+ Gallery: 466 FRGC + 10,000 background Probe: 15,562 PG (partial face) + 10,000 PN 1 image per subject in gallery

Experiments on Holistic Occluded Faces AR+ Gallery: 135 AR + 10,000 background Probe: 1530 PG (occluded) + 10,000 PN 1 image per subject in gallery each subject has 6 (one session) or 12 (two sessions) images All images in PG are with sunglasses or scarf, and illumination variations

Experiments on Holistic Occluded Faces Gallery Probe It can be seen that faces are not aligned very well

Experiments on Holistic Occluded Faces Closed-set identification Wright’s SRC is not robust with only one training sample per class, though manually aligned faces were used Methods Recognition Rate (Rank-1 Rate) MKD-SRC 81.70% SIFT Keypoint Match by Lowe 58.89% FaceVACS 48.76% SRC by Wright et al. 13.20%

Open-set Identification on AR+ Challenging task: Gallery: frontal, no occlusion, 1 image / class PG: sunglasses or scarf, illumination PN: frontal, no occlusion MKD-SRC is able to reject 99% impostors (FAR=1%) while accepting >55% genuine samples at rank-1

Experiments on Labeled Faces in the Wild (LFW) LFW: real faces from the internet, with possible non-frontal view or occlusion 13,233 images of 5,749 subjects Verification scenario; images restricted protocol 10 random subsets for test, with each subset having 300 genuine and 300 impostor pairs

Experiments on LFW MKD-SRC outperforms FaceVACS and the best image-restricted method V1-like Fusion of MKD-SRC and PittPatt outperforms the best method A.P. MKD-SRC-GTP is much better than MKD-SRC-SIFT

Experiments on LFW Correctly (top row) and incorrectly (bottom row) recognized face images from the LFW database by MKD-SRC

Experiments on LFW A subset of partial faces from LFW Sunglasses, hats, occlusions by hand or other objects, large pose variations (>45˚) Note: limitations of LFW and a new benchmark are discussed in http://www.cbsr.ia.ac.cn/users/scliao/projects/blufr/

Experiments on PubFig+ Gallery: 83 PubFig + 5,000 LFW Probe: 817 PG (occluded) + 7,210 PN 1 image per subject in gallery

Summary Addressing the general partial face recognition problem without alignment A unified face recognition framework for both holistic and partial faces Improves SRC for the one-sample-per-class problem Multi keypoint descriptors enables variable-length face description

Suggestions for Future Work PFR is important but difficult. The proposed matching framework is not the only way to recognize partial faces. There are other possibilities, e.g. Weng et al. Robust feature set matching for partial face recognition, ICCV 2013 There may be other elegant partial face description methods Automatic PFR is even more difficult. Partial face detection is still missing

Thank you!