1. Tom Fladsrudtom@fladsrudsolutions.com Face Recognition Software in a border control environment: Non-zero-effort-attacks' effect on False Acceptance Rate MSc thesis – Information Security – NISlab - 2005

2. Tom Fladsrudtom@fladsrudsolutions.com Introduction to the problem The new Visa Information System  Initiated by the end of 2006  Face recognition mandatory?  Is face recognition the solution? – Will face recognition provide an acceptable false acceptance rate when facing non-zero effort impostors? – Is today’s evaluation of false acceptance rate, based on zero- effort attacks, sufficient? – A more realistic false acceptance rate based on non-zero effort attacks – The use of a human supervisor to reduce the false acceptance rate

3. Tom Fladsrudtom@fladsrudsolutions.com Research questions  What efforts does an imposter need to make to decieve a face recognition product in a border control environment? –The attacker must circumvent both the human guard and the computerbased recognition system –Zero effort impostors provide some false acceptances (Kosmerlj’s report 2004) when evaluated by a computerbased face recognition system. Are these also able to fool the human guard?  How will the resources of an attacker influence the security of a face recognition product in a border control environment?

4. Tom Fladsrudtom@fladsrudsolutions.com Research questions  Could today’s procedures for calculating FAR result in a positive evaluation of insecure products?  What effect will non-zero effort attacks have on the FAR of a face recognition product in a border control environment?

5. Tom Fladsrudtom@fladsrudsolutions.com Purpose of the project  Examine whether todays face recognition products are suitable for the new Visa Information System  Make both vendors and employers of face recognition more aware of the threats to face recognition  See what effort an impostor must make in order to circumvent both a human guard and the face recognition system

6. Tom Fladsrudtom@fladsrudsolutions.com Method  Theory review –State of the art Face recognition Human vs. computerbased face recognition Evaluation of face recognition software Multimodal biometrics –Different methods to circumvent face recognition systems  Email correspondence –Information about the Visa Information System and the process of applying for a visa  Experiment –Human abillity to capture false acceptances made by Face Recognition Systems –The effect hair has on human face recognition

7. Tom Fladsrudtom@fladsrudsolutions.com Theory Circumvention of face recognition software in a border control  Identical twins  Replay and alteration of templates  Illegitimate enrollment  Trojan horse  Impersonation attack –Orthodontic treatment –Facial makeup –Eyebrows  Attacks between the sensor and the biometric system  Back doors

8. Tom Fladsrudtom@fladsrudsolutions.com Theory Countermeasures  Controlled environment  Upper part of the face are less affected by facial expression  Frequently re-enrollment of the users  Cancelable biometrics  Liveliness detection –Weight –Human guard –Challenge the user to perform an particular expression –Stress level –Video surveillance  Multimodal biometrics

9. Tom Fladsrudtom@fladsrudsolutions.com The experiment Experiment description  Human comparison of image pairs with and without hair that have already been accepted as of the same individual by the computer- based face recognition system  Based on the results from Kosmerlj’s MSc thesis spring of 2004 (Passport of the future) using the best performing algorithm PCA MahCosine.  Used the images that produced more than one false acceptance  61 participants –31 was presented with imagepairs without hair visible –30 was presented with imagepairs with the hair visible  60 image pairs presented, not in the same order  10 sec. pr imagepair to evaluate

10. Tom Fladsrudtom@fladsrudsolutions.com The experiment

11. Tom Fladsrudtom@fladsrudsolutions.com The experiment Presentation of image with hair removed Presentation of image with hair and background present

12. Tom Fladsrudtom@fladsrudsolutions.com Analysis  Application to collect data from the experiment database –Semicolon separated result file –Developed in Java  Analysis of the results –Semicolon separated result- file were imported into SPSS (Statistical Package for the Social Sciences) for analysis

13. Tom Fladsrudtom@fladsrudsolutions.com Results  Hair –Mean number of false acceptances: With hair 3.90 Without hair 9.61

14. Tom Fladsrudtom@fladsrudsolutions.com Results  Hair – Distribution of errors With hair – 18 image-pairs were never evaluated wrong Without hair – Only 3 image-pairs were never evaluated wrong

15. Tom Fladsrudtom@fladsrudsolutions.com Results  Other aspects – Age of the participants Number of false acceptances are lower for people between the age of 27 and 37 than for those above and below that age

16. Tom Fladsrudtom@fladsrudsolutions.com Results  Other aspects – Educational degree Lower number of false acceptances with higher degree. – Time Error rate is constant Evaluation time decreases – Experience Only one participant had previous experience with such authentication

17. Tom Fladsrudtom@fladsrudsolutions.com Validity  With hair vs. without hair – The Levene’s test for equality and variance on False Acceptance and False Rejections on the two groups of image-pairs shows that there are a significant (sign. 0,019) difference between the false acceptances for human evaluation of face with hair and those where the hair is removed  Age and educational degree – The differeces in evaluation performance due to age could be a result of experience and better eyesight. – The differences in educational degree are propably a result of the age of these participants and not a result of the education of the participant.

18. Tom Fladsrudtom@fladsrudsolutions.com Interpretation of the results  Images where the hair is absent is more difficult to evaluate: –Hair has an influence on human recognition of faces  A human supervisor is able to capture some (less than 80% on average) of the faces that falsely passes a face recognition system –Combined with alteration of beard, makeup, eyebrows and teeth the capture rate could become significantly smaller –The results may indicate that a human supervisor provide some additional security, but may not be satisfactory in a border control setting. Face recognition should be combined with other security measures to achieve higher robustness.  The distribution of errors may indicate that the images resembled –The use of facial make-up, and alteration of beard, eyebrows and hair may provide a higher error rate –The additional security of a human guard may be further reduced

19. Tom Fladsrudtom@fladsrudsolutions.com The influence of the results  More awareness of the limited additional value of a human guard –other methods than only combining human guards and face recognition should be preferred –Use of multimodal biometrics an option for more accurate recognition  More awareness of the threats to face recognition –Hopefully more countermeasures will be made  More awareness of the importance of more accurate face recognition product  Training –Studies show that training improve the error rate, and should be used on border control personell. –In the training process the effect hair has on recognition performance should also be considered.

20. Tom Fladsrudtom@fladsrudsolutions.com Summary  Limited additional security with a human supervisor  Hair influences human evaluation of faces  Other authentication methods may be preferable to a combination of face recognition and a human supervisor

21. Tom Fladsrudtom@fladsrudsolutions.com Face Recognition Software Questions?