Eye Movements Biometrics Where you look shows who you are Paweł Kasprowski, PhD Silesian University of Technology

Outline Eye movements at a glance How to measure eye movements Why do we measure it? Eye movements biometrics (EMB) – advantages EMVICompetition Summary

Physiology of eye movements Non-uniform picture quality across the visual field Area of high visual acuity zone Fovea Acuity drops outside Eyes in constant movement

Eye movements Oculomotor system Movement Elements Six muscles Three nerves Movement voluntary involuntary Elements Fixations Saccades Smooth pursuits Tremors, drifts

Eye movements Oculomotor system Movement Elements Six muscles Three nerves Movement voluntary involuntary Elements Fixations Saccades Smooth pursuits Tremors, drifts

Eye movements Oculomotor system Movement Elements Six muscles Three nerves Movement voluntary involuntary Elements Fixations Saccades Smooth pursuits Tremors, drifts

Eye movement measurement First researches: Emile Javal, 1879 First eye-tracker: Edmund Burke Huey, 1897 Different methodologies: Contact lens Elektro-oculography (EOG) Video-oculography (VOG)

Contact lenses Very accurate but very intrusive

Elektrooculography (EOG) Electric potentials measured with electrodes placed around the eyes Measures rather movement that gaze point

Videooculography (VOG) Detects eyes on face image Measures reflection of infrared light Purkinje images 1-4 Non-invasive May even be covert Computational power needed

Our eye tracker Frequency: 1 kHz Accuracy: about 1 degree

Eye movement visualization X-Y temporal graphs

Eye movement visualization Scan paths Olivier Le Meur, http://www.irisa.fr/temics/staff/lemeur/

Eye movement visualization Heat maps Tommy Strandvall, http://eyetracking.me

Eye movement usages Cognitive studies Medicine Usability of interfaces How our brain works Education, experience Medicine Diagnoses Usability of interfaces What attracts attention on an image? Advertisement, web pages Eye as a pointer Gaze instead of mouse? Adjusting image quality basing of a gaze point

Reading Every man reads differently Reading easy and difficult texts is different It is possible to find out education level!

Yarbus (1967) The way we move our eyes depends on what we are looking for Yarbus, A. L. (1967), Eye Movements and Vision, New York: Plenum

Gender differences EyeTrackShop, MRC International

Gender differences EyeTrackShop, MRC International

Interface usability Checking if application, interface, web page or leaflet is readable and understandable Checking if users pay attention where we want FengGui, www.feng-guii.com

Eye as a pointer Gaze instead of mouse? Problem: vision is our input not output Eye pointing may be not comfortable "Midas touch problem" Possible usages: For disabled Adaptive interface PIN entering (eye signature) CEATEC, Fujitsu eye-tracking technology

Eye Movement Biometrics Identification of people based on their eye movements Another example of biometric identification method Weird Science: 10 Forms of Biometric Authentication http://threatpost.com

Advantages Physiological elements Behavioral elements Acceptability Oculomotor Plant Muscles Behavioral elements Depends on stimulus "Measures" knowledge and experience Subject must be conscious Acceptability Accessibility Multimodality

Acceptability Easy to use Non-intrusive May be done during normal activity Possible covert identification

Accessibility Expensive for now It changes! Simple VOG eye-trackers Tablets Smartphones Mirametrix, Samsung

Multimodal biometrics Video-based eye trackers may be easily combined with: Face recognition Iris scanning Mouse and keystroke dynamics Other imaging methods

Challenges Distinctiveness Repeatability as in every behavioral method

State of the Art Idea of Prof. Ober (1945-2007) First publications inventor of Ober2 eye tracker First publications KASPROWSKI, P., OBER, J. 2003. Eye movement tracking for human identification, In 6th World Conference BIOMETRICS’2003, London. (Best Poster on Technological Advancement) KASPROWSKI, P., OBER, J. 2004. Eye Movement in Biometrics, In Proceedings of Biometric Authentication Workshop, European Conference on Computer Vision in Prague 2004, LNCS 3087, Springer-Verlag. KASPROWSKI P., OBER J. 2005. Enhancing eye movement based biometric identification method by using voting classifiers. SPIE Defense & Security Symposium.

Research centers Texas State University (Komogortsev) Jumping point, reading, OPC, CEM, +Iris SC University, Finland (Bendarik, Kinnunen) Static point stimulus, task independent University of Patras, Greece (Rigas) Static images (faces) Silesian Univ. of Technology (Kasprowski) Jumping point, noninteractive

EMB Challenges Results are not comparable No established techniques Different equipment, experiments, methods No established techniques Lack of common databases Our idea: competition similar to FVC

Competition Competition uses ready-to-use datasets Competition goals Eye movement recordings without proper identification Some training data Search for the best classification algorithm Competition goals Popularize the EMB Provide single reference point with a set of datasets Highlight importance of data quality

EMVIC 2012 Official competition of: Task: Web page: IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS 2012) Task: Assign every test sample to proper user id Web page: www.emvic.org www.kaggle.com/emvic

EMVIC Competition Four different datasets Jumping point type Dataset A (978 recordings of 37 persons) 8 sec, 4-100 rec. per person Dataset B (4168 recordings of 75 persons) 8 sec, 4-200 rec. per person Dataset C (116 recordings of 29 persons) Only left eye, 4 rec. per person, horizontal jumps Dataset D (108 recordings of 27 persons) Only left eye, 4 rec. per person, random jumps

Competition rules Publicly available: Evaluation Labeled training set Unlabeled testing set Evaluation Participants should correctly label unlabeled samples Training samples for building classification model Emvic.org and Kaggle competitions Only Dataset A for Kaggle

Competitors 45 users registered in the main competition 106 submissions Quatar, Wisconsin, Dehli, Brno, Alaska, Luxemburg… 49 competitors in Kaggle competition 524 submissions (only one daily limit per user)

Results Dataset A – 97% Dataset B – 95% Dataset C – 58% Dataset D – 67% Question: Why are the results so different for different datasets?

Why so different results? Number of samples per person About 26 per person in A, 55 in B, exactly 4 for C,D Sequences of samples in A, B Lack of calibration and worse data quality in A, B Horizontal movements only for C Random stimulus for D Last but not least: order of datasets Competitors started with Dataset A and didn't have enough time for C and D

Summary Eye movement biometrics has a lot of advantages Vision is the main human being's input It is not mature enough We need more generally available and comparable datasets

Thank you for your attention Paweł Kasprowski kasprowski@polsl.pl pawel@kasprowski.pl Drinks reception is waiting! Thank you for your attention