1. Ear biometrics Hanna-Kaisa Lammi 19.11.2003

2. Contents  Biometrics in general  Using ear shape as biometrics  Three types of ear biometrics –Pictures –Earmarks –Thermograms  Two cases  Application scenarios and possible errors

3. Biometrics Biometrics are unique physical or behavioral characteristics of an individual which can be measured and thus compared to accurately verify or identify an individual. http://www.biopay.com/

4. Ideal biometric  universal: each person should possess the characteristics  unique: no two persons should share the characteristics  permanent: the characteristics should not change  collectable: easily presentable to a sensor and quantifiable

5. Identification or authentication?  Identification: the biometric is combined with the database to find who the person is: “Who I am?”  Authentication: the biometric is compared with data, which is known to be valid, e.g. data in identification card “Am I who I claim I am?”

6. Ear shape  physical biometric, which is characterized by the shape of the outer ear, lobes and bone structure  unique enough?  new biometric, not widely used yet  no applications available yet

7. Ear Most important parts in ear shape identification are Foseta Antitrago Helix

8. Same ear can look different…

9. Advantages - disadvantages  Ears are smaller than e.g. faces  reduced spatial resolution  Ears are not as variable as e.g. faces  We have almost none adjectives to describe ears: we can recognize people from faces but can we recognize them from ears?

10. Permanence of biometrics Bromba GmbH

11. 3 types of ear biometrics  biometrics from photo of ear: most researched, most famous is Iannarelli’s classification work  “ear marks”: interesting in crime detection  thermogram pictures: a new possibility

12. Alfred Iannarelli  deputy sheriff and campus police in California, no scientific background  compared over 10,000 ears drawn from a randomly selected sample in California  another study was among identical and non- identical twins  Using Iannarelli’s measurements  Result: ears are not identical. Even identical twins had similar but not identical ears.

13. Iannarelli’s measurements (a) Anatomy, (b) Measurements. (a) 1 Helix Rim, 2 Lobule, 3 Antihelix, 4 Concha, 5 Tragus, 6 Antitragus, 7 Crus of Helix, 8 Triangular Fossa, 9 Incisure Intertragica. (b) The locations of the anthropometric measurements used in the “Iannarelli System”. (Burge et al., 1998)

14. Iannarelli’s system - weaknesses  if the first point is incorrect, all measurements are incorrect  localizing the anatomical points is not very well suitable for machine vision  some other methods had to be found

15. Methods using pictures (1/4)  Moreno et al. (1999) –multiple identification method, which combines the results from several neural classifiers using feature outer ear points, information obtained from ear shape and wrinkles, and macro features extracted by compression network –introduce three different classification techniques for outer ear or auricle identifying.

16. Methods using pictures (2/4)  Burge and Burger (1998, 2000) –automating ear biometrics with Voronoi diagram of its curve segments. –a novel graph matching based algorithm for authentication, which takes into account the possible error curves, which can be caused by e.g. lightning, shadowing and occlusion.

17. Methods using pictures (3/4)  Hurley, Nixon and Carter (2000a, 2000b) –force field transformations for ear recognition. –the image is treated as an array of Gaussian attractors that act as the source of the force field –according to the researchers this feature extraction technique is robust and reliable and it possesses good noise tolerance.

18. Methods using pictures (4/4)  Victor, Chang, Bowyer, Sarkar (at least 2 publications in 2002 and 2003) –principal component analysis approach –comparison between ears and faces This method is presented later with 2 cases.

19. Earmarks  when the ear is pressed against some hard material, e.g. glass, the result is called earmark  used in forensic research  at least 4 criminals are judged because of earmarks in decade of 1980 in England and in The Netherlands  later it was decided by a Dutch court that earmarks are not unique and reliable enough

20. Ear thermogram Thermogram of an ear. Image provided by Brent Griffith, Infrared Thermography Laboratory, Lawrence Berkeley, National Laboratory. (Burge et al., 1998) used for removing “non-ear”, e.g. hair and other obstacles in this case hair is between 27.2 and 29.7ºC while the outer ear areas range from 30.0 to 37.2 ºC

21. Case 1: an evaluation of face and ear biometrics  Done by Victor, B., Bowyer, K., and Sarkar S, published in Proceedings of International Conference on Pattern Recognition, August 2002  The used method is principal component analysis (PCA) and the design principle is adopted from the FERET methodology  Null hypothesis: there is no significant performance difference between using the ear or face as a biometric

22. PCA Method

23. Points for normalization

24. Tests of research by Victor et al.  For faces: 1.Same day, different expression 2.Different day, similar expression 3.Different day, different expression  For ears: 1.Same day, opposite ear 2.Different day, same ear 3.Different day, opposite ear

25. Same day, different expression or opposite ear ear

26. Different day, similar expression or same ear ear

27. Different day, different expression or opposite ear ear

28. Victor et al. research result Experiment #Expected ResultResult 1Same day, different expression Same day, opposite ear Greater variation in expressions than ears; ears perform better Face performs better 2Different day, similar expression Different day, same ear Greater variation in expression across days; ears perform better Face performs better 3Differet day, different expression Different day, opposite ear Greater variation in face expression than ear; ears perform better Face performs better Face/Ear compared

29. Case 2: Ear and Face images  ”Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics” by Chang, K., Bowyer, K.W., Sarkar, S., and Victor, B.  Published in IEEE Transactions on Pattern Analysis and Machine Intelligence on September 2003.  Hypothesis: (i) ear provide better biometric performance than images of the face and (ii) exploring whether a combination of ear and face images may provide better performance than either one individually

30. Images used in research by Chang et al. (2003) Same kinds of sets for faces, too. PCA, FERET

31. Tests for the research 1. day variation  other conditions constant 2. different lightning condition  taken in the same day in the same session 3. pose variation  22.5 degree rotation, other conditions constant, taken in the same day Combined face, ear and face+ear What do you think the result was?

32. Day variation test

33. Different lightning conditions

34. Pose variation (22.5 degree rotation)

35. Results  In this research face biometrics seem to be better in constant conditions, ear biometrics in changing conditions  Multimodal biometrics face plus ear gives the best results, why not use them?

36. Ear shape applications  currently there are no applications, which use ear identification or authentication  crime investigation is interested in using ear identification  active ear authentication could be possible in different scenarios

37. Application scenarios  Application scenarios are quite traditional for biometrics: –collecting child from daycare –ATM –any other active identification  passive –e.g. when trying to solve a crime and there’s a picture of ear in the tape of a surveillance camera e.g. gas station robbery in The Netherlands

38. Biometric suitability for authentication purpose Bromba GmbH

39. Error possibilities in ear recognition

40. False rates in ear identification  FRR = false reject rate  FAR = false acceptance rate Normally one of these is tried to minimize.

41. Improving the FRR with ear curve widths, an example width of an ear curve corresponding to the upper Helix rim  better results

42. Removal of noise curves in the inner ear Graph model (Burge et al.) and false curves because of e.g. oil and wax of the ear.

43. Possibilities to enhance ear biometrics  Using accurate measurements, e.g. ear curve and upper helix rim  Removing noise curves  Thermograms  removal of obstacles  Better quality cameras  more accurate pictures  Combined biometrics

44. References (1/2)  Bamber, D. Prisoners to appeal as unique ‘earprint’ evidence is discredited. Telegraph Newspaper (UK). Updated 02/12/2001 [Retrieved October 3, 2003] From: http://portal.telegraph.co.uk/news/main.jhtml?xml=/news/2001/12/02/nearp02.xml  Bromba GmbH, Bioidentification Frequently Asked Questions. Updated 2003-09-12 [Retrieved October 28, 2003] From: http://www.bromba.com/faq/biofaqe.htm.  Burge, M. and Burger, W. Ear Biometrics. In A. Jain R. Bolle and S. Pankanti, editors, BIOMETRICS: Personal Identification in a Networked Society, pp. 273-286. Kluwer Academic, 1998.  Burge, M. and Burger, W. Ear Biometrics in Computer Vision. In the 15 th International Conference of Pattern Recognition, ICPR 2000, pp. 826-830.  Carreira-Perpinan, M.A. Abstract from MSc thesis Compression neural networks for feature extraction: Application to human recognition from ear images, Technical University of Madrid. 1995. [Retrieved October 16, 2003]. From http://www.dcs.shef.ac.uk/~miguel/papers/msc-thesis.html.  Chang, K., Bowyer. K.W., Sarkar, S., Victor, B. Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, September 2003, pp. 1160-1165.  Forensic Evidence News: Ear Identification. Version updated Nov. 1, 2000. [Retrieved October 3, 2003] From http://www.forensic-evidence.com/site/ID/IDearNews.html.

45. References (2/2)  Hoogstrate, A.J., Van den Heuvel, H., Huyben, E. Ear Identification Based on Surveillance Camera’s Images. Version updated May 31, 2000. [Retrieved October 7, 2003] From: http://www.forensic- evidence.com/site/ID/IDearCamera.html.  Hurley, D.J., Nixon, M. S., Carter, J.N. Automated Ear Recognition by Force Field Transformations in Proceedings IEE Colloquium: Visual Biometris (00/018), 2000a, pp. 8/1-8/5.  Hurley, D.J., Nixon, M.S., Carter, J.N. A New Force Field Transform for Ear and Face Recognition. In Proceedings of the IEEE 2000 International Conference on Image Processin ICIP 2000b, pp. 25- 28.  Jain, A., Hong, L., Pankati, S. Biometric Identification. Communications of the ACM, February 2000/Vol. 43, No. 2, pp. 91-98.  Moreno, B., Sánchez, Á., Vélez. J.F. On the Use of Outer Ear Images for Personal Identification in Security Applications. IEEE 33 rd Annual International Carnahan Conference on Security Technology, 1999, pp. 469-476.  Morgan, J. Court Holds Earprint Identification Not Generally Accepted In Scientific Community, State v. David Wayne Kunze. 1999. [Retrieved September 9, 2003]. From: http://www.forensic- evidence.com/site/ID/ID_Kunze.html  Ratha, N.K., Senior, A., Bolle, R.M. Automated Biometrics in Proceedings of International Conference on Advances in Pattern Recognition, Rio de Janeiro, Brazil, March 2001.  Victor, B., Bowyer, K., Sarkar, S. An evaluation of face and ear biometrics in Proceedings of International Conference on Pattern Recognition, pp. 429-432, August 2002.

46. Questions? Thank you for your attention!