1. Ear biometrics Gökhan Şengül

2. Contents Using ear shape as biometrics Three types of ear biometrics Pictures Earmarks Thermograms Two cases Application scenarios and possible errors

3. Biometrics Biometrics are unique physical or behavioral characteristics of an individual which can be measured and thus compared to accurately verify or identify an individual. http://www.biopay.com/

4. Ideal biometric universal: each person should possess the characteristics unique: no two persons should share the characteristics permanent: the characteristics should not change collectable: easily presentable to a sensor and quantifiable

5. Ear shape physical biometric, which is characterized by the shape of the outer ear, lobes, curves, surfaces, geometric measurements, and bone structure unique enough? new biometric, not widely used yet

6. Ear Most important parts in ear shape identification are Foseta Antitrago Helix

7. Ear

8. Ear recognition - advantages The structure of the ear has been observed to be stable despite aging, and ear growth is almost linear after the age of four The ear, unlike other facial features, is minimally impacted by changes in facial expression Image acquisition does not involve explicit contact with the sensor

9. Ear recognition- disadvantages Ears are smaller than e.g. faces  reduced spatial resolution Ears are not as variable as e.g. faces We have almost none adjectives to describe ears: we can recognize people from faces but can we recognize them from ears?

10. Same ear can look different…

11. Permanence of biometrics Bromba GmbH

12. Ear recognition system Ear detection module localizes the position and spatial extent of the ear in an image; Feature extraction module Extracts discriminative features from the ear Matching module Extracts the features extracted from two ear images and generates a match score Decision module processes the match score(s) and establishes the identity of the subject

13. Ear Detection Templete Matching a template of a typical ear is constructed and is matched with each location in the query image. The location giving the highest score is considered as the region containing the ear. The template may consist of an edge image of the ear or a set of descriptors extracted from the ear such as the response to a set of filters or a histogram of shape curvatures in case a 3D image of the ear is being used for recognition.

14. Ear Detection Model-based detection A model-based detection technique assumes certain characteristics of the shape of the ear and tries to find regions that manifest such characteristics. The shape of the helix, for example, is usually elliptical so a generalized Hough transform tuned for detecting ellipses can be used to locate the ear in an edge image.

15. Ear Detection Morphological-operator-based detection Since the structure of the ear is usually more intricate than the structure of the remaining region in a profile face image, morphological transformations such as the Top-hat transformation can be used. A Top-hat transformation essentially subtracts a morphologically smoothened version of an image from itself, thereby highlighting finer details.

16. Ear Detection Face-geometry-based detection Since in a profile image the nose can be easily detected as the point with high curvature, it is possible to constrain the search for the ear in an appropriate location relative to the nose.

17. Ear Recognition Subspace analysis-based techniques projecting the ear image onto a set of principal directions is an effectiveway to obtain a salient and compact representation of an ear. Subspace projection techniques such as PCA, ICA, and LDA have been successfully used in literature for matching ear images.

18. Ear Recognition Sparse-representation-based techniques Optimization techniques that minimize L-1 norm of the distance vector between the transformed query and all the transformed templates in a database have been shown to provide high recognition accuracy in object recognition studies. This technique has also been successfully used for ear recognition.

19. Ear Recognition Point-set-matching-based techniques Scale Invariant Feature Transform (SIFT) is a well known technique for matching two images where a set of salient points can be reliably and repeatably extracted from them. In order to match two images using SIFT features, corner points are detected from two images and matched based on image gradient-based features extracted from the neighborhood region of each point.

20. Ear Recognition Point-set-matching-based techniques

21. Ear Recognition Geometric-measurements-based techniques Features obtained by measuring certain geometric characteristics of the ear can also be used as a set of discriminative features. As an example, the centroid of an ear image obtained from its edge image can be used as a center to draw concentric circles with pre-specified radii. Various measurements, such as number of points on a circle intersecting the edge image or distance between two consecutive intersections, can be used as a feature vector.

22. Ear Recognition Transformation-based techniques Various image transformation techniques such as Fourier transform or wavelet transform can also be applied to extract discriminative features from an ear image. Fourier transform can also be applied in order to obtain a rotation and translation invariant representation of the ear

23. Ear Recognition 3D techniques In some scenarios, acquiring a 3D rendition of the ear entity may be possible. 3D images offer depth information that can be used in conjunction with the 2D texture information to improve the recognition accuracy. In the case of 3D ear images, local histograms of shape curvature values can be used to match two ear images.

24. 3 types of ear biometrics biometrics from photo of ear: most researched, “ear marks”: interesting in crime detection thermogram pictures: a new possibility

25. Methods using pictures (1/4) Moreno et al. (1999) multiple identification method, which combines the results from several neural classifiers using feature outer ear points, information obtained from ear shape and wrinkles, and macro features extracted by compression network introduce three different classification techniques for outer ear or auricle identifying.

26. Methods using pictures (2/4) Burge and Burger (1998, 2000) automating ear biometrics with Voronoi diagram of its curve segments. a novel graph matching based algorithm for authentication, which takes into account the possible error curves, which can be caused by e.g. lightning, shadowing and occlusion.

27. Methods using pictures (3/4) Hurley, Nixon and Carter (2000a, 2000b) force field transformations for ear recognition. the image is treated as an array of Gaussian attractors that act as the source of the force field according to the researchers this feature extraction technique is robust and reliable and it possesses good noise tolerance.

28. Methods using pictures (4/4) Victor, Chang, Bowyer, Sarkar (at least 2 publications in 2002 and 2003) principal component analysis approach comparison between ears and faces This method is presented later with 2 cases.

29. Earmarks when the ear is pressed against some hard material, e.g. glass, the result is called earmark used in forensic research at least 4 criminals are judged because of earmarks in decade of 1980 in England and in The Netherlands later it was decided by a Dutch court that earmarks are not unique and reliable enough

30. Ear thermogram Thermogram of an ear. Image provided by Brent Griffith, Infrared Thermography Laboratory, Lawrence Berkeley, National Laboratory. (Burge et al., 1998) used for removing “non-ear”, e.g. hair and other obstacles in this case hair is between 27.2 and 29.7ºC while the outer ear areas range from 30.0 to 37.2 ºC

31. Case 1: an evaluation of face and ear biometrics Done by Victor, B., Bowyer, K., and Sarkar S, published in Proceedings of International Conference on Pattern Recognition, August 2002 The used method is principal component analysis (PCA) and the design principle is adopted from the FERET methodology Null hypothesis: there is no significant performance difference between using the ear or face as a biometric

32. PCA Method

33. Points for normalization

34. Tests of research by Victor et al. For faces: 1. Same day, different expression 2. Different day, similar expression 3. Different day, different expression For ears: 1. Same day, opposite ear 2. Different day, same ear 3. Different day, opposite ear

35. Same day, different expression or opposite ear ear

36. Different day, similar expression or same ear ear

37. Different day, different expression or opposite ear ear

38. Victor et al. research result Experiment #Expected ResultResult 1Same day, different expression Same day, opposite ear Greater variation in expressions than ears; ears perform better Face performs better 2Different day, similar expression Different day, same ear Greater variation in expression across days; ears perform better Face performs better 3Differet day, different expression Different day, opposite ear Greater variation in face expression than ear; ears perform better Face performs better Face/Ear compared

39. Case 2: Ear and Face images ”Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics” by Chang, K., Bowyer, K.W., Sarkar, S., and Victor, B. Published in IEEE Transactions on Pattern Analysis and Machine Intelligence on September 2003. Hypothesis: (i) ear provide better biometric performance than images of the face and (ii) exploring whether a combination of ear and face images may provide better performance than either one individually

40. Images used in research by Chang et al. (2003) Same kinds of sets for faces, too. PCA, FERET

41. Tests for the research 1. day variation  other conditions constant 2. different lightning condition  taken in the same day in the same session 3. pose variation  22.5 degree rotation, other conditions constant, taken in the same day Combined face, ear and face+ear What do you think the result was?

42. Day variation test

43. Different lightning conditions

44. Pose variation (22.5 degree rotation)

45. Results In this research face biometrics seem to be better in constant conditions, ear biometrics in changing conditions Multimodal biometrics face plus ear gives the best results, why not use them?

46. Ear shape applications currently there are no applications, which use ear identification or authentication crime investigation is interested in using ear identification active ear authentication could be possible in different scenarios

47. Application scenarios Application scenarios are quite traditional for biometrics: collecting child from daycare ATM any other active identification passive e.g. when trying to solve a crime and there’s a picture of ear in the tape of a surveillance camera e.g. gas station robbery in The Netherlands

48. Biometric suitability for authentication purpose Bromba GmbH

49. Error possibilities in ear recognition

50. False rates in ear identification FRR = false reject rate FAR = false acceptance rate Normally one of these is tried to minimize.

51. Improving the FRR with ear curve widths, an example width of an ear curve corresponding to the upper Helix rim  better results

52. Removal of noise curves in the inner ear Graph model (Burge et al.) and false curves because of e.g. oil and wax of the ear.

53. Possibilities to enhance ear biometrics Using accurate measurements, e.g. ear curve and upper helix rim Removing noise curves Thermograms  removal of obstacles Better quality cameras  more accurate pictures Combined biometrics

54. Challenges in ear recognition Large-scale public evaluation of ear recognition has not been conducted. There are no commercial ear recognition system The performances of ear recognition algorithms have been tested on some standard ear datasets. Experiments suggest that ear images obtained under controlled conditions can result in good recognition accuracy

55. References (1/2) Bamber, D. Prisoners to appeal as unique ‘earprint’ evidence is discredited. Telegraph Newspaper (UK). Updated 02/12/2001 [Retrieved October 3, 2003] From: http://portal.telegraph.co.uk/news/main.jhtml?xml=/news/2001/12/02/nearp02.xml Bromba GmbH, Bioidentification Frequently Asked Questions. Updated 2003-09-12 [Retrieved October 28, 2003] From: http://www.bromba.com/faq/biofaqe.htm. Burge, M. and Burger, W. Ear Biometrics. In A. Jain R. Bolle and S. Pankanti, editors, BIOMETRICS: Personal Identification in a Networked Society, pp. 273-286. Kluwer Academic, 1998. Burge, M. and Burger, W. Ear Biometrics in Computer Vision. In the 15 th International Conference of Pattern Recognition, ICPR 2000, pp. 826-830. Carreira-Perpinan, M.A. Abstract from MSc thesis Compression neural networks for feature extraction: Application to human recognition from ear images, Technical University of Madrid. 1995. [Retrieved October 16, 2003]. From http://www.dcs.shef.ac.uk/~miguel/papers/msc-thesis.html. Chang, K., Bowyer. K.W., Sarkar, S., Victor, B. Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, September 2003, pp. 1160-1165. Forensic Evidence News: Ear Identification. Version updated Nov. 1, 2000. [Retrieved October 3, 2003] From http://www.forensic-evidence.com/site/ID/IDearNews.html.

56. References (2/2) Hoogstrate, A.J., Van den Heuvel, H., Huyben, E. Ear Identification Based on Surveillance Camera’s Images. Version updated May 31, 2000. [Retrieved October 7, 2003] From: http://www.forensic- evidence.com/site/ID/IDearCamera.html. Hurley, D.J., Nixon, M. S., Carter, J.N. Automated Ear Recognition by Force Field Transformations in Proceedings IEE Colloquium: Visual Biometris (00/018), 2000a, pp. 8/1-8/5. Hurley, D.J., Nixon, M.S., Carter, J.N. A New Force Field Transform for Ear and Face Recognition. In Proceedings of the IEEE 2000 International Conference on Image Processin ICIP 2000b, pp. 25- 28. Jain, A., Hong, L., Pankati, S. Biometric Identification. Communications of the ACM, February 2000/Vol. 43, No. 2, pp. 91-98. Moreno, B., Sánchez, Á., Vélez. J.F. On the Use of Outer Ear Images for Personal Identification in Security Applications. IEEE 33 rd Annual International Carnahan Conference on Security Technology, 1999, pp. 469-476. Morgan, J. Court Holds Earprint Identification Not Generally Accepted In Scientific Community, State v. David Wayne Kunze. 1999. [Retrieved September 9, 2003]. From: http://www.forensic- evidence.com/site/ID/ID_Kunze.html Ratha, N.K., Senior, A., Bolle, R.M. Automated Biometrics in Proceedings of International Conference on Advances in Pattern Recognition, Rio de Janeiro, Brazil, March 2001. Victor, B., Bowyer, K., Sarkar, S. An evaluation of face and ear biometrics in Proceedings of International Conference on Pattern Recognition, pp. 429-432, August 2002.

57. Questions? Thank you for your attention! G.ŞENGÜL