1. Multiple Biometrics for Personal Verification/Identification
Chaur-Chin Chen
Department of Computer Science
Institute of Information Systems & Applications
National Tsing Hua University
Hsinchu 30013, Taiwan

2. Outline What is Biometrics? Motivation by Evidence
Iris Image Pattern Analysis
Handwriting/Handprinting Verification
Personal Signature Verification
Hand Geometry Verification
Voice (Speech) Pattern Recognition
Face Image Recognition
Fingerprint Image Verification/Identification
Palmprint, Ear shape, Gesture, …
Fingerprint Classification and Verification
Opportunities and Challenges

3. What and Why is Biometrics?
What is Biometrics?
Biometrics is the science and technology of interactively measuring and statistically analyzing biological data, in particular, taken from live people.
Why Biometrics?
(1) The banking industry reports that false acceptance rate (FAR) at ATMs are as high as 30%, which results in financial fraud of US$2.98 billion a year.
(2) In U.S., nearly half of all escapees from prisons leave through the front door, posing as someone else.
(3) Roughly 4000 immigration inspectors at US ports-of-entry intercepted and denied admission to almost 800,000 people. There is no estimate of those who may have gotton through illegally.
(4) Personal verification/identification becomes a more serious job after the WTC attack on September 11, in the year 2001.
The evidence indicates that neither a PIN number nor a password is reliable.

4. Some Biometric Images

5. 美國啟用出入境指紋及影像辨識系統
美國國土安全部基於安全考慮，自(2004)元月五日起，啟用數位化出入境身分辨識系統(US-VISIT)，大部分來美的14歲至79歲旅客，包括來自台灣、大陸、香港的留學生，於進入美國國際機場及港口時，都要接受拍照及留下指紋掃描紀錄以便辨識查核。(27個免簽證國公民之入境待遇略有不同，短期來美者，將受豁免。)，亦將需接受指紋掃描查核。 

6. US-VISIT
US-VISIT currently applies to all visitors (with limited exemptions) holding non-immigrant visas, regardless of country of origin.
2004 – US$ 330 million
2005 – US$ 340 million
2006 – US$ 340 million
2007 – US$ 362 million

7. 入境按指紋 日本11月將實施 日本入境排隊長 指紋掃瞄會更長！ (2007年9月27日)
入境日本將按指紋 日官員赴台宣導新措施 (2007年9月27日)
日11月20日實施外國人入境須按指紋臉部照片 (2007年9月25日)
入境按指紋 日本11月將實施 (2007年9月2日)

8. Implementation of An Automatic Fingerprint Identification System
Peihao Huang, Chia-Yung Chang, Chaur-Chin Chen
Department of Computer Science
National Tsing Hua University
Hsinchu 30013, Taiwan
Presented on May 19, 2007 in IEEE Int’l Conference on EIT

9. A Typical Fingerprint Image

11. Outline A Flowchart of Fingerprint Identification
♪ Type Classification
♪ Minutia Points Detection
♪ Minutiae Pattern Extraction
♪ Pattern Matching
♦ Databases: Rindex28, Lindex101
♦ Experimental Results

12. Flowchart of An AFIS

13. Image Enhancement
Support that A(i, j) is image gray level at pixel (i, j), μ and s2 are the mean and variance of gray levels of input image, and α=150, γ=95, γ must satisfy γ>s.
The enhanced image B( i , j ) is obtained by a contrast stretching given below
B( i , j ) α + γ ＊ ([A ( i , j ) – μ]／s)

14. Result of Image Enhancement

15. Orientation Computation (1/4)
First we apply a 5 by 5 median filter on the image to avoid false gradient vectors generated by noise.
Then compute the gradient (Gx,Gy) at each pixel by a Sobel operator.
mask Sobel Operation z1 z2 z3 z4 z5 z6 z7 z8 z9

16. Orientation Computation (2/4)
The relationship between [Gx, Gy]T and [ρ,θ]T

17. Orientation Computation (3/4)
Because of opposite gradient vectors might offset each other, we double the angles of the gradient vectors before averaging each block, and let the length of the gradient vectors be squared [5][6][10][11]
Let be represented by
The average gradient in each block R (w×w) is

18. Orientation Computation (4/4)
The block gradient direction ψ is defined as
where (a block direction) is defined as
for

19. Results of Block Orientations
Block orientated images

20. Singular Points Detection (1/2)
Because of noisy directions, we have to smooth the direction before computing the Poincaré index.
We regard the direction as a vector, double the angles and use a 3 by 3 averaging filter to smooth the direction.
The average direction of the block is B3 B2 B1 B4 Bc B0 B5 B6 B7 1 2

21. Singular Points Detection (2/2)
We compute Poincaré index by summing up the difference in the direction surrounding the block P. For each block Pj, we compute the angle difference from 8 neighboring blocks along the counter-clockwise directions. P1 P8 P7 P2 P P6 P3 P4 P5
P1 → P2 → P3 → P4 → P5 → P6 → P7 → P8 → P1
Core if the sum of difference is 180°
Delta if the sum of difference is -180°

22. Example of An Ideal Core/  \ | •

23. Detected Singular Points (Blocks)
Singular points of Fingerprint Images

24. Criteria for Type Classification
Arch
(tented arch)
Left loop
Right loop
Whorl
(twins loop)
Others
# of cores
0 or 1 1 2
0 or >2
# of deltas
(middle)
1(right)
1(left)
0~2

25. 4+1 Fingerprint Type Classification
Arch Left Loop Right Loop Whorl miscellaneous

26. Fingerprint Database (1)
Rindex28
Rindex28, is obtained from PRIP Lab at NTHU. It contains 112 images of size 300 by 300 contributed by 28 different individuals. Each contributed 4 times with the same right index finger scanned by a Veridicom FPS110 live scanner with the resolution 500 dpi

27. Fingerprint Database (2)
Lindex101
Lindex101, is obtained from PRIP Lab at NTHU. It contains 404 images of size 300 by 300 contributed by 101 different individuals. Each contributed 4 times with the same left index finger scanned by a Veridicom FPS110 live scanner with the resolution 500 dpi

28. Results of Classification
Experiment on Rindex28: 4x28 right index fingerprint images collected from 28 students
No classification error
Experiment on Lindex101: 4x101 left index fingerprint images are collected from 101 students
17 classification errors
Due to inappropriately pressing, too complex structure?, or poor quality.

29. Inappropriately Pressing
Right Loop Arch ?

30. Inappropriately Pressing
Whorl ? Left Loop ?

31. Too Complex Structure ?
Left Loop or Whorl ? Left Loop or Arch ?

32. Fingerprint Images of Poor Quality
? X ? X

33. Outline A Flowchart of Fingerprint Identification
♪ Type Classification
♪ Minutia Points Detection
♪ Minutiae Pattern Extraction
♪ Pattern Matching
♦ Databases: Rindex28, Lindex101
♦ Experimental Results

34. Flowchart of An AFIS

35. Image Binarization (1/2)
We have to distinguish valley from ridge of a fingerprint image before smoothing and thinning. So the gray value of pixels in the enhanced fingerprint image will be binarized to 0 or 255.
First we compute the gray value of P25 and P50 from the enhanced image, where Pk is the kth percentile of enhanced fingerprint image histogram.
Then we partition an enhanced fingerprint image into w by w blocks and compute the mean of each blocks. We define that Mj is the mean of the j-th block.

36. Image Binarization (2/2)
If the gray value of pixel Si is less than P25, we assign 0 to Si . If the gray value of pixel Si greater than P50, we assign 255 to Si . Otherwise, the pixel value is defined by the following rule:

37. Post-Processing (Smoothing)
After binarization, we find that there is still much noise on ridge region. In order to make the result of thinning better, we have to smooth the fingerprint image first. A smooth stage uses neighboring pixels to remove noise.
First a 5 by 5 filter is used. The pixel pi is assigned by:
pi = { 255 if Σ5x5Nw≧18
0 if Σ5x5Nb≧18
pi otherwise
Then a 3 by 3 filter is further proceed by:
pi = { 255 if Σ3x3Nw≧5
0 if Σ3x3Nb≧5

38. (c) Binarization image (d) Smoothed image
(a) Original image (b) Enhanced image
(c) Binarization image (d) Smoothed image

39. Thinning [9]
The purpose of thinning stage is to gain the skeleton structure of a fingerprint image.
It reduces a binary image consisting of ridges and valleys into a ridge map of unit width.
(d) Smoothed image (e) Thinned image

40. Minutiae Definition
♫ From a thinned image, we can classify each ridge pixel into the following categories according to its 8-connected neighbors.
♫ A ridge pixel is called :
an isolated point if it does not contain any 8-connected neighbor.
an ending if it contains exactly one 8-connected neighbor.
an edgepoint if it has two 8-connected neighbors.
a bifurcation if it has three 8-connected neighbors.
a crossing if it has four 8-connected neighbors.

41. Example of Minutiae Extraction

42. Spurious Minutiae Deletion
Spurious minutia pixels include:
(a) endings that lie on the margins of the region of interest.
(b) two “close” endings with the same ridge orientation.
(c) an ending and a bifurcation that are connected and close enough.
(d) two bifurcations that are too close.

43. Spurious Minutiae Elimination
Due to broken ridges, fur effects, and ridge endings near the margins of an image, we have to remove the spurious minutiae as described below.
(1) Two endings are too close (within 8 pixels)
(2) An ending and a bifurcation are too close (< 8 pixels)
(3) Two bifurcations are too close (< 8 pixels)
(4) Minutiae are near the margins (< 8 pixels)

44. Region of Interest Detection (1/2)
To avoid obtaining false singular points or minutiae, we use mean and standard deviation in each block to determine if the block is “good” (not a marginal block) or not.
where , and is the ratio of distance to the center of the fingerprint image. μ and σ are normalized to be in [0,1].
If v > 0.8, the block is what we want.

45. Region of Interest Detection (2/2)
Enhanced image Region of interest

46. Example of Minutiae Extraction

47. Example of Minutiae Extraction

48. Example of Minutiae Extraction

49. Minutiae Pattern Matching

50. Minutiae Pattern Matching

51. Minutiae Pattern Representation
Minutiae Pattern Representation
Fingerprint Template Data
The information format of fingerprint template data.
Type
#of cores
Core*
# of deltas
Delta*
# of minutiae
Minutiae*
4 bits
2 bits
24 bits
7 bits
26 bits
The information format of singular points, core or delta.
X Coordinate
Y Coordinate
Direction
10 bits
4 bits
The information format of a minutia.
Kind of Minutiae
X Coordinate
Y Coordinate
Direction
2 bits
10 bits
4 bits

52. Fingerprint Matching Score
The matching score of these two fingerprints is calculated by
where M is the number of potential type-matching minutiae within a disk of a certain user-specified radius, R (12 pixels). r measures the distance between a pair of potentially matched minutiae points.

53. Fingerprint Database (1)
Rindex28
Rindex28, is obtained from PRIP Lab at NTHU. It contains 112 images of size 300 by 300 contributed by 28 different individuals. Each contributed 4 times with the same right index finger scanned by a Veridicom FPS110 live scanner with 500 dpi

54. Fingerprint Database (2)
Lindex101
Lindex101, is obtained from PRIP Lab at NTHU. It contains 404 images of size 300 by 300 contributed by 101 different individuals. Each contributed 4 times with the same left index finger scanned by a Veridicom FPS110 live scanner with 500 dpi

55. FVC2000 Fingerprint Database (3)
Sensor Type
Image Size
Resolution
DB1
Low-cost Optical Sensor
300x300
500 dpi
DB2
Low-cost Capacitive Sensor
256x364
DB3
Optical Sensor
448x478
DB4
Synthetic Generator
240x320
about500 dpi

56. Fingerprint Images from FVC2000
Examples of fingerprint images from each database of FVC2000

57. Experimental Results The experimental results of 6 databases Rindex28
Rindex28
Lindex101
DB1
DB2
DB3
DB4
Recognition
rate
99.11%
111/112
82.67%
334/404
92.50%
74/80
90.00%
72/80
87.50%
70/80
Enrolling time
for each fingerprint image
0.25 sec
0.45 sec
0.17 sec
Matching
time
0.359 sec
3.14 sec
0.218 sec
0.234 sec
0.156 sec
The experimental results of 6 databases

58. Mismatched Fingerprint Images

59. Conclusion
€ We reveal three problems, which affect the results of an AFIS which merit further studies.
(1) Noise produces the poor binarization results
(2) Broken ridges result in the false orientation, which causes the
misclassification of a fingerprint type
(3) The shifted fingerprint image is difficult to match the minutiae
pattern well, for example, the type misclassification due to the
missing cores or deltas

60. Are They From the Same Person?

62. SVD-Based Projection for Face Recognition
Chou-Hao Hsu and Chaur-Chin Chen
Department of Computer Science
Institute of Information Systems & Applications
National Tsing Hua University, Hsinchu,Taiwan 30013

63. Training Face Images
Let Fi(j) be the ith face image of m by n from the jth subject, 1≦i≦Nj and 1≦j≦K,
N1+N2+….+Nk=N be the training face images. Define the mean image S as

64. Singular Value Decomposition S=UDVt
Do S=UDVt where U and V are orthogonal.
Select r,c with r≦m, c≦n such that
d11+d22+…+dhh ≧ 85% of trace(D), where h=min{r,c}
Let Ur =[u1,u2,...,ur], Vc =[v1,v2,…,vc]
Where U is an m by m orthogonal matrix
V is an n by n orthogonal matrix

65. Convert a face image into features
For each training image Ak, we represent this Ak as xk =(Ur)tAVc , an r by c feature image
For each test image T, we represent T by
y=(Ur)tTVc , an r by c feature image

66. Distance between training feature images and a test feature image
Compute d(y,xk) by Fröbenius norm
The smaller Fröbenius norm, the closer
Rank the norms in an ascending order
Determine the recognition rates from ranks 1, 2, 3, ...,8 and plot the curve

67. Part of 5*40 Training Face Images

68. Missed Face Images and Their Wrongly-Best Matched Images

69. A Comparison of Difference Projection Methods

70. Challenges and Opportunities
A perfect biometric recognition system did not exist and will never exists
An application based on biometrics usually requests a perfect verification/identification
A collection of biometric data is usually time consuming and more or less intrudes personal privacy
The mechanism of achieving the trade-off between privacy and security merits studies.

71. CC1017