05/12/03B. Dorizzi On-line Signature Identity Verification Bernadette Dorizzi, GET/INT, 9 rue Charles Fourier, Evry

05/12/03B. Dorizzi Outline Generalities Preprocessing Feature extraction Models : Cooperation local/global : Kashi 98 DTW (Jain : 2002) HMM (Rigoll, Dolfing, Salicetti) Evaluation : Signature Competition at Conf SVC 2004

05/12/03B. Dorizzi On-line signatures Acquisition on an electronic tablet or with a special pen, able to record a sequence of points (speed and pression of the signature, not only the static image) Interest : behavioral more than physiological, difficult to imitate. Highly variable intra-class characteristics : enrollment will necessitate several samples of the signature

05/12/03B. Dorizzi Recognition General scheme Sequence of points Sequence of features A signature of a claimed client X Learning Use of the samples of the signature of X to create a model of X The signature is presented at the input of the model of X and a similiraty measure is computed. Comparison to a threshold allows to accept or discard the signature

05/12/03B. Dorizzi Signature samples

05/12/03B. Dorizzi Performance evaluation Two types of errors FR=False Rejection FA=False Acceptation FRR= Nb of FR Nb of clients FAR= Nb ofFA Nb of imposteurs TER= Nb of FR+Nb of FA Total acces Nb

05/12/03B. Dorizzi Performance curves FRR High security ROC curveIn order to make a decision a threshold has to be settled EER: Equal Error rate FAR Low security FRR FAR EER (equal error rate) Threshold

05/12/03B. Dorizzi Data acquisition Depends on the capabilities of the hardware –High-end tablet : robust pressure sensibility, precise pen pressure measure, measure of the pen orientation –PDA : only coordinates and information on pen-up, pen-down Coordinates : x(t),y(t) Pressure p(t) Orientation  (t),  (t)

05/12/03B. Dorizzi Preprocessings Resampling and smoothing of the trajectory –Between 100 and 150 points per second (too many points, noise) Low-pass Filtering (the low frequencies carry the information) Apparent contradiction : point spacing on the trajectory. If irregular (dependant on the speed of the signing process) one capture the speed. But, in some parts of the trajectory there are very few points, thus little spatial information. (cf. Jain cf comprise between the 2).

05/12/03B. Dorizzi Local and contextual features Speed in x and y direction Acceleration in x et y direction Tangential Acceleration Cosine et sine of angle  : dynamical parameters The signature is considered as a sequence of points. A vector of features is computed at each point of the trajectory

05/12/03B. Dorizzi Cosine and sine of the angle which estimates the  (t) variation : Contextual (shape) parameters

05/12/03B. Dorizzi Global Features (Kashi et al., IJDAR 98) Mixture of both shape and dynamical features –2 time-related features : total signature time, ratio of pen-down time to total time –6 other dynamic features depends on the writing velocity and acceleration –13 shape-related features The signature is considered as a whole

05/12/03B. Dorizzi

05/12/03B. Dorizzi Dolfing approach Philips Research Laboratory The signature is split into different portions (part of the trajectory between 2 values of de v y =0) To each portion is associated a vector of 32 features The signature is considered as a sequence of points, these points are regrouped in several sub-parts. A feature vector is associated to each sub-part.

05/12/03B. Dorizzi Feature Description –13 spatial features –13 dynamic features –6 contextual features Spatial features: –sin and cos of the starting and ending angles : thetastart, thetaend –3 intermediate angles –Aspect ratio –La curvature –Existance of a pen-up

05/12/03B. Dorizzi Figure associated to the spatial features

05/12/03B. Dorizzi Dynamic features Number of samples nt min, max, moy of speed v acceleration a pressure p variation of pressure delta p vmax-vmoy pen-tilt with 2 angles

05/12/03B. Dorizzi Contextual features Sin et cos of angles psi1, psi2, psi3 which are the angles of the 3 lignes with x axe which start from the gravity center of the current portion towards the gravity center of the 3 preceeding segments. seg1 seg2 seg3 Seg courant

05/12/03B. Dorizzi Several Models of the signatures and associated similarity measures « A Hidden Markov Model approach to online handwritten signature verification », Kashi et al, IJDAR Computation of a global distance between 1 signature and a set of references signatures of writer i. « On-line signature verification », Jain et al., Pattern Recognition, DTW to compare two signatures considered as 2 sequences of features. Rigoll, Dolfing, Salicetti etc… : Modelization by a HMM of each writer (several signatures considered as sequences of features are considered) : computation of a likelihood measure for a signature to be produced by the HMM of writer i

05/12/03B. Dorizzi Global feature-based verification Kashi et al. A signature model for entrant i is a set of means  and standard deviations , obtained during training from 6 instances of signatures Error measure E i for a given signature claimed to be that of i: N is the total number of global features M i,k is the value of the K-th feature of the signature to verify  i,k and  i,k are the mean and standard deviation of feature k over the reference set of i.

05/12/03B. Dorizzi Dynamic Time Warping Local features are computed at each point of the trajectory A signature = a string (sequence of feature vectors) A signature model for a person is composed of 3 different samples of the signature String matching (DTW Dynamic Time Warping) allows the comparison of strings of different lengths. Finds an alignment between the points in the 2 strings such that the sum of the differences between each pair of aligned points is minimal To find the minimal difference, all possible alignments must be investigated. Dynamic programming is a method to implement that

05/12/03B. Dorizzi Mise en comparaison de 3 signatures d’une même personne S 1, S 2, S 3

05/12/03B. Dorizzi Verification A test signature is compared to the model of signer i (represented by 3 signatures). 3 possible strategies : minimum of all the dissimilarity values, average of all the dissimilarity values, maximum of all the dissimilarity values Decision : comparison of this value to a threshold The threshold can be identical for all the writers or set individually for each writer. In this article, no forgeries data is used to calculate the thresholds.

05/12/03B. Dorizzi Writer Modelization by Hidden Markov model

05/12/03B. Dorizzi What is a HMM? Non deterministic automata with one or several states A double stochastic process A Markov chain representing the states of the HMM: S = {S 1, S 2, S 3,……S N } A process which induces a sequence of observations

05/12/03B. Dorizzi What is a HMM? state 1state 2state 3 O = (O 1,..., O t,...)

05/12/03B. Dorizzi Why a HMM? The different signatures of a same writer are variable. This variability will be well modelized by a HMM. This modelization will allow to consider a non stationary signal (the signature) as a piecewise-stationary signal.

05/12/03B. Dorizzi Components of a HMM N: number of states in the model: S = {S 1, S 2,……,S N } A: Matrix of probability transitions a ij =P[q t+1 =S j |q t =S i ], 1  i, j  N Initial distribution of the states:  i = P[q 1 = S i ], 1  i  N Emission law of the observations in each state B j (O t )=P[O t |q t =S j ], 1  j  N Discrete HMM: B j is a matrix Continuous HMM: B j is a mixture of gaussian probalility density functions

05/12/03B. Dorizzi Markovian modelization of a signature

05/12/03B. Dorizzi Learning phase A process allowing the reestimation of the parameters of the HMM, in order to maximize the loglikelihood of the true signatures.

05/12/03B. Dorizzi Signature Verification Comparison of the loglikelihood of the signature, knowing the HMM model of the writer, with a threshold in order to take the decision «Distance» decision threshold –Accept if |Log(P(S| ) - L mean )|< , otherwise reject where L mean = mean loglikelihood on the learning database of the declared i client. MMC of the writer SignatureLog-likelihood

05/12/03B. Dorizzi Systems Evaluation Difficult because of the non availability of common databases : A lot of « home-made » databases with no connection between them (between 9 and 100 individuals). In general the EER lies between 1% and 6% Evaluation in presence of forgeries of more or less good qualities (skilled, over the shoulder, random, rough etc…) For instance, the Philips data base (very difficult due to the presence of high quality imitations, including dynamics) 1500 true signatures sur 51 persons, 1470 imitations « over the shoulder » 1530 imitations « home enhanced » 240 professional imitations Non identical evaluation protocols : personal threshold versus global one The threshold is generally determined in order that FAR=FRR (EER Equal Error Rate) or in order to minimize TER (Equal Error Rate) on a development database (some signers that will not be considered in the test base) using forgeries.

05/12/03B. Dorizzi SVC 2004 First International Signature Verification Competition –In conjunction with the First International Conference on Biometric Authentication in Hong Kong (ICBA 2004) Two tasks: –Coordinate input only –Coordinate, pen orientation and pressure inputs Database for each task:100 writers –Training set: 5 among 20 genuine signatures per writer –Evaluation set: Unknown genuine signatures per writer 20 skilled forgeries from 5 other contributors

05/12/03B. Dorizzi Conclusion Signature scan : a quite variable modality, resistant to forgeries, well accepted, but not suitable for each person Not so many applications : Natural with PDA, in banking contexts Some tools already available : smartpen, etc…

05/12/03B. Dorizzi References J.G.A. Dolfing, "Handwriting recognition and verification, a Hidden Markov approach", Ph.D. thesis, Philips Electronics N.V., M. Fuentes, S. Garcia-Salicetti, B. Dorizzi "On-line Signature Verification : Fusion of a Hidden Markov Model and a Neural Network via a Support Vector Machine", IWFHR8, Août J. Ortega-Garcia, J. Gonzalez-Rodriguez, D. Simon-Zorita, S. Cruz-Llanas, "From Biometrics Technology to Applications regarding face, voice, signature and fingerprint Recognition Systems", in Biometrics Solutions for Authentication in an E-World, (D. Zhang, ed.), pp , Kluwer Academic Publishers, July A. Jain, F D. Griess, S.D. Connell « On-line signature verification », Pattern Recognition,, vol 35, pp , 2002 J.G.A. Dolfing, "On-line signature verification with Hidden Markov Models", Proc. of ICDAR, pp , R. Kashi, J. Hu, W.L. Nelson, W. Turin, "A Hidden Markov Model approach to online handwritten signature verification", Intl. J. on Document Analysis and Recognition, Vol. 1, pp , G. Rigoll, A. Kosmala, "A systematic comparison of on-line and off-line methods for signature verification with Hidden Markov Models", Proc. of ICPR, pp , 1998.