1. Visual-based ID Verification by Signature Tracking
Mario E. Munich and Pietro Perona
California Institute of Technology

2. Outline Biometric ID Visual Signature Acquisition
Description of the system
Tracking results
Signature Verification
Dynamic Time Warping
Experimental Results
Conclusions and further work
AVBPA99 - March 23rd, 1999

3. Biometric ID
Identification based on measurements of human biological characteristics
Vision-based:
Face Recognition [Taylor et. al.,Turk & Pentland, Wiskott et.al.]
Fingerprint Recognition [Jain et. al.]
Iris Scanning [Daugman]
Retina Scanning
...
Other:
Voice Recognition
Signature Verification
AVBPA99 - March 23rd, 1999

4. Why visual signature acquisition?
tablet digitizer
screen
keyboard
Current I/O computer
interfaces have limitations
for decreasing their size.
Cell phone + PDA
PDA
mouse
Future new I/O computer
interfaces will involve Audio
and Visual techniques.
camera
microphone
Advantages:
smaller size implementing them in VLSI.
more natural way for people to communicate
with computers.
AVBPA99 - March 23rd, 1999

5. Visual signature acquisition
AVBPA99 - March 23rd, 1999

6. Visual signature acquisition
Preprocessing
Pen Tip
Tracker
Ballpoint
Detector
Filter
Signature
Verification
True!
AVBPA99 - March 23rd, 1999

7. Preprocessing Get the template of the pen tip
Mouse-clicking (manual)
Pen familiar to the system
Unknown pen
Get the portion of image where the pen tracker looks for the pen tip
(size = 25x25 pixels)
(size = 31x31 pixels)
AVBPA99 - March 23rd, 1999

8. Pen tip tracker
Portion of the image
extracted in order to
compute correlation
Location of maximum
correlation
Predicted position
of the pen tip
Pen tip template
Predicted position
of the pen tip
The most likely position of the pen tip is given by the location of
maximum correlation
AVBPA99 - March 23rd, 1999

9. Filter Predicts the position of the pen tip in the following image
Speeds up computations
Smoothes out the trajectory
Estimates the position of the pen tip for missing frames
Model of pen tip’s dynamics
where:
x(t): 2D pen tip’s position
v(t): 2D pen tip’s velocity
a(t): 2D pen tip’s acceleration
y(t): location of max. correlation
AVBPA99 - March 23rd, 1999

10. Real Time Implementation
Camera
Pentium 230
Frame
grabber
PCI Bus
The frame grabber is a PXC200 from Imagination.
The system runs at 60 Hz with a total processing time of 15ms per frame.
No calibration needed.
AVBPA99 - March 23rd, 1999

11. Acquired signatures Example signature Prototype Forgery
AVBPA99 - March 23rd, 1999

12. Signature Verification
Off-line Signature Verification
Works on a static image of the signature, i.e., the result of the act of signing.
On-line Signature Verification
Works on the dynamic process of generation of the signature, i.e., the action of signing itself.
(“Automatic signature verification and writer identification, the state of the art”, by M.Parizeau and R. Plamondon is a good survey paper)
AVBPA99 - March 23rd, 1999

13. Signature Verification
New signature
True signature
or forgery?
Comparison
Prototype
Training set
AVBPA99 - March 23rd, 1999

14. Signature Verification
Previous work on Signature Verification using Dynamical Time Warping:
Sato and Kogure, 1982
Parizeau and Plamondon, 1990
Huang and Yang, 1995
Wirtz, 1995
Nalwa, 1997
Munich and Perona 98, 99
...
AVBPA99 - March 23rd, 1999

15. Signature Verification
Comparison of two signatures
Elementary distance of matching P1(1(t-1)) with P2(2(t-1)) and P1(1(t)) with P2(2(t)).
P1(1(t))
Signature 1 (C1)
P1(1(t-1))
v(t)
v(t-1)
P2(2(t))
Matching function
P2(2(t-1))
Signature 2 (C2)
Similarity measure
(“distance” between
C1 and C2)
AVBPA99 - March 23rd, 1999

16. Dynamical Time Warping(DTW)
Given C1 = (P1(1),…,P1(T1)), C2 = (P2(1),…,P2(T2)) and the distance function D(C1,C2), DTW finds a warping function  = [1, 2]T that minimizes the dissimilarity between C1 and C2:
with the following recursion [Bellman et.al.,1957, Sakoe & Shiba,1978]:
Cumulated distance
up to step t
Cumulated distance
up to step (t-1)
Elementary distance added by matching P1(1(t-1)) with P2(2(t-1)) and P1(1(t)) with P2(2(t)).
AVBPA99 - March 23rd, 1999

17. Dynamical Time Warping
2(t) = j
Warping plane 1 2
Recursion (for the discrete case):
1(t) = i
2(t) = j
Local slope
constraints
AVBPA99 - March 23rd, 1999

18. Dynamical Time Warping
Maximum deviation from
linear warping constraint 2
Dynamical Time
Warping solution
2(t)
Global slope constraint 1
1(t)
The algorithm is O(N) both in time and spatial complexity.
Constraints could be used in order to reduce the complexity
since they will reduce the number of nodes to be explored.
AVBPA99 - March 23rd, 1999

19. Dynamical Time Warping
x(t)
y(t)
2 examples from s05
x(t) after DTW
y(t) after DTW
Alignment path
AVBPA99 - March 23rd, 1999

20. Dynamical Time Warping
x(t)
y(t)
2 examples from s030
x(t) after DTW
y(t) after DTW
Correspondence
AVBPA99 - March 23rd, 1999

21. Signature alignment Main inertia axis of the signature
In order to obtain some degree of invariance w.r.t. rotations,
align the main inertia axis of the signature with the
horizontal axis before performing DTW.
AVBPA99 - March 23rd, 1999

22. Evaluation of the SV system
There are two types of errors to evaluate in order to asses the performance of the system:
FAR (False Acceptance Rate): percentage of false signatures classified as true.
FRR (False Rejection Rate): percentage of real signatures classified as false.
AVBPA99 - March 23rd, 1999

23. Evaluation of the SV system
FAR
FRR
FAR
FRR
Error trade-off curve
Error rate
Equal error rate
(Indicator of the perf. of the system)
Classif. Thres.
Ideal case: FAR = FRR = 0%
AVBPA99 - March 23rd, 1999

24. Experiments Collected a database of signatures 56 subjects
25-30 sample signatures per subject
10 signatures were collected per session, and each session took place on a different day
10 forgeries per subject
each signature acquired with the real-time pen tracking system
AVBPA99 - March 23rd, 1999

25. Experiments The algorithm was tested as follows:
Training set: 10 signatures per subject.
Test set (true): remaining 15 signatures per subject.
Test set (false): all 1375 signatures from all other subjects (“random forgeries”).
Test set (false): 10 intentional forgeries.
AVBPA99 - March 23rd, 1999

26. Experiments examples references False rejects False accepts
AVBPA99 - March 23rd, 1999

27. Results
FRR
FRR
FAR
FAR
AVBPA99 - March 23rd, 1999

28. Conclusions
Presented the performance of a vision-based technique for personal identification.
Demonstrated the feasibility of having such a system working in real-time with high performance in verification.
AVBPA99 - March 23rd, 1999

29. Future Work
Evaluate different representations of the signature and different similarity measures in order to improve performance and achieve scale invariance.
Overcome the lack of examples in order to extract more meaningful estimates of the generalization error.
AVBPA99 - March 23rd, 1999

30. Future Work
Match all examples in a way such that the prototype would be a more robust representative of its class.
Once we have the matching function, devise other similarity measures that improve performance.
Collect a new, clean and bigger database of examples.
AVBPA99 - March 23rd, 1999