Handwriting: Registration and Differential Equations

Printed the letters “fda” by hand 20 times. Printed the letters “fda” by hand 20 times. Optotrak system recorded positions of pen, 200 times per second: Optotrak system recorded positions of pen, 200 times per second: X (left to right) Y (front to back) Z (up and down, vertically) There are different rates of printing, even for same person. There are different rates of printing, even for same person.

Why study time records of handwriting? Fraud detection: handwriting is personal. Fraud detection: handwriting is personal. We usually only see static images – but what about our time-signature? Are our writing dynamics unique? We usually only see static images – but what about our time-signature? Are our writing dynamics unique?

How can we prepare the data for analysis? Problem: Timing of letters is different for each replication. Problem: Timing of letters is different for each replication. “Hand times” are different than “clock time.” Amplitude differences: having to do with shape. Phase differences: having to do with timing. Both are important components of functional data. How can we transform (warp) the clock time so that we’re left with only amplitude differences? How can we transform (warp) the clock time so that we’re left with only amplitude differences?

What does a warping function look like? Target function is sin(4pit). Target function is sin(4pit). Individual function is running is fast: Individual function is running is fast: So time-warping function is So time-warping function is

What’s the process to align our data? Idea: Rescale time with a warping function. Idea: Rescale time with a warping function. To do this, we need a “target” curve. To do this, we need a “target” curve. Can use a gold-standard curve, or use a mean curve, or iterate. Can use a gold-standard curve, or use a mean curve, or iterate. Time-warping function maps “clock time” to “hand time.” Time-warping function maps “clock time” to “hand time.” Continuous-Time Registration

First Problem: Need curves to fit in same interval. First Problem: Need curves to fit in same interval. Each curve i lasts over interval [0, T i ] Each curve i lasts over interval [0, T i ] Want to rescale each curve to fit a target interval [0, T 0 ] Want to rescale each curve to fit a target interval [0, T 0 ] Solution: Find a time-warping function h i (t) such that Solution: Find a time-warping function h i (t) such that h i (0) = 0 (starts at same time) h i (0) = 0 (starts at same time) h i (T 0 ) = T i (ends at same time) h i (T 0 ) = T i (ends at same time) h i (t 1 ) > h i (t 2 ) iff t 1 > t 2 (events occur in same order) h i (t 1 ) > h i (t 2 ) iff t 1 > t 2 (events occur in same order)

Second Problem: Want features of the curves to line up Second Problem: Want features of the curves to line up Want the curves to differ only in amplitude Want the curves to differ only in amplitude Solution: Find a time-warping function h i (t) that minimizes a “misalignment” criterion. Solution: Find a time-warping function h i (t) that minimizes a “misalignment” criterion. Good one is M(h) = smallest eigenvalue of [matrix] Good one is M(h) = smallest eigenvalue of [matrix] When M(h) is close to zero, the registered curve values plot against the target curve values as a straight line. When M(h) is close to zero, the registered curve values plot against the target curve values as a straight line.

Can we register transformations of the curves, too? Yes! It’s often better to register the derivatives of the curves than the curves themselves. Yes! It’s often better to register the derivatives of the curves than the curves themselves. More distinctive features available to align. More distinctive features available to align. Sometimes closer to the process of interest! Sometimes closer to the process of interest!

Registered curves for handwriting Handwriting data have both X and Y directions. Handwriting data have both X and Y directions. Register tangential acceleration TA(t) = [X’’(t) 2 + Y’’(t) 2 ] 1/2 Peaks more cleanly aligned after registration. Peaks more cleanly aligned after registration.

Can we imagine a differential equation for handwriting? First look at the X-coordinate – the other two are analogous First look at the X-coordinate – the other two are analogous Relates the physics of the pen movement, written in a standard differential equation template. Relates the physics of the pen movement, written in a standard differential equation template. Physical characteristics: Physical characteristics: D 3 x i (t) = jerk of i th pen. D 3 x i (t) = jerk of i th pen. D 2 x i (t) = acceleration of i th pen. D 2 x i (t) = acceleration of i th pen. Dx i (t) = velocity of i th pen. Dx i (t) = velocity of i th pen. Functional coefficients, which describe relationships among physical characteristics: Functional coefficients, which describe relationships among physical characteristics: a x (t) = time-varying intercept. a x (t) = time-varying intercept. b x1 (t) = time-varying coefficient relating velocity to jerk. b x1 (t) = time-varying coefficient relating velocity to jerk. b x2 (t) = time-varying coefficient relating acceleration to jerk. b x2 (t) = time-varying coefficient relating acceleration to jerk. Linear Differential Equation

Analogy to linear regression Analogy to linear regression Jerk is response. Jerk is response. Acceleration and velocity are predictors. Acceleration and velocity are predictors. The difference is that response and predictors are all estimated from same data. The difference is that response and predictors are all estimated from same data. Estimate coefficients through Principal Differential Analysis (PDA) Estimate coefficients through Principal Differential Analysis (PDA)

Principal Differential Analysis Estimate coefficient functions a x (t), b x1 (t), and b x2 (t) in the linear differential equation Estimate coefficient functions a x (t), b x1 (t), and b x2 (t) in the linear differential equation First find good estimates of the derivatives D 2 x and D 3 x. First find good estimates of the derivatives D 2 x and D 3 x. Similar to linear regression: Find coefficients that minimize Similar to linear regression: Find coefficients that minimize How do we do this? How do we do this?

How do we do PDA? Use regularization again to balance goals: Use regularization again to balance goals: Want the estimated weight functions to be close to the “true” weight functions. Want the estimated weight functions to be close to the “true” weight functions. Want smooth weight functions. Want smooth weight functions. Minimize Minimize Lambda is smoothing parameter Lambda is smoothing parameter Lambda = 0: weight functions rough Lambda = 0: weight functions rough Lambda  infinity: weight functions go to 0 (differential operator no different than mth derivative) Lambda  infinity: weight functions go to 0 (differential operator no different than mth derivative)

Estimated coefficient functions First coefficient function: First coefficient function: Average value 289 Horizontal oscillation once every.2 seconds Second coefficient function: Second coefficient function: Average value around 0 Exponential growth or decay (none on average here)

Checking adequacy of fit Residual functions Residual functions small relative to third derivative. concentrated around zero. R 2 goodness of fit (proportion of variability explained): R 2 goodness of fit (proportion of variability explained): X-coordinate: Y-coordinate: Z-coordinate: 0.994

Can we use this differential equation to identify writers? Two writers: JR and CC. Two writers: JR and CC. 20 replications each. 20 replications each. Estimate linear differential equation for each writer. Estimate linear differential equation for each writer. Apply each estimated equation to self and to other. Apply each estimated equation to self and to other. Regression analogy: Regression analogy: Build an equation on one dataset; Build an equation on one dataset; predict the responses for new dataset; predict the responses for new dataset; compare fitted values to actual values (residuals). compare fitted values to actual values (residuals). JR to JR JR to JR JR to CC JR to CC CC to JR CC to JR CC to CC CC to CC

Residuals for x coordinate

Residuals for Y coordinate

Residuals for Z coordinate

What have we learned? Continuous time registration Continuous time registration Differential equations Differential equations Functional data in three dimensions Functional data in three dimensions