Handwriting Vector Quantizer Capturing Printed and Cursive Text Using Domain Knowledge to Efficiently Compress Data While Maintaining Critical Information

Goals Reversibility – enhances application flexibility and testability Flexibility comes from being able to use the subsystem in either recognition or synthesis mode. Testability – Running a domain representation subsystem in synthesis mode (give it a representative feature sequence and output Speech/Writing) allows us to see/hear what the representation captures about the domain. Produce an early result (preferably mid-term) so that the Recognition Engine teams can have some good test data. The team can then enhance their subsystem performance as the semester continues. The output format should just be a sequence of named objects (e.g. integer digits 0-255) plus a developed closeness matrix that a recognition engine can use as a metric to help in “matching” dissimilar object sequences.

Vector Quantizer Handwriting – both printing and cursive Strokes Pieces of characters Model each as a small number of vector moves Normalization – reduce variability Closeness - Word-Level Heuristic Deal with smoothness violations – Dotting i’s, crossing t’s and q’s Reversibility – works in both directions

Handwriting Input - a sequence of {x,y,z} (z is either binary black/white or perhaps pressure) coordinates from using a stylus to write on the screen. msdn.microsoft.com – Pen programming in Windows Output - a sequence of named vectors from a limited set [perhaps 16 angles and 8 lengths (1, 2 ,4, … , 128) plus B/W – One Byte]. Each written character is then accurately represented by 10 to 30 bytes vs about 104 pixels – a data compression ratio of better than 40:1.

Strokes Sequence of strokes Black – Leaves ink behind White – An off the page move Critical Points - Reliably determined by several methods -allow the generated vector sequence within each stroke to have more consistency. Newton – sudden change in velocity maxima/minima – Along the direction of the writer’s “tilt” pressure changes – Especially on/off the paper

Normalization Character size stylus speed writer “Tilt/Slant” (very different for right handed vs left handed writers) Each of these makes the problem larger for the recognition engine. A Vector Quantizer should attempt to “normalize” these variables, but retain a set of expansion parameters for use when generating the user’s handwriting.

Closeness “Euclidean Distance” – A metric that determines the similarity between features (our 2-D vector set in this case). Closeness Matrix – A data structure containing the pairwise distances between the vectors. The Recognition Engine can use this to generalize the input to get a match while attach “penalties” when generalizing.

Word-Level Heuristic Smoothness violations Dotting i’s, crossing t’s and q’s - tend to be done by writers at the word level instead of per character. A long white stroke to the left - a move backwards in time detected by the Quantizer Determine the where in the stroke/vector sequence where the black stroke belongs suitable triplet of strokes (white-black-white) can be inserted into the stroke sequence of the appropriate letter. Allow normal writing habits to continue. The writer could get improved results by modifying his/her behavior to do these strokes in sequence at the letter level.

Reversibility Smoothing - smooth out the quantized vectors when operating in synthesis mode to produce clean strokes Remembering the writer’s parameters re-introducing them is useful in replicating the writer’s handwriting Tilt Size Pen nib – for cursive/calligraphic output