Combining Neural Networks and Context-Driven Search for On- Line, Printed Handwriting Recognition in the Newton Larry S. Yaeger, Brandn J. Web, and Richard F. Lyon

Overview Tentative Segmentation Neural Net Classifier Context Search (x,y)& Pen-lifts Words Character Class Hypotheses Character Segmentation Hypotheses

Classifier Backpropogation Network Multiple Inputs –20 x 9 Stroke Feature –14 x 14 Image –5 x 1 Stroke Count –1 x 1 Aspect Ratio All run in parallel and combined at output layer.

Picking the Top Choices Ambiguity in character recognition Problem: if the full error is propagated back, the network gives one high answer, but no secondary answers. Solution: Normalize the error before propagating back.

Ambiguity Example

ASS

Ambiguity Example uh

Removing Ambiguity Use the context Look up possible words in a dictionary Use word spacing to help Could use sentence structure, but this doesn’t

LVQ Selection of Networks An LVQ Network will be used to select an appropriate BP network for a given context. EX: Driving in heavy traffic on the freeway vs. driving in town.

Reasons for Combining Networks Allows for a separation of tasks –A BP Network must be carefully trained to adapt to different situations –It would be very difficult to add a different subtask to a BP network. Hope that this technique will scale to more complex tasks better.

Training Manual Construction –Train BP networks for sub-tasks separately, and then train the LVQ network to select the correct network. Two-Stage Construction –Use the LVQ to classify all of the training set into different classes. Train a BP network for each of these classes.

Training (cont.) Simultaneous Construction –Instead of using the standard distance measures, use the output of the BP networks as the distance measure. For each case, the LVQ network will be adjusted so that the winner moves closer to the result. Each case that passes through a BP network will be used to adjust the weights of that network.

Project Goals Create the network described and implement all three training methods Compare the results to those obtained from a single large BP network.