Download presentation
Presentation is loading. Please wait.
Published byFrancis Griffin Modified over 9 years ago
1
A Search Engine for Historical Manuscript Images Toni M. Rath, R. Manmatha and Victor Lavrenko Center for Intelligent Information Retrieval University of Massachusetts SIGIR2004
2
Introduction The first known automatic retrieval system for handwritten historical manuscript The obvious approach to this problem is to use handwriting recognition but the error rate is excess 50% This is an system search handwritten manuscript using text queries without recognition
3
Introduction Seem the problem as an image annotation or cross-lingual retrieval problem that use text word to query image word Learn a statistical relevance model by training on a transcribed set (word for word) of pages Two models Probabilistic Annotation Model Direct Retrieval Model
4
An Example An example image from George Washington collection
5
Related Work Obvious approach Handwriting recognition + text search engin Image annotation Duygulu – Translation model Blei – Latent dirichlet allocation model Jeon – Cross-media relevance model (CMRM) Lavrenko – Cross-lingual relevance model
6
Related Work (2) Handwriting recognition+ text search engine Advantage Can be used for every English word Disadvantage Well Know segment error Image Annotation (Convert)
7
Related Work (3) The different between image annotation and their model Use shape feature instead color and texture feature Do not using cluster or blob Learn the relation between features and English texts to instead blobs and English texts
8
Image Annotation
9
Model In This Paper
10
System Overview Probabilistic Annotation Model 1.Training relations between features and English word 2.Each word image in the testing set is annotated with every term in the annotation vocabulary and a corresponding probability 3.The result in 2 will be store in an inverted list for quick access so typical query times are less than one second
11
Probabilistic Annotation Model
12
System Overview Direct Retrieval 1.Training relations between features and English word 2.Use query to estimate a distribution over the feature vocabulary that one would expect to observe jointly with the query 3.Compare this distribution with a distribution of the feature vocabulary of each word image using Kullback-Liebler divergence,one may rank all word images in the testing set at query time
13
Direct Retrieval
14
Demo A demo at http://ciir.cs.umass.edu/research/wordspotting
15
Word Image Representation Simple shape features: like width and height. Use a total 5 such feature Fourier coefficient of profile feature: detail descriptions of a word ’ s shape can be obtain with profile features, such as the upper and lower profiles (see the picture), each profile have 7 features and one image word obtain 3x7=21 profile feature One image word have totally 21+5= 26-dimensional continuous-space feature vector
16
Word Image Representation Dividing the range of observed values in each feature dimension into 10 bins of equal size, and associate a unique feature vocabulary term with each bin Repeat the process but 9 bins in this time Each word image will have 2x26=52 features There are (10+9)x26=494 features in the feature vocabulary
17
Model Formulation Probabilistic Annotation Model w: an English word f : a feature word k: feature number=52 I: image word in the training set i: position in the training set |T|:words number in the training set
18
Model Formulation V : vocabulary in training set Smoothing δ ( x ∈ { w i.f i1 …f tk } )= 1, if x ∈ {w i.f i1 …f tk }, else 0 x : w or f ג : 0~1
19
Model Formulation Page Retrieval Pg : a pageQ:query text q 1 ~q m
20
Model Formulation Direct Retrieval Q : query wordW : image word Use (6) to estimate P(Q|W) and P(f|W)
21
Reordering An image word can be represent by a vector of 494 entries and 52 1’s Change retrieved images and training images for the given query to that form Reordering was performed using the average dot product of the retrieved images and training images for the given query
22
Data Collection George Washington collection at Library of Congress Contains 150000 ages Image were digitized from microfilm at 300dpi, 8 bit grayscale from thesepages Training set 100 pages (24665 words,3087 vocabulary) Testing set 987 pages (234754 words)
23
Experimental Eval. - queries Mixture of proper names, places, nouns, number in the form of a year Have reasonably frequent words in the training set It is possible that some of the query words may not occur in the test set
24
Eval. – Word Image Retrieval A number of words are incorrect segment Direct retrieval model did not retrieve any instances of deserter and disobedience, while probabilistic annotation model found one disobedience The low turnout may be caused either by insufficient training or the lack relevant images in the testing collection
25
Eval. - Page Image Rtrieval In one word queries the performance is quite good, even higher than in the single word retrieval without reordering. In two word queries the results seem low, but believe that a more thorough evaluation with ground true data would yield better results.
26
Conclusions Results show that retrieval can be done even when recognition of handwriting remain a challenging task Adapting statistical relevance models produce good results, much remains to be done. Better models are needed Large datasets can be handled either by using a cluster of processors, or by improving the efficiency of both the feature processing and retrieval model stages
27
Conclusions The lack of training data requires attention. We are current investigating synthetic training data as a possible solution
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.