Using the Gamera framework for the recognition of cultural heritage materials Levy Project II Digital Knowledge Center, Sheridan Libraries, Michael Droettboom, Ichiro Fujinaga, Karl MacMillan, G. Sayeed Choudhury, Tim DiLauro, Mark Patton, Teal Anderson

Overview Application area: cultural heritage materials The Gamera system Demo

Ingestion of cultural heritage materials Digitizing images is a good first step......however, new kinds of inquiry and retrieval require converting the images into a more useful format

The problem Commercial OCR systems are often inadequate for non-standard documents The market for specialized recognition of historical documents is very small Researchers performing document recognition often “re-invent” the basic image processing wheel

The solution Provide easy to use tools to allow domain experts (people with specialized knowledge of a collection) to create custom recognition applications

Gamera Allows rapid development of domain-specific document recognition applications Domain experts can customize and control all aspects of the recognition process Includes an easy-to-use interactive environment for experimentation

How Gamera is like Matlab Very high-level programming language Close to the problem at hand Supports incremental (test-and-refine) approach to software development User can see the results of the program interactively Provides a large set of pre-designed building blocks

How Gamera is unlike Matlab Designed specifically for structured documents Narrower scope makes it simpler to use Open-source and open standards-compliant Gamera scripts are production applications Can be batched, to process a number of images consecutively

Tools provided by Gamera Pre-processing Document segmentation and analysis Symbol segmentation and analysis Syntactical or structural analysis Output

Pre-processing Photoshop-like filtering of the image Noise removal Blurring De-skewing Binarization

Document segmentation and analysis Dividing the document into high-level units

Symbol segmentation and classification Dividing the image into individual symbols Using a classifier to label them Classifiers can include k-nearest neighbor, neural-nets, hidden Markov models, etc.

Symbol segmentation and classification Original image

Symbol segmentation and classification Staff removed image

Symbol segmentation and classification Connected component segmentation

Symbol segmentation and classification Labeling by a classifier

Syntactical or structural analysis Grouping the symbols into meaningful units and performing basic analysis on their relationships In text: lines, paragraphs, columns, etc.

Demonstration A whirlwind tour of the Gamera system