Extracting and Organizing Facts of Interest from OCRed Historical Documents Joseph Park, Computer Science Brigham Young University

Motivation 2 Large collection of scanned, OCRed books Stated facts Implied facts ▫ Inferred ▫ Same-as (entities)

Stated Facts of Interest 3 William Gerard Lathrop ▫ married Charlotte Brackett Jennings in 1837 ▫ is the son of Mary Ely ▫ was born in 1812

Inferred Facts of Interest 4 William Gerard Lathrop has gender Male Maria Jennings has gender Female Maria Jennings has surname Lathrop

Same-as (Entities) William Gerard Lathrop Gerard Lathrop Mary Ely 5

FROntIER 6 OntoES convert to RDF Jena reasoner comparators parameters owl:sameAs Abigail Huntington Lathrop Female RDF output [(?x rdf:type source:Person) -> (?x rdf:type target:Person)] [(?x source:Person-Name ?n),(?n source: NameValue ?nv), isMale(?nv),makeTemp(?gender) -> (?x target:Person-Gender ?gender),(?gender rdf:type target:Gender), (?gender target:GenderValue `Male'^^xsd:string)] Duke convert to csv

FROntIER 7 OntoES convert to RDF Jena reasoner comparators parameters owl:sameAs Abigail Huntington Lathrop Female RDF output [(?x rdf:type source:Person) -> (?x rdf:type target:Person)] [(?x source:Person-Name ?n),(?n source: NameValue ?nv), isMale(?nv),makeTemp(?gender) -> (?x target:Person-Gender ?gender),(?gender rdf:type target:Gender), (?gender target:GenderValue `Male'^^xsd:string)] Duke convert to csv

Extraction Ontologies Conceptual model Instance recognizers 8

Lexical Object-Set Recognizers 9 BirthDate external representation: \b[1][6-9]\d\d\b left context: b\.\s right context: [.,] …

Non-lexical Object-Set Recognizers 10 Person object existence rule: {Name} … Name external representation: \b{FirstName}\s{LastName}\b …

Relationship-set Recognizers 11 Person-BirthDate external representation: ^\d{1,3}\.\s{Person},\sb\.\s{BirthDate}[.,] …

Ontology-snippet Recognizers 12 ChildRecord external representation: ^(\d{1,3})\.\s+([A-Z]\w+\s[A-Z]\w+) (,\sb\.\s([1][6-9]\d\d))?(,\sd\.\s([1][6-9]\d\d))?\.

FROntIER 13 OntoES convert to RDF Jena reasoner comparators parameters owl:sameAs Abigail Huntington Lathrop Female RDF output [(?x rdf:type source:Person) -> (?x rdf:type target:Person)] [(?x source:Person-Name ?n),(?n source: NameValue ?nv), isMale(?nv),makeTemp(?gender) -> (?x target:Person-Gender ?gender),(?gender rdf:type target:Gender), (?gender target:GenderValue `Male'^^xsd:string)] Duke convert to csv

Canonicalization 14 “Sam’l” and “Geo.” -> “Samuel” and “George” “New York City” -> “New York, NY” “Boonton, N.J.” -> “Boonton, NJ”

Schema Mapping 15 author’s view our view

Direct Schema Mapping Rule 16 [(?x rdf:type source:Person) -> (?x rdf:type target:Person)] Person 7

Name Decomposition Rule 17 [(?x rdf:type source:Person),(?x source:Person-Name ?y),(?y rdf:type source:Name),… -> (?x rdf:type target:Person),(?x target:Person-Name ?y),(?y rdf:type target:Name),…] Person 7 William Gerard Lathrop Gerard Lathrop William Name 7 Person 7

Person has gender Male Rule 18 [(?x rdf:type source:Son),makeSkolem(?gender, ?x) -> (?x target:Person-Gender ?gender),(?gender rdf:type target:Gender), (?gender target:GenderValue 'Male'^^xsd:string)] Name 7 William Gerard Lathrop Gerard Lathrop William Male Person 7 Name 7

FROntIER 19 OntoES convert to RDF Jena reasoner comparators parameters owl:sameAs Abigail Huntington Lathrop Female RDF output [(?x rdf:type source:Person) -> (?x rdf:type target:Person)] [(?x source:Person-Name ?n),(?n source: NameValue ?nv), isMale(?nv),makeTemp(?gender) -> (?x target:Person-Gender ?gender),(?gender rdf:type target:Gender), (?gender target:GenderValue `Male'^^xsd:string)] Duke convert to csv

Resolving Mary Elys Example Person 2 (1 st Mary Ely) owl:sameAs Person 8 (3 rd Mary Ely)

Resolving Gerard Lathrop Example Person 3 (1 st Gerard Lathrop) owl:sameAs Person 9 (2 nd Gerard Lathrop) ~

Experiment The Ely Ancestry, page 419 ▫ Extracted facts: full page ▫ Inferred facts: excerpt ▫ Same-as (entities) facts: excerpt Annotated gold standard files ▫ Extracted facts: automated evaluation tool ▫ Inferred facts: hand checked with partial matches ▫ Same-as (entities) facts: hand checked 22

Extracted Facts Results 23

Implied Facts Results 24 *Name is composed of GivenName and Surname concepts

Conclusions Proof of concept Current and Future work: ▫ Automated evaluation tool for inferred & same-as facts ▫ Corpus of 50,000+ books provided by LDS Church ▫ 200 randomly selected pages ▫ Estimate the following:  Time required  Expertise required  Accuracy (precision & recall) 25