Implementing Dictionary-Based NER Solutions for Mining Biomedical Literature Karen Dowell, Monica McAndrews-Hill, David Hill, Harold Drabkin, Judith Blake 7 th Fraunhofer Symposium on Text Mining October 6, 2009

 ProMiner at Mouse Genome Informatics (MGI)  Background on MGI and our biocuration process  Applying Named Entity Recognition (NER) applications to improve MGI curator efficiency and minimize bottlenecks  Our implementation and results to date using ProMiner to annotate full-text scientific journal articles in HTML and PDF format

 A comprehensive, integrated public information resource for mouse genetics, genomics and biology  Facilitates use of the laboratory mouse as a model for human biology  Provides extensively curated mouse data

The MGI website presents information on mouse biology in a publically accessible, content rich, continually updated online database

MGI content spans from DNA sequence to disease phenotype

MGI integrates information on mouse genes and experimental data through a combination of manual curation, computational curation, and collaboration with other online resources.

Primary Triage Secondary Triage Master Bibliography Indexing Expert Curation  For literature curation we  Review more than 160 scientific journals each month  Screen more than 12,000 articles a year

Primary Triage Secondary Triage Master Bibliography Indexing Expert Curation  Curators pick papers based on  Expression  Mapping  Homology  New Genes  Gene Ontology (GO)  Alleles & Phenotypes  Sequences  Inbred Strain  Tumor  Nomenclature  General Interest Screen for references to mouse, mice, murine

Primary Triage Secondary Triage Master Bibliography Indexing Expert Curation Selected articles are assigned reference numbers and entered into a master bibliography In 2009… 10,097 articles added ~1122 per month (as of September 29, 2009)

Primary Triage Secondary Triage Master Bibliography Indexing Expert Curation Indexing is our internal process of associating article reference numbers to at least one entity within the MGI database. For gene indexing that entity is a gene.

Primary Triage Secondary Triage Master Bibliography Indexing Expert Curation  Curators read each paper and enter information into MGI database using controlled vocabularies  Articles annotated based on  Expression  Mapping  Homology  New Genes  Sequences  Inbred Strains  Tumors  Alleles & Phenotypes

Papers Added Master Bibliography12,97913,23114,190 Phenotype Papers9681 (75%)10,322 (78%)10,689 (75%) GO Papers8364 (64%)7716 (58%)9913 (70%) Selected for Both5974 (46%)6,688 (51%)7231 (51%)

 Many areas could benefit from text mining (as tools, not replacements for human curators)  Selected gene indexing as a prototype project to  Minimize a bottleneck within our curation workflow Articles added to pipeline each month % are selected for GO 770 Articles gene indexed each month 200 More than 2000 articles in gene indexing pipeline

 A dictionary-based named entity recognition (NER) system that  Complements our existing biocuration processes and workflow  Processes full-text PDF files in batch  Uses MGI or comparable dictionaries of mouse symbols, synonyms, and human orthologs  Produces meaningful reports that aid curators  Provides visualization tools  Achieves high F-scores in published evaluations

 Of all the dictionary-based NER tools we evaluated, ProMiner most closely fit our needs  Rule-based protein and entity recognition using pre-processed dictionaries (Entrez Gene, SwissProt, ATTC, and ECACC)  Batch processing of PDF Files (beta release)  Standard and custom reports  Customizable annotation projects and dictionaries/term lists  Initiated collaborative pilot project between SCAI and MGI

 System requirements  Runs on Linux systems, Sun-Ultra, and other UNIX-based systems  Requires minimum 1 GB RAM, 500 MB disk space Java (v1.5 or higher) and Perl (v5.8 or higher)  Uses GeneDB to retrieve data (requires 1 GB to store index files). Includes an HTML-based (CGI) viewer  One processor can update ~1000 articles per project  On a cluster of 16 processors, ProMiner can search the entire MEDLINE literature base with 1 dictionary in ~2 hours

 MGI Operating Environment  Dedicated Sun Fire X4100 Server with two dual core AMD Opteron processors, 2.8 Ghz, 64 bit  Solaris 10 V. 508 operating system, Java5 built-in  Adobe Acrobat Pro Version 9.1  SCAI delivered…  Installation scripts, ProMiner scripts and dictionaries  Documentation and demos  MGI project definition files for annotation using human and mouse dictionaries

 HTML Version 6.4 implemented in March  PDF Version 7.1 delivered in August

This paper was indexed to mouse genes Tlr4 and Ly96

 1 part-time curator working 5.5 hours a day processing batches of 10 articles at a time  8 of 10 PDFs processed correctly, without errors  Some PDF format (PDF/A) and color labeling errors  We provide feedback to SCAI to enhance dictionaries and PDF formatting Manual IndexingIndexing with ProMiner 30 minutes per article18-24 minutes per article 50 articles per week60-70 articles per week F-Score performance measurements in progress

ProMiner 7.1 annotates 75 full-text articles in PDF format in less than 20 minutes on our server Processing time = (No. Articles ) R² =

 Complete performance testing and evaluate status of pilot project with SCAI  Consider extending pilot to continue testing ProMiner 7.1  Explore future collaborations  Gene Ontology terms  Protein-protein interactions  Other curation functions at MGI

 MGI  Judith Blake  Nancy Butler  Harold Drabkin  Alex Diehl  David Hill  Monica McAndrews-Hill  Sue McClatchy  David Shaw  Dmitry Sitnikov MGI System Administration  Matt Baya  Mike McCrossin  Iry Witham  Fraunhofer SCAI  Juliane Fluck  Heinz-Theodor Mevissen  Symposium Organizers  MITRE Corporation  Lynette Hirschman  Journal of Immunology