Concept-based Image Retrieval: The ARRS GoldMiner ® Image Search Engine Charles E. Kahn Jr., MD, MS Medical College of Wisconsin Milwaukee, Wisconsin, USA

Disclosure No commercial interests American Roentgen Ray Society (ARRS) Executive Council member (Publications) Consulting stipend

Google® Images circa 2006

ARRS GoldMiner ® Image search engine “Open access” images Built by and for radiologists Powered by a major radiology organization Available freely to all

goldminer.arrs.org

Overview ARRS GoldMiner ® How it works Get the most for your search New features Advanced search Spell check GoldMiner Global What’s on the horizon

Search term

Click on the thumbnail to view the full-size original image Click on the title to link to the original article

Dynamically filter results by age, sex, and imaging modality

ARRS GoldMiner ® Image search engine Radiology images Peer-reviewed journals Combines search strategies Keywords (text strings) Medical concepts Filters search results Age, sex, and imaging modality

Concept-based Search Semantic knowledge Lexical variants “oesophagus” = “esophagus” = “esophageal” Abbreviations “HCC” = “hepatocellular carcinoma” Synonyms “renal calculi” = “kidney stones” Powered by UMLS® Unified Medical Language System

“Searching for Meaning” GoldMiner understands relationships between concepts Expands queries using UMLS Metathesaurus concept hierarchy

UMLS Metathesaurus SNOMED-CT Systematized Nomenclature of Medicine Foundational Model of Anatomy Medical Subject Headings MeSH vocabulary Indexes MEDLINE / PubMed

Software Harvester A Web "robot" intelligently crawls journal web sites to gather figures and their text captions Indexer Discovers medical concepts in free-text figure captions Indexes the figures to speed retrieval Search engine Retrieves images to answer your search query

Image Library 7 core journals including AJR, Radiology, and RadioGraphics 250+ peer-reviewed journals Image collection grows weekly

Outstanding Performance Fast retrieval Mean = sec Accurate search results High precision and recall (86%+) Image metadata (~85% accuracy) Age, sex, imaging modality

A “How-To” Guide Keep it simple ! Use a search term that represents A disease An imaging finding, or An anatomic feature Filter your search results Limit images by age group, sex, and/or imaging modality

New Features GoldMiner Spelling corrector Advanced search Image Gallery GoldMiner Global 10 languages

Spelling Correction

Advanced Search Combine up to 3 search queries e.g., “Lung metastases” and “colon cancer” Limit to core journals Search by age Age group (“Child”) or specific range Sex Imaging modalities

Image Gallery Instantly add images to a web site All it takes is one line of HTML code! GoldMiner “inlines” relevant images into web pages based on their content

GoldMiner inserts up to four relevant images, based on the page’s title and subject The caption pops up when your mouse hovers over an image; click on the image to view it at full size GoldMiner ® Image Gallery at the CHORUS web site (chorus.rad.mcw.edu)

GoldMiner Global Multi-lingual search engine 10 other languages Uses MeSH Medical Subject Headings Arabic Chinese French German Italian Japanese Korean Portuguese Russian Spanish Arabic Chinese French German Italian Japanese Korean Portuguese Russian Spanish

العربية 日本語 Deutsch Español Русский 中文 Português Italiano Français A customized NLM web interface translates the query term into an English-language MeSH term The ARRS GoldMiner ® server sends a translation request; it uses the English-language query term to retrieve images. Users can search for images using search terms in 10 languages other than English.

Searching in Chinese for “Liver Cancer” “ 肝癌 ”

In Development Content-based image retrieval Imaging modality classifier Find similar images Point-of-care learning Category 1 CME

Image Modality Classifier Use visual info to detect modality 95% accuracy Combine with existing text data “CT”

“Visual Search” for Images

Point of care learning “Just-in-time” knowledge CME credits for clinically directed search ARRS member benefit

Point of Care Learning Learning LogSearch Results

goldminer.arrs.org Any questions?