1. Retrieval of Similar Electronic Health Records using UMLS Concept Graphs Laura Plaza and Alberto Díaz Universidad Complutense de Madrid

2. ª When facing complex and untypical cases, physicians need to refer to similar previous cases ª The adoption EHR by office-based physicians and hospitals is increasing ª But still the time required to find them can be prohibitive if no effective access is provided Motivation Motivation Given a reference record, retrieve others from the clinical database that are similar to the reference one Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 2

3. ª A mix of highly structured information + idiosyncratic narrative text ª Unique sublanguage characteristics:  Verbless sentences, punctuation, spelling errors.  Synonyms and homonyms  Neologisms  Acronyms and abbreviations ª When two HR can be considered as similar? A Different IR Task A Different IR Task Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 3

4. Two EHR are Similar if… Two EHR are Similar if…  Same symptom or sign (e.g. fever or 5 kg weight loss)  Same diagnosis (e.g. bacterial pneumonia)  Same test or procedure (e.g. cerebral NMR or endoscopy biopsy)  Same medicament (e.g. clopidogrel)  But … absent criteria are not relevant for the task!!! Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 4

5. ª UMLS consists of three main components: the Specialist Lexicon, the Metathesaurus and the Semantic Network ª We use MetaMap to translate free-form text to Metathesaurus concepts ª Advantages:  Broad coverage  Performs word sense disambiguation  Numerous entries for acronyms and abbreviations  Etc. Using UMLS for Concept Annotation Using UMLS for Concept Annotation Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 5

6. ª A four-step graph-based method : 1.Extraction of UMLS concepts 2.Negation detection 3.Semantic graph-based representation 4.Ranking similar EHR Our Proposal Our Proposal CLINICAL HISTORY: Eleven years old with ALL, bone narrow transplant on Jan.2, now with 3 day history of cough. IMPRESSION: No focal pneumonia. Likely chronic changes at the left lung base. Mild anterior wedging of the thoracic vertebral bodies. Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 6

7. ª We use MetaMap to extract the UMLS concepts from the Metathesaurus and their semantic types from the Semantic Network ª But, according to the expert, not all concepts are relevant to the task ª Thus, the expert mapped these criteria to semantic types and only concepts from those types are considered Our Proposal: Extracting UMLS Concepts Our Proposal: Extracting UMLS Concepts Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 7

8. Our Proposal: Extracting UMLS Concepts Our Proposal: Extracting UMLS Concepts CategoryUMLS Semantic Types Symptoms and Signs Sign or Symptom Finding Diseases Disease or Syndrome Pathologic Function Procedures Therapeutic or Preventive Procedure Diagnosis Procedure Body Parts Body Location or Region Body Part, Organ, or Organ Component MedicamentsPharmacologic substance Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 8

9. ª According to the expert, absent or negated criteria (e.g. On admission, the patient had no internal bleeding) are not relevant for the task ª Thus, negated UMLS concepts are ignored ª Negations in medical records usually appears in a reduced number of forms, easy to identify using a simple lexical scanner from regular expressions Our Proposal: Negation Detection Our Proposal: Negation Detection Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 9

10. Our Proposal: Negation Detection Our Proposal: Negation Detection Lexical PatternExamples no|without| rule out + concept + (or concept)* No pneumonia Without fever or cough no|without|rule out + adj + concept + (or concept)* No significant hydronephrosis Rule out cardiac abnormality no|without + noun + of + concept + (or concept)* No signs of tuberculosis Without evidence of hydroureter evaluate for + (noun|adj)? + concept + (or concept)* Evaluate for foreign body Evaluate for abnormalities lack of|absence of + (noun|adj)? + concept + (or concept)* Lack of kyphosis Absence of heart murmur Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 10

11. ª First, the concepts are retrieved from the UMLS Metathesaurus along with their complete hierarchy of hypernyms (is-a relations). ª Second, all concept hierarchies for each category are merged, building a unique graph for each category in the EHR ª Finally, each concept is assigned a weight, using the Jaccard similarity coefficient, attaching greater importance to specific concepts than to general ones Our Proposal: Semantic Graph Representation Our Proposal: Semantic Graph Representation Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 11

12. Our Proposal: Semantic Graph Representation Our Proposal: Semantic Graph Representation Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 12 1/5 2/5 3/5 4/5 5/5

13. ª We compute the similarity among the reference EHR and all records in the database, and rank them ª Given two graphs, A and B, so that the similarity of A to B has to be measured:  First, each concept of A which is not in B assigns a score equal to 0, while each concept of A which is also in B assigns a score equal to its weight in the graph A  Next, the sum of the scores for all concepts in A is computed.  Finally, this result is normalized in the interval [0, maximum similarity]. Our Proposal: Ranking Similar EHR Our Proposal: Ranking Similar EHR Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 13

14. Our Proposal: Ranking Similar EHR Our Proposal: Ranking Similar EHR Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 14 Finding by site Clinical finding Disease Bacterial pneumonia Infectious disease Disorder by body site Pneumonia due toStreptococcus Mycoplasma pneumonia Respiratory finding Functional finding of respiratory tract Coughing Clinical finding Disorder by body site Finding by site 1/11 2/11 3/11 8/11 9/11 10/11 3/5 4/5 5/5 Bacterial pneumonia Pneumococcal pneumonia 11/11 Pneumonia due to anaerobic bacteria Pneumonia due topleuropneumonia Graph A Graph B... Virus Diseases

15. ª Test collection: 50 radiology reports from the CMC-NLP 2007 Challenge corpus ª Query collection: a subset of 20 reports from the test collection ª Two hospital physicians were asked to select, for each report in the query collection, the most similar reports within the test collection ª There is a substantial agreement between judges (Kappa test, k=0.7980) ª Precision and Recall of our method are compared with those obtained by a term-based approach Experiments Experiments Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 15

16. Results Results Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 16 Graph-based method Term-based Method PrecisionRecall F-score PrecisionRecallF-score Union-5 0.5300.765 0.626 0.4700.7080.564 Intersection-5 0.4200.884 0.569 0.3400.7290.463 Union-3 0.7170.676 0.696 0.4670.4760.471 Intersection-3 0.6000.788 0.681 0.3330.4870.395 Graph-based method Term-based Method Precision F-score PrecisionF-score Union0.7070.6920.5940.632 Intersection0.7450.7420.5030.598

17. ª The method achieves relatively high precision and recall which are also well balanced ª UMLS occasionally fails to recover relevant concepts especially when expressed in their shortened forms ª Another impairment to concept identification comes from the spelling errors in the clinical records ª Future work will test the method on a different evaluation collection which will present longer medical records structured in different sections Conclusion and Future Work Conclusion and Future Work Retrieval of Similar Electronic Health Records Using UMLS Concept Graphs (Plaza and Díaz, 2010) 17