1. Stefan Schulz Philipp Daumke Holger Stenzhorn Incremental Semantic Enrichment of Narrative Content in Electronic Health Records Institute of Medical Biometry and Medical Informatics, Freiburg University Medical Center Freiburg, Germany Kornél Markó Michael Poprat AVERBIS GmbH, Freiburg, Germany

2. Narrative Content Structured Content The Dilemma of the Electronic Health Record

3. Narrative Content Structured Content Advantages and Disadvantages Ease of production Quality human communication machine communication data analysis + ‒ ‒ +

4. Narrative Content Structured Content Overlaps Diseases Procedures Problems Drugs Risk Factors Lab …

5. Narrative Content Structured Content Redundant documentation Diseases Procedures Problems Drugs Risk Factors Lab …

6. Narrative Content Structured Content Diseases Procedures Problems Drugs Risk Factors Lab … How to bridge this gap and make medical documentation more efficient?

7. Structured Content Narrative Content Semantic enrichment of clinical narratives using natural language technologies

8. sentence detecting spell checking acronym expansion NE recognition POS tagging NP extraction context detection Paciente HAS, tabagista, dislipidêmico, sem aderência ao tratamento, história de 2 IAMs prévios, cateterismo de 09/2002 com ADA sem lesões significativas, lesão severa em MgCX -implantando stent, ACD com lesões severas mas é vaso de pequeno calibre, não-dominante. Interna no CTI-HCPA alcoolizado, com dor torácica, sendo diagnosticado angina instável de baixo risco. Realizado tratamento clínico, com supressão imediata da dor torácica, permancendo assintomático desde então. Não será realizado investigação não-invasiva ou invasiva, pois a ACTP + stent foi realizado há 1 ano, havendo baixa probabilidaded ded reestenose e paciente apresentou dor torácica em situação sui generis, sem uso de medicação. Medicações: - AAS 100 mg/d - Propranolol 80 mg 3x/d - Sustrate 1cp 3x/d - Captopril 25 mg 3x/d - Sinvastatina 20 mg/d term mapping Natural language processing pipeline

9. death Ontology mapping by term recognition acute abdomen CT abdomen aortic aneurysm aneurysm- ectomy vascular prosthesis systemic infection multiple organ failure vasoacti ve drug hemo dialysis Ontology, e.g. SNOMED CT Patient admitted with acute abdomen. Abdominal CT: leaking abdominal aortic aneurism. Emergency aneurysmectomy with prosthesis. Postoperative evolution with systemic inflammatory response syndrome, multiple organ failure and hemodynamic instability. Despite application of vasoactive drugs, volume replacement and hemodialysis, the patient's condition worsened evolving to death. death

10. Ontology mapping is not enough! incisional hernia herniorrhaphy operation room liver transplant operation exists doesn’t exist reference to a plan operation reference to a suspended plan exists reference to a plan NOT referring to the same patient reference to a new plan not yet executed Patient with incisional hernia admitted for herniorrhaphy, but operation was suspended because operation room was urgently needed for liver transplant. Discharged with orientation and rescheduled operation. Ontology, e.g. SNOMED CT

11. A useful representation of a document’s content requires –Identification of the semantic identifiers (ontology) –Identification of the context of use (epistemology, pragmatics) Important epistemic contexts in medical narratives –Subject of record: “father died of aortic aneurysm” –Uncertainty “…was admitted with suspected aneurysm” –Negation: “no aneurysm” –Prognosis: “elevated risk of aneurysm rupture” –Plan: “in the case of acute abdominal pain, an abdominal CT must be order” Context recognition: challenging but feasible if not fully automated Meaning & Context

12. Sketch of Incremental Semantic Annotation Workflow

13. author writes or dictates text (speech recognition) background coding engine detects terms and proposes codes author acknowledges or corrects proposed terms / codes text finished background text processing engine detects and proposes contexts / negation scopes / moods author acknowledges or corrects proposed scopes if specified, system adds data to information template system displays text together with information template user acknowledges data in information template system modifies text accordingly user adds or modifies data in information template

14. Template filling Text completion Context recognition Term recognition author writes or dictates text (speech recognition) background coding engine detects terms and proposes codes author acknowledges or corrects proposed terms / codes text finished background text processing engine detects and proposes contexts / negation scopes / moods author acknowledges or corrects proposed scopes if specified, system adds data to information template system displays text together with information template user acknowledges data in information template system modifies text accordingly user adds or modifies data in information template

15. Physical examination on admission revealed purpura of the upper and lower extremities, 419620001 110714004 65124004 {113279002, 426210003} 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] 49401003 76197007 14016003 420510009 103213002 53945006 35105006 10828004 16788000 260385009 16788331 16788021 Clinical narrative SNOMED CT codes

16. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. 419620001 110714004 65124004 {113279002, 426210003} 116223007 91637004 Clinical narrative SNOMED CT codes

17. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. 419620001 110714004 65124004 {113279002, 426210003} 116223007 91637004 Clinical narrative SNOMED CT codes gingiva gum - product X

18. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. 419620001 110714004 65124004 113279002 116223007 91637004 Clinical narrative SNOMED CT codes

19. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. 419620001 110714004 65124004 113279002 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] Clinical narrative SNOMED CT codes

20. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts 419620001 110714004 65124004 113279002 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes

21. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts 419620001 110714004 65124004 113279002 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes Selected context: “on admission”

22. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts 419620001 110714004 65124004 113279002 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes Selected context: “on admission”

23. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts 419620001 110714004 65124004 113279002 116223007 91637004 252275004 111583006 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes Selected context: “negation”

24. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts ADM{419620001 110714004 65124004 113279002 NEG{116223007 91637004 252275004 111583006} 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] } 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes

25. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts ADM{419620001 110714004 65124004 113279002 NEG{116223007 91637004 252275004 111583006} 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] } 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes Monoblast stainingposnegn/a [alpha]-naphthyl butyrate esterase naphthol ASD chloroacetate esterase

26. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts ADM{419620001 110714004 65124004 113279002 NEG{116223007 91637004 252275004 111583006} 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] } 49401003 76197007 14016003 420510009 103213002 53945006 35105006 Clinical narrative SNOMED CT codes Monoblast stainingposnegn/a [alpha]-naphthyl butyrate esterasex naphthol ASD chloroacetate esterasex

27. Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts. The Monoblasts were positive for staining by [alpha]-naphthyl butyrate esterase and negative for staining by naphthol ASD chloroacetate esterase. ADM{419620001 110714004 65124004 113279002 NEG{116223007 91637004 252275004 111583006} 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] } 49401003 76197007 14016003 420510009 103213002 53945006 35105006 10828004 16788000 260385009 16788331 16788021 Clinical narrative SNOMED CT codes Monoblast stainingposnegn/a [alpha]-naphthyl butyrate esterasex naphthol ASD chloroacetate esterasex

28. Structured Content Narrative Content Result: seamless integration of narrative with structure Physical examination on admission revealed purpura of the upper and lower extremities, swelling of the gums and tonsils, but no symptoms showing the complication of myasthenia gravis. Hematological tests revealed leucocytosis: WBC count 68 700/µl (blasts 11.5%, myelocytes 0.5%, bands 2.0%, segments 16.0%, monocytes 65.5%, lymphocytes 4.0%, atypical lymphocytes 0.5%), Hb 7.1 g/dl (reticulocytes 12%) and a platelet count of 9.1 × 104/µl. A bone marrow aspiration revealed hypercelllar bone marrow with a decreased number of erythroblasts and megakaryocytes and an increased number of monoblasts. The Monoblasts were positive for staining by [alpha]-naphthyl butyrate esterase and negative for staining by naphthol ASD chloroacetate esterase. ADM{419620001 110714004 65124004 113279002 NEG{116223007 91637004 252275004 111583006} 767002[68700] 271040006[11.5] 313696224[0.5] 313696667[2.0] 313696009[16.0] 271037006[65.5] 271036002[4.0] 271036013[0.5] 365809007[7.1] 45995003[12] 365632008[91000] } 49401003 76197007 14016003 420510009 103213002 53945006 35105006 10828004 16788000 260385009 16788331 16788021 Monoblast stainingposnegn/a [alpha]-naphthyl butyrate esterasex naphthol ASD chloroacetate esterasex Narrative Content