1. Summarisation and Visualisation of e-Health Data Repositories Catalina Hallett, Richard Power, Donia Scott Centre for Research in Computing The Open University

2. Overview Background Background The need for textual and visual summaries The need for textual and visual summaries Typical input Typical input Requirements Requirements Textual summaries Textual summaries Visual navigator Visual navigator Conclusions Conclusions

3. Background Research conducted under the MRC-funded CLEF project Research conducted under the MRC-funded CLEF project CLEF aims at establishing a technical infrastructure for managing repositories of aggregated patient data in the domain of cancer CLEF aims at establishing a technical infrastructure for managing repositories of aggregated patient data in the domain of cancer Currently, the CLEF repository contains about 22k patient records Currently, the CLEF repository contains about 22k patient records The aim of the current research is to provide solutions for accessing and visualising individual patient histories for use in clinical care The aim of the current research is to provide solutions for accessing and visualising individual patient histories for use in clinical care

4. Rationale Clinicians and other health professionals use patient health summaries at the point of care, where time is a critical resource Clinicians and other health professionals use patient health summaries at the point of care, where time is a critical resource Reports provide quick access to an overview of a patient’s medical history Reports provide quick access to an overview of a patient’s medical history –Typically, an electronic patient record contains around 1000 messages –Even structured, this volume of data is very large –Access to relevant information about particular patients is difficult Easy visual access to time-oriented data and in particular EPRs has long been a topic of research: Easy visual access to time-oriented data and in particular EPRs has long been a topic of research: –Knave-II (Shahar et al 2003) –Lifelines (Plaisant et al 2001) –Vie-Visu (Horn et al 2002) Our research emphasises the role of textual reports, enhanced by visual navigation tools Our research emphasises the role of textual reports, enhanced by visual navigation tools

5. Typical input A complex representation of an Electronic patient record, termed Chronicle A complex representation of an Electronic patient record, termed Chronicle Structured data: Structured data: –Demographics:  Age, gender, postal district, ethnical group, occupation –Laboratory findings:  32 types of haematology findings  51 types of chemistry findings  Cytology reports  Histopathology reports –Imaging studies:  Radiology procedure, site, diagnosis, morphology, topography, report, indication, department –Treatments:  Prescription drugs  IV chemotherapy  Radiotherapy  Surgical procedures –Diagnoses  Clinical diagnosis  Cause(s) of death Clinical narratives: Clinical narratives: –Additional facts not included in the structured data (e.g., non-cancer conditions) –Relations between facts - not only what happened, but why Structured data + Processed clinical narratives + Inference engine = CLEF chronicle

6. time The CLEF chronicle - the story of an illness Human: 1382 Mass: 1666 locus Pain: 5735 locus Radio: 1812 plans Chemo: 6502 plans treats locus target attends Ulcer: 1945 finding Cancer: 1914 finding Breast: 1492 locus Clinic: 4096 reason Biopsy: 1066 reason Clinic: 1024 plans Clinic: 2010 plans reason

7. Requirements Fast, on-the-fly generation of textual reports from complex EPRs Information extracted selectively and further aggregated to allow a 30 second overview of a patient’s history Presentation in an easily understandable format – natural language and graphics Alternative views of the patient record, e.g.: – –Summaries from various viewpoints – –Graphics, timelines and other visual aids Easy visual navigation, with the possibility of drilling down into events

8. Text vs Graphics Textual reports: Textual reports: –are easy to read and understand - natural language is the best medium of aggregating events, expressing causality relations, describing temporal relations –can be customised to the type of information needed –provide a quick way of identifying errors in the patient record However… However… –If all information is to be included, textual reports are likely to be very large –It can be difficult to find specific pieces of information in a large text document –Large amounts of text can be difficult to navigate through

9. Text vs Graphics Text is better for: Text is better for: –Presenting snippets of information, particularly if they contain relations between facts –Producing printable documents –Communicating information between medical practitioners Graphical visualisations are better for: Graphical visualisations are better for: –Presenting large amount of data in a small space (i.e., one computer screen) –Data navigation Solution: textual summaries backed up by a visual navigator

10. Textual summaries Types of summary: Types of summary: –Longitudinal: events are presented chronologically and schematically –Focused: events are presented in logical order according to the incidence of more important events, selected by the user (a summary of interventions, a summary of investigations of type x-ray)

11. Textual summaries Three steps: 1.Content selection - Select important events to include in a summary 2.Text structuring - Having a pool of events, arrange them in a coherent way to convey a certain viewpoint 3.Realisation -realise the defined text structure as paragraphs, sentences, phrases -format according to the summary type

12. Content selection Users can select types of information they want in a summary (e.g., a summary of interventions, a summary of investigations of type x-ray) Users can select types of information they want in a summary (e.g., a summary of interventions, a summary of investigations of type x-ray) Facts corresponding to these focal events will form the spine of the summary Facts corresponding to these focal events will form the spine of the summary Facts linked to spine events play a less important role in the summary and will be selected only if the depth of linking is lower than a set threshold Facts linked to spine events play a less important role in the summary and will be selected only if the depth of linking is lower than a set threshold

13. pain cancer breast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Problem pain breast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Interventions painbreast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Investigations

14. Textual summaries Three steps: 1.Content selection - Select important events to convey in a summary 2.Text structuring - Having a pool of events, arrange them in a coherent way to convey a certain viewpoint 3.Realisation -realise the defined text structure as paragraphs, sentences, phrases -format according to the summary type

15. Text structuring The discourse structure is governed by two types of information: The discourse structure is governed by two types of information: 1. relations present in the chronicle (19 different types of relations), which can be:  Attributive: Problem has_locus Locus –no discourse role, but impose local constraints (phrase expressed as a possessive, in the same sentence)  Rhetorical: Problem caused_by Intervention –Define the discourse relation to be used (e.g., causality, consequence, elaboration) 2. the type of summary: –Longitudinal: events occurring in the same week should be grouped together and further grouped into years –Logical: arrange chronologically and then group similar events (e.g., liver panels, screening consults)

16. Textual summaries Three steps: 1.Content selection - Select important events to convey in a summary 2.Text structuring - Having a pool of events, arrange them in a coherent way to convey a certain viewpoint 3.Realisation -realise the defined text structure as paragraphs, sentences, phrases -format according to the summary type

17. Realisation Aggregation Aggregation Enlargement of the liver + No enlargement of the spleen => Enlargement of the liver but not of the spleen Ellipsis Ellipsis Examination of the left breast revealed no recurrent cancer in the left breast => Examination of the left breast revealed no recurrent cancer Syntactic realisation Syntactic realisation –Simple phrase structure grammar –Spine events dictate which facts are most salient (i.e., will be in a more salient position in a sentence)

18. pain cancer breast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Problem The patient identifies pain in the left breast. A lump in the breast is found through a mammogram. A biopsy performed on the breast reveals cancer in the left breast. The patient receives radiotherapy to treat the cancer. Skin ulceration develops in the left breast as a result of radiotherapy, which is treated with hyperbaric oxygenation.

19. pain breast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Interventions Radiotherapy on the breast is initiated to treat cancer in the breast. A first radiotherapy cycle is performed. The radiotherapy causes skin ulceration. The patient receives hyperbaric oxygenation to treat the ulcer.

20. painbreast radiotherapy cycle Hyperbaric oxygenation radiotherapy lump mammogram biopsy cancer ulcer Investigations A mammogram is performed because of pain in the left breast, which identifies a lump in the breast. A biopsy of the lump identifies cancer in the left breast.

21. Typical output of the NL generator Year 1 Week 0 A mammography screening was scheduled at the clinic. A mammography screening was scheduled at the clinic. Week 1 Primary cancer of the right breast; histopathology: invasive tubular adenocarcinoma. Primary cancer of the right breast; histopathology: invasive tubular adenocarcinoma. YEAR 2 Week 131 Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast. YEAR 5 Week 287 Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast. YEAR 8 Week 443 Xray revealed cancer of the right breast. Xray revealed cancer of the right breast. Week 446 Examination (indicated by primary cancer of the right breast) revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Examination (indicated by primary cancer of the right breast) revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Testing (indicated by primary cancer of the right breast) revealed no abnormality of the haemoglobin concentration and no abnormality of the leucocyte count. Testing (indicated by primary cancer of the right breast) revealed no abnormality of the haemoglobin concentration and no abnormality of the leucocyte count. An Xray (indicated by primary cancer of the right breast) was performed. An Xray (indicated by primary cancer of the right breast) was performed. Very high level of the ESR concentration. Very high level of the ESR concentration. Very high level of the Creatinine concentration. Very high level of the Creatinine concentration. Very high level of the Alkaline Phosphatase concentration. Very high level of the Alkaline Phosphatase concentration. Very high level of the Bilirubin concentration. Very high level of the Bilirubin concentration. Very high level of the GGT concentration. Very high level of the GGT concentration. No abnormality of the platelet count. No abnormality of the platelet count. Week 449 An initial treatment planning was completed at the clinic. An initial treatment planning was completed at the clinic. Excision biopsy revealed no metastatic lymphnode count of the right axilla. Excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Histopathology revealed primary cancer of the right breast. Cancer staging revealed stage1 cancer. Cancer staging revealed stage1 cancer. Hormone anatagonist therapy was started to treat primary cancer of the right breast. Hormone anatagonist therapy was started to treat primary cancer of the right breast. Lumpectomy was performed on the breast to treat primary cancer of the right breast. Lumpectomy was performed on the breast to treat primary cancer of the right breast. Primary treatment package was started to treat primary cancer of the right breast. Primary treatment package was started to treat primary cancer of the right breast.…………………. YEAR 17 Week 893 Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast. Long chronological report

22. Typical output of the NL generator Focus on Problems In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular adenocarcinoma. In weeks 131 and 287 Xray revealed no cancer of the right breast. In week 446, there was no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes revealed by examination. There was no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration or of the ESR concentration. In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Lumpectomy was performed on the right breast. Hormone anatagonist therapy was initiated to treat primary cancer of the right breast. In weeks 457 to 737, there was no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. There was no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration and of the ESR concentration. In weeks 457 to 893, Xray revealed no cancer of the right breast. Compact reports Focus on Interventions In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular adenocarcinoma. In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Lumpectomy was performed on the right breast. Hormone anatagonist therapy was started to treat primary cancer of the right breast. Focus on Investigations In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular adenocarcinoma. In weeks 131 and 287 Xray revealed no cancer of the right breast. In week 446, examinations revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Testing revealed no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration or of the ESR concentration. In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. In weeks 457 to 737, examinations revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Testing revealed no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration and of the ESR concentration. In weeks 457 to 893, Xray revealed no cancer of the right breast

23. Visual navigator A single screen high level overview of a patient’s medical history A single screen high level overview of a patient’s medical history Various types of facts are displayed along timelines and colour- coded Various types of facts are displayed along timelines and colour- coded User interacts with the visualiser by selecting either: User interacts with the visualiser by selecting either: –Individual facts (e.g., a specific instance of radiotherapy) –Fact categories (e.g., Treatments) –Groups of facts Each selection is reflected in a generated textual summary build around spine events identified through the user selection Each selection is reflected in a generated textual summary build around spine events identified through the user selection Relations between facts can be highlighted on demand Relations between facts can be highlighted on demand Complex facts can be further delved into, revealing component facts Complex facts can be further delved into, revealing component facts Charts displaying the trend of numerical values are available for data that is suitable for such representation (i.e., blood test results) Charts displaying the trend of numerical values are available for data that is suitable for such representation (i.e., blood test results)

24. Visual navigator

25. Conclusions A method of presenting complex clinical data for use in medical care, using a combination of text and graphics A method of presenting complex clinical data for use in medical care, using a combination of text and graphics Unlike previous visualisation tools for medical data, our system works on a much richer input – the CLEF chronicle Unlike previous visualisation tools for medical data, our system works on a much richer input – the CLEF chronicle –No complex domain knowledge required –No complex system of data mining and inferences