LOCAL EXPERIENCES Innovation practices and experiences related to FIC development and implementation Xavier Pastor, Artur Conesa, Raimundo Lozano-Rubí. Medical Informatics Unit. Hospital Clínic. University of Barcelona. Natural Language Processing and Automatic Diagnosis Coding over the discharge reports of an Emergency Dep. at Hospital Clínic in Barcelona

Departure situation (January 2010) Hospital Clinic of Barcelona: Average activity of the Emergency Department (ED): 350 visits/day. Patient diagnoses in HIS are typed directly as a literal expression by the ED physicians. Unsupervised ICD-9-CM coding by the administrative staff of the ED using some help (summary list of the main codes). Difficulty of reliable analysis of clinical information. Commitment of an Emergency Minimum Data Set by the Health Authority. Funding cuts-off because the economical crisis.

To implement a coding engine based on artificial intelligence technology that allows: the semantic analysis of natural language of diagnostic expressions. the assignment of ICD-9-CM codes in relation to available models. To accomplish the commitment with savings To return better information about ED activity with a shorter delay. Objectives of the project

EPR System (SAP-ISH*Med TM ) ED Physicians introduce the diagnoses typing text in an specific field of EPR Control and supervision of coding CodingSuite CodingSuite™: Platform of automatic coding CodingSuite™ receives diagnoses typed by physicians at EPR and, through a webservice, analyzes the semantic closeness in relation to models and finally assigns an ICD-9-CM code with a confidence index. CodingSuite™ includes a software to monitor and control the results of coding. It allows the inclusion in the database of new benchmarks with a CI=100. Methodology Ok? CI>75% 5 6 code No

The likelihood of a code identifying a diagnosis is the result of the product of probabilities assigned to each level of analysis performed. Automatic diagnoses coding Coding S uite™ fr diafisaria de tibia i fractura diafisaria tercio distal de tibia/peroné fractura diafisaria tibia izq fractura diafisaria tibia-peroné d fractura diafisaria tibia y perone fractura diafisaria tibia y perone izq fractura diafisari tibia izquierda fractura diafisari tibia-perone izq fractura diafisiaria tibia fx diafisiaria tibia + peroné fx diafisiaria tibia d fractura diafisiaria tibia peroné fract. diafisis de tibia con peroneabierta fr diafisis tibia cerrada st Identification – 98,35% nd Identification – 98,35% * 99,57 = 97,92% rd Identification – 97,92 * 95 = 93,03% webservice Analysis of semantic closeness with the models (NLP)

Initial results (2010) Initial training of CodingSuite™ with a coded corpus of one year of previous activity at the ED.(130,000 pairs of diagnoses-codes). Initial CI set at 75% of CI. CodingSuite™ learns: increase the percentage of automatic coding: June: 69%, July: 73%, August: 82% … but Assignment of wrong codes because of consistent errors in the training database. Dispersion in the reference models: Decrease of the previously existing CI Gradual increase in the review queue Small effect of expert proposed coding

Corrective actions  January 2011: Removal of all diagnoses not having been validated.  March 2011:  Automatic recoding of the review queue  More frequent (weekly) database training  Progressive decrease of CI level from 75 to 66%  September 2011: ERROR RATE OF AUTOMATIC CODING: 1.25% AUTOMATIC CODING RATE (including confirmations): 90%

Poster 424 presented by Dr. Artur Conesa in the EIC session at 2014 WHO-FIC network meeting on Monday, 13 rd of October Details of corrective actions

Global results

Workstation for clinical coders Machine learning

CodingSuite™ (5 sec.)

3940 ICD-9-CM codes assigned to textual expressions (1:26) Number of daily ED discharge reports to review: Knowledge about diagnosis at ED (Abdominal pain): 2949 (2,79%) (Urinary tract infection): 2359 (2,23%) (Premature delivery threat): 2034 (1,93%) 2014 data

Conclusions 1. CodingSuite™ is able to learn to code automatically diagnoses in an Emergency Department of a hospital. However, the quality of the initial data used with an automatic diagnosis coding software is essential to ensure its efficiency and continuity. 2. CodingSuite™ has demonstrated: Usefulness for automatic coding (92% right now). Capacity to treat big volumes of clinical data. Easy integration with the EPR. 3. This methodology can be extended to other environments with higher activity (Outpatient clinic, One-day-stay hospital, etc…) 4. It is advisable a continued monitoring of coding results, especially in the early stages of implementation of such a system. 5. Accomplishment of the objectives: Delivery on time of the ED Minimum Dataset No more human resources needed Better quality of the information about ED Faster availability of the information about ED

Thank you very much for your attention !