LOCAL EXPERIENCES

Innovation practices and experiences related to FIC development and implementation

Xavier Pastor, Artur Conesa, Raimundo Lozano-Rubí. Medical Informatics Unit. Hospital Clínic. University of Barcelona.

Natural Language Processing and Automatic Diagnosis Coding over the discharge reports of an Emergency Dep. at Hospital Clínic in Barcelona

Departure situation(January 2010)

• Hospital Clinic of Barcelona: Average activity of the Emergency Department (ED): 350 visits/day.

• Patient diagnoses in HIS are typed directly as a literal expression by the ED physicians.

• Unsupervised ICD-9-CM coding by the administrative staff of the ED using some help (summary list of the main codes).

• Difficulty of reliable analysis of clinical information.

• Commitment of an Emergency Minimum Data Set by the Health Authority.

• Funding cuts-off because the economical crisis.

To implement a coding engine based on artificial intelligence technology that allows:

• the semantic analysis of natural language of diagnostic expressions.

• the assignment of ICD-9-CM codes in relation to available models.

To accomplish the commitment with savings

To return better information about ED activity with a shorter delay.

Objectives of the project

EPR System(SAP-ISH*MedTM)

ED Physicians introducethe diagnoses typing text in an specific field of EPR

Control and supervision of codingCodingSui

te

CodingSuite™: Platform of automatic coding

• CodingSuite™ receives diagnoses typed by physicians at EPR and, through a webservice, analyzes the semantic closeness in relation to models and finally assigns an ICD-9-CM code with a confidence index.

• CodingSuite™ includes a software to monitor and control the results of coding. It allows the inclusion in the database of new benchmarks with a CI=100.

Methodology

1

2

3

4

Ok?CI>75

%

5

6

code

No

The likelihood of a code identifying a diagnosis is the result of the product of probabilities assigned to each level of analysis performed.

Automatic diagnoses coding

CodingSuite™

fr diafisaria de tibia i

fractura diafisaria tercio distal de tibia/peroné

fractura diafisaria tibia izq

fractura diafisaria tibia-peroné d

fractura diafisaria tibia y perone

fractura diafisaria tibia y perone izq

fractura diafisari tibia izquierda

fractura diafisari tibia-perone izq

fractura diafisiaria tibia

fx diafisiaria tibia + peroné

fx diafisiaria tibia d

fractura diafisiaria tibia peroné

fract. diafisis de tibia con peroneabierta

fr diafisis tibia cerrada

824823

822

1st Identification –

98,35%

823.0 823.

1 823.2823.

0 823.0 823.

9 823.6

2nd Identification – 98,35% *

99,57 = 97,92%

823.20

823.20823.22 3rd Identification

–97,92 * 95 = 93,03%webservi

ce

Analysis of

semantic closeness with the models(NLP)

Initial results (2010)

• Initial training of CodingSuite™ with a coded corpus of one year of previous activity at the ED.(130,000 pairs of diagnoses-codes).

• Initial CI set at 75% of CI.• CodingSuite™ learns: increase the percentage of automatic

coding:June: 69%, July: 73%, August: 82% …

but

• Assignment of wrong codes because of consistent errors in the training database.

• Dispersion in the reference models: • Decrease of the previously existing CI• Gradual increase in the review queue • Small effect of expert proposed coding

Corrective actions

January 2011: Removal of all diagnoses not having been validated.

March 2011: Automatic recoding of the review queue More frequent (weekly) database training Progressive decrease of CI level from 75 to

66%

September 2011: ERROR RATE OF AUTOMATIC CODING: 1.25% AUTOMATIC CODING RATE (including confirmations): 90%

Poster 424 presented by Dr. Artur Conesa in the EIC session at

2014 WHO-FIC network meeting on

Monday, 13rd of October

Details of corrective

actions

2010 2011 2012 2013 2014

78.6% 80.9%

89.4% 90.5% 91.4%

7.8% 7.3%3.2% 3.4% 3.7%

13.5% 11.8%7.4% 6.1% 4.9%

Coded Confirmed Reviewed

Global results 2010 - 2014

Workstation for clinical coders

Mach

ine learn

ing

362.76

362.53

CodingSuite™(5 sec.)

Chapter Description 2012 2013 201416 SYMPTOMS, SIGNS, AND ILL-DEFINED CONDITIONS 17,8% 17,5% 17,1%17 INJ URY AND POISONING 18,2% 16,8% 15,9%6 DISEASES OF THE NERVOUS SYSTEM AND SENSE ORGANS 11,3% 10,3% 11,4%V FACTORS INFLUENCING HEALTH STATUS AND CONTACT WITH HEALTH SERVICES 6,9% 7,4% 7,3%13 DISEASES OF THE MUSCULOSKELETAL SYSTEM AND CONNECTIVE TISSUE 7,3% 7,0% 7,0%8 DISEASES OF THE RESPIRATORY SYSTEM 6,8% 6,6% 6,8%10 DISEASES OF THE GENITOURINARY SYSTEM 5,7% 6,4% 6,8%9 DISEASES OF THE DIGESTIVE SYSTEM 7,1% 6,5% 6,3%7 DISEASES OF THE CIRCULATORY SYSTEM 6,6% 6,1% 5,5%11 COMPLICATIONS OF PREGNANCY, CHILDBIRTH, AND THE PUERPERIUM 1,2% 4,2% 4,5%12 DISEASES OF THE SKIN AND SUBCUTANEOUS TISSUE 2,9% 2,7% 2,8%5 MENTAL DISORDERS 2,5% 2,5% 2,8%1 INFECTIOUS AND PARASITIC DISEASES 2,1% 2,1% 2,2%2 NEOPLASMS 1,6% 1,5% 1,4%3 ENDOCRINE, NUTRITIONAL AND METABOLIC DISEASES, AND IMMUNITY DISORDERS 1,2% 1,2% 1,2%4 DISEASES OF THE BLOOD AND BLOOD-FORMING ORGANS 0,8% 1,0% 0,8%14 CONGENITAL ANOMALIES 0,1% 0,1% 0,1%15 CERTAIN CONDITIONS ORIGINATING IN THE PERINATAL PERIOD 0,0% 0,0% 0,0%

3940 ICD-9-CM codes assigned to 101113 textual expressions (1:26)

Number of daily ED discharge reports to review: 15-20

Knowledge about diagnosis at ED

789.00 (Abdominal pain): 2949 (2,79%)599.0 (Urinary tract infection): 2359 (2,23%)644.13 (Premature delivery threat): 2034 (1,93%)

2014 data

Conclusions1. CodingSuite™ is able to learn to code automatically

diagnoses in an Emergency Department of a hospital. However, the quality of the initial data used with an automatic diagnosis coding software is essential to ensure its efficiency and continuity.

2. CodingSuite™ has demonstrated:• Usefulness for automatic coding (92% right now).• Capacity to treat big volumes of clinical data.• Easy integration with the EPR.

3. This methodology can be extended to other environments with higher activity (Outpatient clinic, One-day-stay hospital, etc…)

4. It is advisable a continued monitoring of coding results, especially in the early stages of implementation of such a system.

5. Accomplishment of the objectives:Delivery on time of the ED Minimum DatasetNo more human resources neededBetter quality of the information about EDFaster availability of the information about ED

Thank you very much for your attention !

xpastor@clinic.cat