View
31
Download
4
Category
Tags:
Preview:
DESCRIPTION
IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID. Balaji Polepalli Ramesh , Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu. Outline. Background The NoteAid System & Evaluation Results Discussion Conclusion & Future Work. Background. - PowerPoint PPT Presentation
Citation preview
IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID
Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu
Outline
Background The NoteAid System &
Evaluation Results Discussion Conclusion & Future Work
Background
Patients reading their clinical notes has the potential to Enhance medical understanding Improve healthcare management and
outcome
Background
“A patient with hx of active tobacco abuse, bronchitis, and psoriasis presented to ED earlier today with c/o SOB, mild wheezing, chest congestion and chills”
The NoteAid System
NoteAid system - automatically Identifies clinically relevant concepts Links concepts to their definitions
The NoteAid System
The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.
Knowledge Resources
(Medline Plus, UMLS, Wiki)
NLP Approaches
NoteAid
The NoteAid System
The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.
EGD – Acronym for Esophagogastroduodenoscopy.It is a test to examine the lining of the esophagus (the tube that connects your throat to your stomach), stomach, and first part of the small intestine. It is done with a small camera (flexible endoscope) that is inserted down the throat. EGD may be done if you have symptoms that are new, cannot be explained, or are not responding to treatment, such as: Black or tarry stool, Feeling full sooner than normal or after eating less than usual, Swallowing problems or pain with swallowing, heart burns and others.
Knowledge Resources
(Medline Plus, UMLS, Wiki)
NLP Approaches
NoteAid
The NoteAid System
A knowledge resource MedlinePlus UMLS – Unified Medical Language System Wikipedia
NLP approaches
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concepts
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
Evaluation
Four NoteAid implementations MedlinePlus UMLS Wikipedia Hybrid (MedlinePlus+UMLS+Wikipedia)
Evaluation Data
From the Pittsburgh NLP repository 20 Progress Note reports 20 Discharge Summary reports
Subjects
The Amazon Mechanic Turk Has shown to be reliable for medical
annotations and evaluations Recruited 64 subjects
8 systems (4 implementations X 2 types of EHR notes)
8 subjects per system 59 subjects completed the evaluation
Demographic Information
Evaluation Process
Each subject read 20 EHR notes before and after NoteAid implementation Randomly assigned to an implementation Self-reported comprehension on a scale of
1 to 5
Readability
Flesch-Kincaid grade level (Grade Level)
Evaluation Data Statistics
Type Discharge Summaries
Progress Notes
No of Reports 20 20
Total (Avg) # of sentences
355 (17.8) 473 (23.7)
Total (Avg) # of words
2362 (118) 4862 (243)
Avg Flesch-Kincaid Grade
Level
8.8 9.8
Readability and Self-Reported Comprehension
Grade Level correlated with self-reported comprehension score (ρ=-0.47, p<0.05, Spearman rank
correlation)
Results
Average comprehension scores before and after each implementation
*p<0.05, Non-parametric Wilcoxon signed-rank test
System
Discharge Summaries Progress Notes
Before After diff (%) Before After diff (%)
MedlinePlus
3.52 3.49-0.03 (-
0.9)3.18 2.86
-0.32 (-10.1)
UMLS 3.80 3.81 0.01 (0.3) 3.75 4.01 0.26 (7.0)
Wiki 3.57 4.140.57
(16.0)3.45 4.53
1.08* (31.3)
Hybrid
3.86 4.02 0.16 (4.1) 3.40 4.54 1.14 (33.5)
Number of Concepts Identified Total number of concepts that were
recognized by three different NoteAid implementationsSystem Discharge
SummariesProgress Notes
MedlinePlus 37 53
UMLS 171 362
Wiki 190 427
Readability
UMLS Definition – Coagulopathy
“Hemorrhagic and thrombotic disorders that occur as a consequence of abnormalities in blood coagulation due to a variety of factors such as COAGULATION PROTEIN DISORDERS; BLOOD PLATELET DISORDERS; BLOOD
PROTEIN DISORDERS or nutritional conditions” Wiki Definition – Coagulopathy
“Coagulopathy is a condition in which the blood’s ability to clot is impaired. This condition can cause prolonged or excessive bleeding, which may occur spontaneously or following an injury or medical and dental procedures. The normal clotting process depends on the interplay of
various proteins in the blood”
Discussion
Text readability correlated with the comprehension scores
Wiki performed the best Content coverage Readability
Limitations
Limitation Lay people not patients performed evaluation Order-effect bias Score subjects’ EHR note comprehension but
not to what extent they accurately comprehended the EHR note content
Conclusion and Future Work
NoteAid improved EHR note self-reported comprehension
Future Work Improve concept coverage and filtering Evaluate quality of the definitions Evaluate content comprehension Evaluate system in which patient read their
own EHR notes
Acknowledgements
1R01GM095476 University of Massachusetts Medical
School National Center for Advancing
Translational Sciences of the National Institutes of Health under award number UL1TR000161
http://clinicalnotesaid.org
Thank You and Questions
Recommended