Upload
george-gkotsis
View
280
Download
1
Embed Size (px)
Citation preview
Don't Let Notes Be Misunderstood:
A Negation Detection Method for Assessing Risk of Suicide in
Mental Health RecordsGeorge Gkotsis, Sumithra Velupillai, Anika Oellrich
Harry Dean, Maria Liakata and Rina Dutta
Biomedical Research Centre Nucleus – King’s College London
e-HOST-ITElectronic health records to predict HOspitalised Suicide attempts:Targeting Information Technology solutions
AimTo determine whether structured and free-text data in Electronic Health Records (EHRs) can be used to quantify changes in symptoms, behaviour patterns and health service-utilisation and predict serious suicide attempts
Motivation
• Health records contain many structured fields, such as:• Personal information/contact• Diagnosis• Prescription• Interventions• Scans & Measurements
Most of the structured fieldsare left blank
The mysterious case of health records (1/2)
Max Weber
Theory ofFormal Rationality
The mysterious case of health records (2/2)• Free text contains a lot of information
• Traditional information access technology returns many false positives
Example1. Patient is suicidal
2. Patient is not suicidal
• Meaning can be expressed in multiple ways
Example1. He has suicidal thoughts2. He wants to end his life3. She wants to kill herself
CRIS database• 226,000 patients• 18.6 million documents (Event)
• Suicide-related data• 783,000 documents contain the word suicid*• 111,000 patients
Anonymous Reviewer:“Overall,I think the paper is well thought out and written, and I am
envious of their access to such a large patient dataset”
Problem description
Negation detection – definition (1/2)
“The determination of whether a finding or disease mentioned within narrative medical
reports is present or absent”*
Negex, Chapman et al.Journal of Biomedical Informatics, 2001
Negation detection – definition (2/2)
Negation Detection
Sentence
Target keyword
Positive/Negative
Towards negation detection resolution• Fundamental NLP taskReduced to identifying the scope of negation
Examples:
No issues other than her indicating that she might commit suicide
He continues to deny any suicidal thoughts and is happy to come to the XXX for medical review tomorrow
+
-
State-of-the-art: Negex (1/2)• Lexical-based approach
• Collection of negation cues/expressions• Pseudo-negation expressions• Termination cues for scope• Search scope of 6 words surrounding the target keyword
• pyConTextNLP
State-of-the-art: DEEPEN (2/2)• Wrapper over NegEx• Applied over the (predicted to be) negated sentences• Uses a dependency parse tree
“Negation’s Not Solved”*• Optimizable, but not generalizable• Annotation guidelines are different• Spans considered can be nouns or whole phrases• Amount of overlap allowed (or not)
*Wu et al.PLOS One, 2014
Proposed solution
Workflow
Annotated Dataset
Preprocessing
CoreNLPSentence
Parse tree
Target Node
Negation Detection
1. Pruning
2. Identification of
dominating
subordinate
clause
3. Identification of
negation
governing the
target-node
4. Negation
resolution
Positive/Negative
TargetKeyword‘suicid*’
MHRs
‘suicid*’
Annotation
Dataset and annotation
Proposed Methodology
2941
3125
Positive Negative
Dataset and annotation• Generation
• Random sampling from SLAM Events of 6k sentences containing the word “suicid*”
• Annotation• One expert annotated the complete corpus• Another expert repeated the annotations for 25% of the sentences𝛋=0.93 (IAA=97.9%)
Limitations• Linguistic focus• Patient-agnostic
1. PruningLabels• Subordinate conjunctions• ,• S• SBAR• SINV
2. Identification of dominating subordinate clause
SBAR
3. Identification of governing nodesS
Evaluation
Comparison1. Proposed Model (uses 15 negation words)2. NegEx (uses 272 rules)3. pyConTextNLP-N (uses [2])4. pyConTextNLP-O (uses [1])
no, without, nil,not, n't, never, none, neith, nor, nondeny, reject, refuse, subside, retract
Results (1/3)
Positive NegativePositive 2782 331Negative 159 2794
Total 2941 3125Pred
ictio
n
Class
Results (2/3)
Precision Recall FM AccuracyNegEx 93.4 92.1 92.8 93
pyContextNLP-N 94.1 92.9 93.5 93.7pyContextNLP-O 80.7 86 83.2 83.2
Proposed 89.4 94.6 91.9 91.9
Results (3/3)
Discussion & Conclusion
Distribution of sentence length
Proper punctuation and sentence chunking are crucial!
Discussion• Corpus of 6k sentences from Mental Health Records• Annotation of high quality• Evaluation - focus on positive cases
• Parse trees+Require minimum number of negation keywords+Further potential
• Statement extraction (subject-predicate-object)• Temporal characteristics• Degree of suicidality
- Expensive - Error prone for long sentences
Future work (1/2)Expand on expressions of suicidality
oStudy how negation detection can be used to strengthen predictive power of mental health recordsoOngoing cohort study (pupils with ASD)o Large scale study based on hospitalisation events
Future work (2/2)• Evaluate our tool against other datasets/domains
• Consider syntactic dependency parser (instead of constituency-based)• spaCy• SyntaxNet
https://github.com/gkotsis/negation-detection
Source code