Next Generation Electronic Medical Records and
Search: A Test Implementation in Radiology
David Piraino,MD Daniel Palmer, PhD Cleveland Clinic John Carroll University
Introduction
• Most patient specific medical information is document oriented with varying amounts of meta-data.
• Most of patient medical information is textual and semi-structured. • Electronic Medical Record Systems (EMR) are not optimized to
present textual information • EMRs currently show information in reverse time order only.
• This talk describes the construction and use of Solr search
technologies to provide relevant historical information at the point of care while interpreting radiology images.
Grand challenges (2008) in clinical decision support
• Improve the human–computer interface • Disseminate best practices in CDS design, development, and implementation
• Summarize and prioritize patient-level information • Prioritize and filter recommendations to the user • Create an architecture for sharing executable CDS modules and services • Combine recommendations for patients with co-morbidities • Prioritize CDS content development and implementation • Create internet-accessible clinical decision support repositories
• Use free text information to drive clinical decision support
• Mine large clinical databases to create new CDS
Dean F. Sittig et al, Journal of Biomedical Informatics 41 (2008) 387–392
Too Much Information (2012)
• In the time-pressured clinical setting, clinicians faced with large amounts of patient data in formats that are not readily interpretable often feel ‘information overload’.
Ketan Mane et al, Journal of Biomedical Informatics 45(2012) 101-106
What is out of place?
• Blue
• Green
• Cleveland
• Red
• Yellow
What is out of place?
• Boston
• new york
• Cleveland
• Chicago
• Denver
• San Diego
• atlanta
• Toronto
• Mexico City
• Columbus
• Nashville
• Paris
• Seattle
• Vancouver
• Washington DC
• Miami
• dallas
• Houston
Large number of images,
varying levels of
applicability, incomplete
histories, data stored in
many different locations
Chaos in Primary Care(2011)
Information Overload
Information Scatter
Unrelated Information
Mental Workload
Situation Awareness
Further Cognitive Influences Problem solving Problem identification Decision making Diagnosis Treatment
Moderators Interruptions Expertise Time
Information Chaos in Primary Care: Implications for Physician
Performance and Patient Safety
John W Beasley, MD1,2, Tosha B. Wetterneck, MD, MS3, Jon Temte, MD, PhD1, Jamie A
Lapin, MS2, Paul Smith, MD1, A. Joy Rivera-Rodriguez, MS2, and Ben-Tzion Karsh, PhD*,1,2
Journal of the American Board Family Medicine. 2011 November; 24(6): 745–751
1Department of Family Medicine, UW-Madison School of Medicine and Public Health
2Department of Industrial and Systems Engineering, UW-Madison
3Department of Medicine, UW-Madison School of Medicine and Public Health
Existing Information Confusion
ED visit
Telephone
Office
ED
Office
Admission
Surgery
Optho
ED visit
Telephone
Office
ED
Office
Admission
Surgery
Optho
Labs
CBC
PSA
Glucose
Potassium
Glucose
Urinalysis
Patient Image history presented as a list
Key components missing
Inspiration
• Boston 2012
And we hope to be inspired again this week with your help
Warning 28 Days Later
• One person with other full time job
• Running on moderately high end workstation
• Indexed 7 million radiology reports
• Providing types of searches that would otherwise be “impossible”
MRI shoulder without contrast
Relevant Previous Reports
MRI shoulder without contrast There is evidence for a full thickness tear of the supraspinatus tendon
Updated relevance
MRI shoulder without contrast There is evidence for a full thickness tear of the supraspinatus tendon
There is a partial tear of the subscapularis tendon with anterior medial
dislocation of the long head of the biceps tendon
Additional Update to Relevance
Evaluation
• 15 cases reported during clinical practice were used as test cases to determine if "similar" historical cases were found.
• For these 15 cases all searches completed within 3 seconds
• Considered only the top 10 matches returned by search
• Number of cases that illustrated the questioned diagnosis as determined by a bone and joint radiologist.
Results for the 15 cases
• Average performance: – 7.8 out of the 10 highest rated reports showed a
similar case highly related to the present case.
• Best performance: – 10 out of 10 cases relevant
• Worst performance: – only 2 out of 10 cases relevant
In Practice
• An example case: – Medical image: vascular mass in the hand
– LucidWorks search considered first 10 results • Based on text, eliminated unrelated cases
– Found and studied 2 pertinent cases • Showed similar masses with similar uncertainty
• Used to generate data sets for other research projects
Input Flow
Input Stream HL7 stream
or Delimited File
Solr XML with new fields
Solr Index and
repository Preprocess algorithm
Solr processing
Input Stream (HL7 Protocol)
XXXX|Date|XXX-01-01 |XXXX|XX:17:00.0|14||XXX-XXX-RADIOLOGY-CCF|XXX|XXX|CCF|I|XXXX|LMBR |XXXX|A|MRA OF HEAD|MR||||||* * *Final Report* * * DATE OF EXAM: XXXXX 12:07AM LMM 0432 - MRA OF HEAD / ACCESSION # XXXXX PROCEDURE REASON: cva * * * * Physician Interpretation * * * * RESULT: MRA OF THE HEAD WITHOUT CONTRAST HISTORY: Subarachnoidxxxx TECHNIQUE: Time of flight MRA of the cervical circulation was performed. COMPARISON: none FINDINGS: Examination is xxxxxxxx. IMPRESSION: Small xxxxxxxx. Transcriptionist: PSC Transcribe Date/Time: Jan 1 XXXX 10:14P Dictated by : XXXXXX, MD This examination was interpreted and the report reviewed and electronically signed by: XXXXX, MD On Date|
<add> <doc> <field name="department">Radiology</field> <field name="category">report</field> <field name="pid">EXXXXXX</field> <field name="sex">Male</field> <field name="id">XXXXX</field> <field name="did">XXXXX</field> <field name="modality">CT</field> <field name="title">MRI of the HEAD</field> <field name="date">XXX-01-09T09:34:00Z</field> <field name="year">XXX</field> <field name="month">01</field> <field name="day">09</field> <field name="hour">09</field> <field name="history">Subarachnoidxxxx</field> <field name="site">WRC</field> <field name="physician">XXXXX</field> <field name="body"> On the head XXXXXXXXXX on the base of the neck. </field> <field name="impression"> 1. XXXX. 2. XXXXXXX. 3. XXXXXXXX </field> <field name=“positive">XXXXXXXX</field> <field name=“negative">XXXX</field> <field name=“neutral">XXXX</field> <field name=“anatomy”>skull</field> <field name=“side”>none</field> </doc> </add>
Solr Input XML stream
Search Flow
Extracted text
Solr Query
Relevant Documents
Preprocess algorithm
Query Solr
Clinical encounter Radiology report
Data Extractor Data Extractor
More information
Processed Text
Solr query constructor
Similar Imaging Diagnosis
Patient: Anatomy, Modality, Diagnosis, and Time
Patient
Pathology
Lab
Patient
Clinical
notes
(Provider
Diagnosis
Time)
Speculative Interface
Solr – current implementation
Similar Imaging Diagnosis
Patient: Anatomy, Modality, Diagnosis, and Time
Patient
Pathology
Lab
Patient
Clinical
notes
(Provider
Diagnosis
Time)
ImageSphere
Challenges to Building Prototype
• Time vs. Data
• Sensitivity of queries
• Automating human scan/evaluation step
• Lack of a non-radiologist fitness function
• Migration from development-only LucidWorks platform to embedded Solr API queries
Time vs. Data
• 2-3 cases max viewed (10 considered)
• High relevance required
• Potentially 10’s of thousands to select from
Sensitivity of Queries
• Many query parameters
– proximity, boost, not
• Yields range of results
– 10/10 through 0/10
• 2 orders of magnitude in query times
• (wrist fracture)
• (wrist fracture)~2
• (wrist fracture)~10
• wrist^3 fracture
• -(no near fracture)
Queries: Good News/Bad News
• Basic queries provide great results
– Better than expected
– Top 10 results quickly yield cases to view
• Query refinement proves to be difficult
– Little or no correlation between query modifications and changes in results
– No consistent direction to investigate
Human in the Loop
• Top 10 results displayed in text form
• Human quickly scans and selects best
• Must maintain this ability in visual GUI
• Evaluation difficult because…
Fitness Function == Radiologist
• Need expert to determine value of query results
• Large impact on debugging…
• “Live” statistics gathering and provisional data gathering techniques
Migration for Prototype
• Manual process using LucidWorks proved concept
• Use Solr API to implement an automated delivery/display system
• Dependent on an intuitive user interface
Thank You and…
any Answers?
CONFERENCE PARTY
The Tipsy Crow: 770 5th Ave
Starts after Stump The Chump
Your conference badge gets
you in the door
TOMORROW
Breakfast starts at 7:30
Keynotes start at 8:30
CONTACT (optional)
David Piraino MD