Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ASSESSMENT OF NAS MILESTONES CAROLYN STALVEY, MD ERIK W. BLACK, PHD NICOLE M. PARADISE BLACK, MD MED
OVERVIEW
Interns by Department
Anesthesiology (n=12)
Emergency Med (n=8)
Family Medicine (n=10)
Internal Medicine (n=40)
Categorical (n=24)
Preliminary (n=16)
Neurosurgery (n=3)
Obstetrics and Gynecology (n=3)
Otolaryngology (n=2)
Orthopedics (n=4)
Pediatrics (n=17)
Psychiatry (n=8)
Surgery (n=13)
Urology (n=3)
Interns tested: N = 123
Stations Details Categories Post-Encounter
Chest Pain unstable angina with SP
Hx, PE, Communication Differential and Read EKG
SOB Pulmonary embolism with SP
Hx, PE, Communication SOAP note
RLQ Pain Ectopic pregnancy with SP
Hx, PE, Communication Handoff note
Mock Page Answer page, Infected PICC line with SN
Interprof Comm, Hx, Clinical Reasoning
Stations Details Categories Post-Encounter
Discharge Pneumonia Discharge instruction with SP
Education, Communication
Discharge Summary, CXR
Medical Error Disclosure medication error to SP
Education, Safety, Communication
Documentation note
Sterile Gown & Glove Nurse/grader in room Safety
Informed Consent Blood transfusion, with SP
Education, Communication
Test Blueprint - Categories
Case Name History Physical Exam Patient Safety Patient Education
InterProfession Communication
Clinical Decision Making
Communication Skills
Chest Pain X X X X X Gowning and
Gloving X Informed Consent X X
Medical Error X X X Mock Page X X X Discharge/ Pneumonia X X
RLQ Pain X X X X Shortness of
Breath X X X X
Test Blueprint – Notes, other skills
Case Name SOAP Note Discharge Note Documentation of Error Handoff Note EKG
Interpretation X-ray
Interpretation
Chest Pain X Medical Error X
Discharge/ Pneumonia X X
RLQ Pain X Shortness of
Breath X
Evaluators
Standardized patients/nurses History Physical Exam Communication items
OSCE Committee Notes EKG CXR
Milestones Addressed - IM
Case Name IM Milestones
Chest Pain 2MK: Knowledge of core content; 1PC: Diagnostic tests; 1PC Historical Data Gathering; 1PC Physical Examination; 4ICS Communicate Effectively;
Gowning and Gloving 6SBP: Recognizes systems error and advocated for system improvement
Informed Consent 1PC: Patient Management; 4ICS Communicate Effectively; 4ICS: Demonstrate Compassion and Respect
Medical Error 5P: Adhere to Basic Ethical Principles; 4ICS: Communicate Effectively
Mock Page 4ICS Interprofessional Team;1PC Clinical Decision Making; 1PC: Patient Management
Discharge/ Pneumonia
2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Clinical Reasoning
RLQ Pain 2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Historical Data Gathering; 1PC Physical Examination
Shortness of Breath 2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Historical Data Gathering; 1PC Physical Examination
Milestone Alignment
High frequency Milestones (IM) 1. Works effectively within an Interprofessional
Team (9) 2. Improves via Self Assessment (7) 3. Diagnostic Tests (6) 4. Participates in the education of all members
of health care team (6) 5. Communicates Effectively (6) 6. Demonstrates Compassion and Respect to
Patients (6)
PERFORMANCE SUMMARY
Performance by Category
Category Items MEAN SD
Patient Safety 15 87.99 22.50
Communication Skills 84 81.78 9.93
History 40 76.76 15.80
Physical Examination 30 74.43 21.00
Patient Education 29 73.44 19.40
Interprofessional Communication 2 60.97 26.80
Clinical Decision Making 7 45.12 25.00
Post Encounter
Case Post Encounter Mean SD
Chest Pain Differential Diagnosis 82.16 20.61
SOB SOAP Note 77.04 17.75
Pneumonia Interpret X-ray 75.10 43.19
RLQ Pain Handoff Note 67.56 16.21
Pneumonia Discharge Note 65.75 18.72
Error Documentation of Error 57.56 22.42
Chest Pain Interpret EKG 49.96 25.61
HCAHPS Items
Case Name Patient Education
Communication Skills
Chest Pain
X
Gowning and Gloving
Informed Consent X
Medical Error X
Mock Page
Pneumonia X
X
RLQ Pain
X
Shortness of Breath
X
1. Description symptoms to look for post discharge
2. Description of medication side effects
3. Physician listened 4. Physician
explained 5. Physician respected
HCAHPS
Item (Scale: 1 – 5) Mean Physician listened 3.67
Physician explained 3.60
Physician respected 3.84
Item (Yes/No) Frequency The resident told me about possible side effects of diarrhea or loose stool. 93%
The resident told me to seek medical attention if I have chills or fever higher than 101.5. 40%
The resident told me to contact my primary care doctor if I get worse. 37% The resident told me to seek medical attention if I have problems breathing. 33%
The resident told me to seek medical attention for chest pain. 25%
The resident told me to seek medical attention if my cough gets worse. 21%
General Areas of Weakness
Discharge Instructions symptoms to watch for smoking cessation (done 76%)
Sterile Gown and Glove Nearly all contaminated themselves at some point
Informed Consent Often don't mention it is ok to refuse transfusion
(done 80%) Allergic reactions (done 87%, 59%)
General Areas of Weakness
Handoffs What needs to be followed up What to do if..
Pulmonary exam (65%) Medical Error
Many don't ask if patient understood (done 44%) Neglecting to say it will be investigated and will get
back to you (done 73%)
UF vs. Non-UF Graduates
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Informed Consent
Medical Error
RLQ Pain Chest Pain Gown and Glove
Shortness of Breath
(PE)
Pneumonia Mock Page
Mea
n Sc
ore
UF
Non-UF
p =.02
p < .01
UF vs. Non-UF Graduates
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Mea
n Sc
ore
UF
Non-UF
p = .05
PD Reactions
Program directors described the OSCE as a positive experience
Helpful method of evaluating residents and identifying issues
A strong majority expressed interest in a retest (particularly primary care)
Next Steps
Continuous improvement and evaluation of OSCE 1 Remediation? OSCE 2 Continue to increase alignment with Milestones
Thank you