Upload
randell-james
View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Residents In Trouble
M. Rebecca Hoffman, MD, MSPH
December 8, 2008
Academy for Scholarship in Education
Overview
Scope of the Problem Literature Review Study Design Study Results (in progress)
Scope of the Problem
The goal of Family Medicine Residency
Residents’ backgrounds
No guidance
What does the RRC say?
Must have advising system
Must give regular feedback
BUT… no real guidance regarding what/how
Consequences of RIT
For the individual RIT Psychological stress Financial stress Eventual job security
For the program Adequate coverage of duties Morale Faculty time/energy Dismissal
Consequences…
Other consequences Recruiting Impact on patient care
Literature Review
Not much on the prevalence of RIT Reamy et al (1): 25 year in-depth study of one
family medicine residency program 9.1%
Yao and Wright (2): Survey of Internal Medicine PDs– 6.9%
Williams et al (3): SIU Gen Surgery Residency 22%
Kinds of Problems
Knowledge Deficits Attitudinal Problems Interpersonal Conflict Psychiatric Illness Substance Abuse Family Stress Relationship Disruption
Other Classification
Academic Performance Classroom, test
Clinical Performance Applied knowledge/skills
Professional Behavior Workplace behavior, relationships with other
professionals, etc Others
Difficult to classify
So we can name it…
But what do we do with the information?
Literature reviews: “Need to do better” Core content review, quizzes Counseling Treatment of illnesses Repetition (rotation, year) Probation Dismissal Others
Summary of the problem
We know it’s there, but we aren’t sure exactly how common it is
We aren’t good at predicting who will struggle
Once we identify them, we don’t know what to do with them
RIT Family Medicine Study
Primary Aims
1) Determine the prevalence of RIT in the SIU Family Medicine Residency Programs
2) Describe and categorize the types of troubles encountered in the FCM residency programs
RIT Aims...
3) Identify and describe the kinds of processes the programs employ to address deficiencies with RIT
4) Describe outcomes of the residents who underwent remediation
5) Identify early indicators to predict RIT
RIT Study Design
Chart review/resident record -based
Prevalence over significant timeframe 10 years of entering classes Entering class of 1993-entering of 2003
Why?
De-identified data from all 4 programs To further assure confidentiality No one program singled out
RIT Study
Initially modeled after Reamy et al study (1) after discussion with Dr. Reamy about the process in his residency program
Assistance from Reed Williams, PhD and Nicole Roberts, PhD
Methods
Rosters of all entering residents for the 10 years from all programs
Random ID numbers assigned
Records from each program reviewed in their entirety Mostly paper Some electronic
Methods…
Data collection instrument created based on personal experience, literature, and discussion with Reed Williams and Nicole Roberts
Feedback from academic faculty
Suggestions incorporated as needed
Collected for all residents:
US Medical School Grad (Y/N) USMLE or NBOME or COMLEX scores
Step 1, Step 2, Step 3 (when available) Gender (M/F) ITE Scores
PGY1, PGY2, PGY3 (when available) Did the resident have substantial difficulties?
(Y/N)
For those who struggled
Time frame- when first identified Categorization of the main problem
Academic, Clinical, Professionalism, Other Main performance area involved
See list Breadth of performance areas
Academic, Clinical, Professional, Other All performance areas involved
See list
For those who struggled
What remediation attempted? See list
What was the final resolution? Graduated with concerns Graduated without concerns Voluntarily left Dismissed Finished on probation Not documented
Severity Rating
Compared to other residents with performance problems, this resident’s problems were: Likert scale, 1-5
1 among the least serious 5 among the most serious
Narrative descriptions of overall case, time course, etc
Example of Survey Instrument Survey Monkey for data entry
Data Collection
2 raters
Independently reviewed same 5 charts at first program to establish inter-rater reliability, then 3 charts at each other program
Data Collection…
Then independent record review
Any case the reviewer identified as a RIT was then reviewed by the other reviewer independently, ratings discussed after each case
“Borderline cases” also reviewed by both
Data Analysis
Data entered into Survey Monkey for ease of collection
Analysis: descriptive and basic statistics
When all data complete, analysis using SPSS
Results
96 Residents reviewed (as of 12/4/2008)
US Med School Grads: 69% (66)
Male: 65% (62)
Female: 35% (34)
Prevalence of documented RIT: 38.5% (37)
RIT (N=37)
US graduates: 59.5% (22) Foreign graduates: 40.5% (15)
Male: 70.3% (26) Female: 29.7% (11)
Male, US: 40.5% (15) Male, Non-US: 29.7% (11) Female, US: 18.9% (7) Female, Non-US: 10.8% (4)
RIT
Performance Area Number of Residents
with the problem
Academic 6
Clinical 19
Professional 11
Other 1
Most Important Single Problem Area? Knowledge (10) Putting everything together (3) Data interpretation/diagnosis (2) Data collection (2) Incomplete paperwork/charts (2) Lack of motivation/interest (2) Treatment/management (2)
Breadth of Problems
Area(s) of Problems Number
Academic Only 1
Academic and Clinical 7
Academic and Professional 3
Clinical Only 1
Clinical and Professional 12
Academic, Clinical, and Professional 6
Professional Only 6
Other 1
Breadth of problems
Number of performance areas involved ranged from 1 to 23 Average: 7
See breakdown of individual RIT on handout
Remediation
Ranged from “none” or “told to improve” to 12 or more interventions for a single resident
Most commonly reported: Increased meetings with advisor/mentor
Outcomes of RIT (residency)
26 graduated from the program (70.3%) 11 did not graduate (29.7%)
Left voluntarily: 6 Changed specialty: 2 Dismissed: 3
More complete follow up not complete Have initiated licensure status and board
certification F/U but too little data to report
Outcomes
Outcome Number
Finished with no concerns 15
Finished with ongoing concerns 10
Voluntarily left 8
Dismissed 3
Resolution not documented 1*
Conclusions?
RIT more common than suspected based on lit. Increased prevalence compared to other studies
? Over-diagnosis For “mild cases”
? Truly higher prevalence Due to population? Due to program characteristics?
All RIT who had concerns in more than 8 areas had ongoing concerns at graduation or did not complete program
RIT who struggled in all 3 performance areas also either did not complete or graduated with ongoing concerns
Professionalism issues very common
Problems Encountered
Information in files varies between programs Missing data
Esp. test scores (fortunately, may be able to acquire) Even within programs, varying data Classification of single most important area for major
problem residents We collected data for an additional group of
residents: those with ITE performance less than 10th %ile, even if no other problems, will follow those as well (n=4 right now)
To come!
Full data reporting (approx 270)
Including USMLE and ITE data
Predictive model?
Follow up data (post-graduation)
Special Thanks to:
Nicole Roberts, PhD data spelunker extraordinaire Reed Williams, PhD Steve Verhulst, PhD Jerry Kruse, MD, MSPH SIU Family Medicine Residency Program Directors:
Janet Albers, MD John Bradley, MD Tom Miller, MD Penny Tippy, MD
Wiley Jenkins, PhD
Referenced Articles
1. Reamy BV, Harman JH. Residents in trouble: an in-depth assessment of the 25-year experience of a single family medicine residency. Fam Med 2006;38(4):252-7.
2. Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. JAMA 2000;284:1099-1104.
3. Williams RG et al. The nature of general surgery resident performance problems (publication pending)