Upload
collin-clarke
View
212
Download
0
Embed Size (px)
Citation preview
Competency Tools to Measure Resident Progress
Carol Vandenakker-Albanese, MDResidency Program Director
Department of PM&R UC Davis Health System
November 15, 2014
ACGME Outcome Project Competency-based education focused on learner
performance in reaching specific objectives Long-term initiative by ACGME to emphasize educational
outcomes in the accreditation of residency training programs
Six core competencies for resident education• Medical Knowledge
• Patient Care
• Practice Based Learning and Improvement
• Systems Based Practice
• Professionalism
• Interpersonal and Communication Skills
The ‘Next Accreditation System’ in a Nutshell
Continuous Accreditation Model – annually updated• Based on annual data submitted, other data requested, and
program trends
RRCs will measure compliance through the evaluation of annual program data elements including:• Milestones (submitted twice a year)
• ACGME Resident Survey
• ACGME Faculty Survey
• Procedural /Case Log Data
Procedural Logs: Required(Observed and Performed)
EMG/NCSAxial epidural injectionAxial facetPeripheral joint/intraarticular injectionTendon sheath/bursa injectionTrigger pointsPeripheral nerve injectionBotulinum Toxin InjectionPhenol injectionProgramming baclofen pumpRefilling intrathecal baclofen pumpUltrasound extremityUltrasound guidance
Case Logs: Optional(admission, consult, outpatient)
Acute neuromuscular painAmputationBrain injuryChronic pain managementGeriatric rehabNeurologic disordersOncologic rehabPediatric rehabPost fracture and post-operative joint arthroplastyPulmonary and cardiacRheumatologicSoft Tissue disordersSpinal Cord InjuryStroke
Milestones are based on concept of Entrustable Professional Activity(EPA’s)
Real life patient care episodes, usually composed of elements of most if not all “competencies”
Benchmark of performance is the ability to be entrusted to perform care with “indirect supervision with direct supervision available”
Progression is then the achievement of EPA’s of increasing difficulty, risk, or sophistication
Proficiency is then the achievement of the most sophisticated EPA’s required of the resident
Uses and Implications of Milestones
ACGME• Accreditation –continuous
monitoring of programs; lengthening of site visit cycles
• Public Accountability –report at a national level on competency outcomes
• Community of practice for evaluation and research, with focus on continuous improvement
Residency Programs• Guide curriculum
development
• More explicit expectations of residents
• Support better assessment
• Enhanced opportunities for early identification of under-performers
Challenges/Opportunities Culture Change and Faculty Development
• Program Directors, Designated Institutional Officials
• Faculty Selecting Review Committee Members Improving quality of faculty observations/documentation The “Community of Educators” in each specialty has
come together and agree on:• core elements of the competencies (Milestones)
• levels of performance
• core methods of assessment
Program’s Assessment of Residents
Evaluation tools Competency Committee Semi-Annual Reporting to ACGME on
each milestone for all residents Annual reporting to ABPMR on resident
status and evaluations
Toolbox: Competancy Evaluation Tools
Begin by using existing tools and faculty observations • Exams: Written, Oral, SAE
• Clinical Observation: Simulation, ROCA
• Multi-Source Feedback: 360 evaluations, chart reviews
• Procedural Skill Evaluation• Workshops
• Clinical assessments
• Case Logs
Milestone Assessment
Determine which tools you will use to assess each milestone
Determine which rotations/ faculty are best to evaluate individual milestones
Determine which milestones may require additional tools • (Yearly updated Resident CV/ self-evaluation)
27 PM&R Milestones
Patient Care: 7 Systems Based Practice: 3 Practice-Based Learning and
Improvement: 3 Professionalism: 3 Interpersonal and Communication Skills: 2 Medical Knowledge: 9
PC 2. Physiatric Physical Exam (including medical, neurological, and gait adapted for age and impairment)
Rotations:• Inpatient: General
rehab, SCI, TBI
• Outpatient: MSK, neurorehab
• Peds
Tools:• ROCA
• PE workshop
SBP 2. Team approach to enhance patient care coordination. Rehabilitation team members may include OT, PT, SLP, etc.
Rotations:• Inpatient adult and
pediatric rehab
• Consults
• Outpatient multidisciplinary clinics
Tools:• 360 evaluations
• Faculty evaluations
PROF 3. Professional behaviors and accountability to self, patients, society and the profession
Rotations:• Any (not all)
• May want to pick most stressful
Tools• Faculty evaluations
• Program director/ coordinator evaluations
• Review of CV – involvement in service activities
MK 6. Musculoskeletal Disorders.
Rotations: • Outpatient clinics:
sports and spine
Tools:• Faculty evaluations
• ROCA
• Chart reviews
• SAE/ Written test
• Mock Oral Exam
Using Milestones as Evaluation Tool
Self Assessment on milestones annually
Schedule milestone assessments throughout year
Incorporate milestones into rotation objectives/ evaluations
Incorporate selected milestones into electronic evaluations
Faculty Education
Faculty must be familiar with milestones Reminder: Milestone levels NOT equivalent
to PGY level Distribute evaluation tools at start of rotation Identify which faculty are appropriate to
evaluate which milestones Review/ re-educate faculty based on quality
of evaluations
Key Elements of QualityEvaluation
Trained Observers- Core Faculty• Common understanding of the expectations
• Sensitive “eye” to key elements
• Consistent evaluation of a given level of performance
Establish Minimum Numbers of Quality Observations (CCC)
Interpreter/Synthesizer Experts• Clinical Competency Committee (Resident Evaluation
Committee)