Defining and Assessing Entrustable Professional

Activities  

Karen E. Hauer, MD

H. Carrie Chen, MD, MSEd

John Q. Young, MD, MPP

Patricia S. O’Sullivan, EdD

With thanks to Olle ten Cate, PhD

UCSF

June 5, 2013

Agenda1-1:15 Welcome1:15 Introduction to EPAs1:45 individual writing of an EPA2:05 overview of EPA development2:35 small group activity, design an EPA (steps 1-4) Break3:15 small group reporting3:45 overview of steps 5-74:00 second activity, evidence for an EPA (steps 5-

7) 4:30 small group reporting and wrap up (all)

Workshop objectivesBy the end of this workshop,

participants will be able to:Define an entrustable professional

activity (EPA)Develop an entrustable professional

activity that can be used for assessment of medical trainees

What do you hope to learn today?

ACGME core competencies:Patient CareMedical Knowledge Practice-based Learning and

Improvement Interpersonal and Communication Skills Professionalism Systems-based Practice 

Current Assessment System

Practice-based Learning and Improvement (Housestaff)  (Question 5 of 9 - Mandatory)

Assimilates evidence from scientific studies or consultants related to their patients' health problems. Participates in

organized transitions of care curriculum as specified at each site. Actively seeks out and acts on performance feedback from the supervising attending and other team members.

Insufficient Contact to

Judge 

1 & 2 = Unsatisfactory3 = Marginal 

  

Satisfactory/Meets Expectations 

  

7 & 8 = Excellent9 = Outstanding 

0 1 2 3 4 5 6 7 8 >> 9 <<

Current Assessment System

Current Assessment System

What are some limitations of current assessment systems?

Core competencies and sub-competencies: long checklists of behavioral descriptors

Relies on traditional but limited assessment methods – knowledge exams, ward evaluations

Not a holistic summative view of the trainee

CompetencyCompetency

An observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes.

The International CBME Collaborators, 2009

CompetentCompetent

Possesses the required abilities in all domains in a certain context at a defined stage of medical education or practice.

The International CBME Collaborators, 2009

CompetenceCompetence

Competence entails more than the possession of knowledge, skills and attitudes; it requires you … to apply these [abilities] in the clinical environment to achieve optimal results.

does the job

ten Cate, Med Teach, 2010

Can you trust the learner to function independently?

How to Drive a CarHow to Drive a CarCompetency

Can accelerate and brake smoothlyCan approach an intersection and can turn left

CompetentPasses driver’s education classesPasses driver’s exam to get the license

CompetenceDrives safely on interstate or during bad

weather, avoids accidents, no traffic ticketsParents hand over the keys and walk away

What is the goal with assessment?

Integrated, longitudinal, learner-centered assessment system

Promote skill acquisition in multiple domains concurrently

Assess what learners actually do in practice

Be able to conclude: this is a trustworthy trainee

Features of genuine competency based medical

education

Outcome-based, not process-based: what is attained is key, not just what is done or taught

Integrates knowledge, skill, attitude

Time-independent: length of training for defined outcomes is not pre-set

Individualized: trainees and contexts vary

Workplace-learning based

Lifelong learning oriented

Competence. What criteria would you use

to select your doctor?

Passed all tests and exams?

Grades and scores?

Years of training?

Follows protocols and guidelines?

Trust that s/he will manage a case in the best possible way?

Entrustable Professional Activities

(EPAs)

Entrustable Professional Activity (EPA)

Define important clinical activitiesLink to competencies / milestonesInclude professional judgment of competence

by cliniciansMake “decisions of entrustment” for

“entrustable” activitiesSomething changes for the learner….

Build collection of mastered EPAs (portfolio) to document full competence

Definition of an Entrustable Professional

ActivityA core unit of work, reflecting a responsibility that

should only be entrusted upon someone with adequate competencies

More specific: Part of essential professional work in a given context Independently executable, within a time frame Leads to recognized output of professional labor Observable and measurable in process and outcome,

leading to a conclusion (“well done” or “not well done”) Must require sufficient, specific knowledge, skill and

attitude, generally acquired through training Should reflect competencies, important to be acquired Usually confined to qualified personnel only

Competencies versus EPAs

person-descriptors

knowledge, skills, attitudes, values

• content expertise• collaboration ability• communication ability• management ability• professional attitude• scholarly approach

work-descriptors

essential parts of professional practice

• discharge patient• counsel patient• lead family meeting• design treatment plan• perform paracentesis• resuscitate if needed

Competencies EPAs

Competencies and EPAs combined

Patie

nt C

are

Med

ical

Kn

ow

led

ge

PB

LI

Com

mu

ni-c

atio

n

Pro

fessio

-nalis

m

Syste

m-B

ased

P

ractic

e

Develop and implement a safe discharge plan for a patient from the acute care setting

xxx x xx xxx x xx

Lead a family meeting to discuss serious news with patient/family and other health providers

xxx x x xxx xx x

ExerciseComplete Worksheet A on your own

List good examples of essential EPAs your trainees should be able to do independently at the end of training.

List bad examples of EPAs

Pair Share: share your examples with one person next to you.

Recommended full EPA description

1. EPA Title (max 20 words, avoid skill and avoid adjectives)

2. Description of the activity (to serve universal clarity, include limitations)

3. Expected KSA (to serve trainee)

4. Link with competencies and predefined milestones (to embed within the existing framework)

5. Sources of information to determine progress (to serve observation and assessment)

6. Basis for formal entrustment decision (who will have a say in the decision -- signatures if formal and documented)

7. Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee?)

Example from PediatricsOne of the 16 draft EPAs developed by AAMC and ABP team for pediatric residency training:

1.EPA Title: Manage patients with acute, common single system diagnoses in an ambulatory, emergency, or inpatient setting

2.Description: Merged with expected KSA and clarified in descriptive vignettes

Example EPA cont’d3. Expected KSA:

Gathering info thru hx, PE, and initial labs Sound clinical reasoning driving development of DDx

to allow proper diagnostic testing and initial therapy Knowledge of evidence related to primary problem Application of evidence to management plan Patient and family-centered care with bidirectional

communications Documentation of plan and reasoning that is

transparent to other members of health care team

Example EPA cont’d4. Link with competencies/milestones:

Patient care Perform complete and accurate PE Make informed diagnostic and therapeutic

decisions that result in optimal clinical judgment Develop and carry out management plan

Medical knowledge Interpersonal and communication skills

Communicate effectively with patients, families, and the public as appropriate, across a broad range of socioeconomic and cultural backgrounds

Maintain comprehensive, timely, and legible medical records

PE Sub-competencyPerforms essentially the same rote head-to-toe physical examination of the patient regardless of presenting complaint; does not use diagnostic hypotheses from the history to anticipate or look for specific positive or negative findings on physical examination.

With a broad list of diagnostic hypotheses after the history, uses a head-to-toe approach to the physical examination to anticipate and look for a myriad of potential positive and negative physical examination findings for multiple diagnostic considerations. This approach can lead to failure to identify pertinent and important physical findings that are present, misinterpretation of physical findings, and attribution of importance and meaning to irrelevant findings.

Uses a narrow list of diagnostic hypotheses generated through the history to anticipate and look for specific positive or negative physical examination findings of only the most relevant diagnostic considerations; open to new diagnostic possibilities in the process of performing a survey physical examination to elicit unexpected abnormalities but may dismiss these as unimportant when it is difficult to integrate these findings into the working differential diagnosis.

Uses a narrow list of diagnostic hypotheses generated through the history as well as through extensive clinical experience to anticipate and look for key specific physical examination findings that will discriminate between competing similar diagnoses; uses surprises that result from a survey physical examination to rethink and retest diagnostic hypotheses; actively looks for physical exam findings that disconfirm the working diagnosis or rule in or out rare but high-risk alternative diagnoses.

DescriptionBehaviors of early, more advanced, competent,

proficient, and expert learnerExpected elements from milestones

Vignette – 2 yo with wheezing, resp distressReassurance provided during hxHas FB aspiration on differential so focuses on

differential BS in addition to wheezing and WOBPresents focused hx/PE with reasoned assess/plan

consistent with family’s wishes and health literacy, including SW referral for loss of insurance

Small Group Activity: Design an EPA

Work in small groups

Use resources provided Internal medicine milestonesSOM milestones

Develop an EPA for Internal medicine

ORMedical students

Small Group Debrief

When is “competence” reached?

When you trust the traineeWhen a professional activity is mastered• on a threshold level• that permits unsupervised practice• and full entrustment

It happens all the time: when trainees work without direct supervision

Level of supervision

Level 1: not allowed to practice the EPA

Level 2: practice with full supervision

Level 3: practice with supervision on demand

Level 4: “unsupervised” practice allowed

Level 5: supervision task may be given

Recommended full EPA description

1. EPA Title (max 20 words, avoid skill and avoid adjectives)

2. Description of the activity (to serve universal clarity, include limitations)

3. Expected KSA (to serve trainee)

4. Link with competencies and predefined milestones (to embed within the existing framework)

5. Sources of information to determine progress (to serve observation and assessment)

6. Basis for formal entrustment decision (who will have a say in the decision -- signatures if formal and documented)

7. Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee?)

Dreyfus and Dreyfus Model

• Novice – Don’t know what they don’t know

• Advanced Beginner – Know what they don’t know

• Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth

• Proficient – In depth knowledge concerning the discipline – often rule based – know what they know

• Expert – Expert thrives with situations that break the rules – who the proficient practitioners go to for help

training deliberate professional practice

Dreyfus & Dreyfus, 1986

proficient

expert

competent

advancednovice

Development of Competence

training deliberate professional practice

EPA1

EPA4

EPA2

EPA3

EPA5

Competence

Threshold

Justified entrustment decisions

Sample Competency Curve

Small Group Activity:Choosing Evidence

What evidence will inform the EPA that you developed

Small Group Reporting

Wrap Up