Getting Students to Entrustment on the Graduation EPAs

Robert Englander, MD, MPHUCSFFebruary 25th, 2015

Disclosures

I have no financial Conflicts of Interest to disclose

EPA

• Entrustment refers to the ability to effectively perform a professional activity unsupervised (GME) or without direct supervision (UME)

• Brings trust and supervision into assessment which are intuitive for faculty working with trainees

• Entrustment decisions allow inference about a learner’s competence

Premise: A framework based on Entrustment decisions requires a

foundational quality in all learners

Trustworthiness is Based On:• Ability or level of KSA

• Conscientiousness (Reliability? Accountability?)

• Telling the truth – absence of deception (truthfulness)

• Knowing one’s limits (discernment)

Kennedy et al., Acad Med. 2008;83(10 Suppl):S89–92

Team Activity #1: Table Discussions

• How do you currently determine if your learners are trustworthy:

• Implicitly?• Explicitly

• What have been effective strategies at other places you have been/worked?

• What is one thing UCSF can/should do going forward to make this determination more explicit?

EPA

Professional is a modifier of activities that refers specifically to:

• Area of practice (e.g. specialty)

• Scope of practice (e.g. profession)

• Point in an education-training-practice continuum (e.g. UME-to-GME or GME-to-practice transitions)

EPA

The Activities:• Represent the essential work that

defines a discipline (in aggregate)• Lead to a recognized outcome• Should be independently executable

within a given time frame• Are observable and measurable units

of work in both process and outcome• Require integration of critical

competencies and milestones

EPAs as the framework for assessment in the UME space: The

Core Entrustable Professional Activities for Entering Residency

• Timothy Flynn, Chair

• Stephanie Call

• Carol Carraccio

• Lynn Cleary

• Tracy Fulton

• Maureen Garrity

• Steve Lieberman

• Brenessa Lindeman

• Monica Lypson

• Rebecca Minter

• Jay Rosenfield

• Joe Thomas

• Mark Wilson

Drafting Panel

AAMC Staff:• Carol Aschenbrener• Bob Englander

Rationale• Graduate Medical Education

competencies well established

• Gaps identified between:• Expectations of Program Directors

and the skills of entering residents• What residents do without

supervision and what they have been documented as competent doing without supervision

• International focus on transitions

Project Charge

Develop a clear, concise list of what graduating medical students should be entrusted to do without direct supervision on DAY ONE of residency

Conceptual Framework

Two frameworks considered:

• Competencies

• Entrustable Professional Activities

EPAs Make sense to faculty, trainees, and the

public

Represent the day-to-day work of the professional

Situate competencies and milestones in the clinical context in which we live

Make assessment more practical by clustering the milestones into meaningful activities

Explicitly add the notions of trust and supervision into the assessment equation

Competencies Have been the basis for assessment in

the GME space for a decade

In the aggregate define the “good physician”

Already have a good literature base on assessment in the “traditional” domains (medical knowledge and patient care)

Have established or developing milestones of performance for at least the GME years

Conceptual Frameworks: Benefits

EPAs

Relatively recent introduction in the literature

Little operationalization of the concept worldwide

Original concept designed for the residency-to-practice transition

Competencies

Are abstract

Are granular: not the way we think about or observe learners

Conceptual Frameworks: Disadvantages

CoreEPAs

For Entering Residency

EPAsFor any

Practicing Physician

Expectationsfor the

Medical School Graduate

EPAsFor

Specialties

The Relationships of EPAs, Competencies and Milestones

• Each EPA “mapped” to its critical competencies using a sorting technique to prioritize

• Milestones established for the pre-entrustable and entrustable learner for each competency

• Expected behaviors for the pre-entrustable and entrustable learner delineated based on the milestones

• Vignettes created to illustrate the expected behaviors for the pre-entrustable and entrustable learner

EPA

DOC

DOC

DOC

M1Mx

M1Mx

M1Mx

M1Mx

M1Mx

M1Mx

EPA: Entrustable Professional ActivityDOC: Domain of CompetenceC: CompetencyM: Milestone

C2

C3

C1

C4

C2

C5

M1Mx

M1Mx

M1Mx

M1Mx

M1Mx

M1Mx

Narrative description of an early (novice) learner

Narrative description of a learner at “x” level

Core EPAs for Entering Residency1) Gather a history and perform a physical

examination

2) Prioritize a differential diagnosis following a clinical encounter

3) Recommend and interpret common diagnostic and screening tests

4) Enter and discuss orders/prescriptions

5) Document a clinical encounter in the patient record

6) Provide an oral presentation of a clinical encounter

7) Form Clinical Questions and retrieve evidence to advance patient care

Core EPAs for Entering Residency

8) Give or receive a patient handover to transition care responsibility

9) Collaborate as a member of an interprofessional team

10)Recognize a patient requiring urgent or emergent care, and initiate evaluation and management

11)Obtain informed consent for tests and/or procedures

12)Perform general procedures of a physician

13)Identify system failures and contribute to a culture of safety and improvement

Contents of the Complete Final Document Online (for

Curriculum Developers)• Full details for each EPA (description,

critical competencies, milestones, expected behaviors and vignettes)

• Appendix A: Bulleted list of behaviors for the pre-entrustable and entrustable learner for all EPAs

• Appendix B: List of competencies used in the mapping process

• Appendix C: Grid of EPAs mapped to their critical Competencies

Contents of the Faculty Manual

For each EPA:• Title• Description• List of expected behaviors, narrative of

expected behaviors, and vignette describing the pre-entrustable learner

• List of expected behaviors, narrative of expected behaviors, and vignette describing the entrustable learner

Identifying the Gaps/Impact on Patients between Expected Performance and Actual Performance of Residents on Day 1 of Residency

1=EPA with the Largest Product of the Gap Between Expectations on Day 1 and Residents’ Abilities to Perform without Direct Supervision and the impact on patient safety

5= EPA with the Smallest Product of the Gap Between Expectations on Day 1 and Residents’ Abilities to Perform without Direct Supervision and the impact on patient safety

1 2 3 4 5

Team Activity 2: CEPAER Q Sort Exercise

Sample text here for a transition slide

GQ 4 o 5

Program DirectorsAll or most

1. Gather a history and perform a physical examination

94.9% 88.4%

2. Prioritize a differential diagnosis following a clinical encounter

88.3% 66.2%

3. Recommend and interpret common diagnostic and screening tests

83.9% 66.0%

4. Enter and discuss orders/prescriptions 54.9% 59.7%

5. Document a clinical encounter in the patient record

89.4% 80.2%

6. Provide an oral presentation of a clinical encounter

88.6% 81.2%

7. Form clinical questions and retrieve evidence to advance patient care

87.9% 63.8%

Sample text here for a transition slide

GQ 4 or 5

Program DirectorsAll or most

8. Give or receive a patient handover to transition care responsibility

77.3% 54.8%

9. Collaborate as a member of an inter-professional team

91.9% 78.3%

10. Recognize a patient requiring urgent or emergent care, and initiate evaluation and management

82.9% 55.5%

11. Obtain informed consent for tests and/or procedures

79.9% 49.6%

13. Identify system failures and contribute to a culture of safety and improvement

67.1% 30.7%

Sample text here for a transition slide

GQ 4 or 5

Program Director’sAll or most

12. Perform general procedures of a physician (IV line insertion, Phlebotomy, BVM ventilation, CPR)

24.2%

IV line insertion 39.8%

Phlebotomy 43.6%

Bag-valve-make ventilation (BVM) 67.8%

Cardiopulmonary resuscitation (CPR) 62.9%

Identifying the sequencing of when you could make entrustment decisions in your current curriculum

1=EPA that could be entrusted to most learners earliest in the current UCSF curriculum

5= EPA that could be entrusted to most learners at the latest point in the current UCSF curriculum

1 2 3 4 5

Team Activity 3: CEPAER Q Sort Exercise

Next Steps: Pilot• Goal to study the implementation of EPAs

in four areas:• Curriculum • Assessment• Entrustment (including the UME to GME

handoff)• Faculty Development

• Recruited 10 pilot schools

• To begin design phase with face-to-face meeting of core teams in DC October 2014

Next Steps: Beta Testing

• Much of what we learn will come from Beta-testing in the field

• Partnering with institutions outside the pilot to create a learning community

• Core EPAs listserve coming soon…stay tuned

Questions/Comments