Realizing the Promise of CBME for Patients and Professionals€¦ · Realizing the Promise of CBME for Patients and Professionals ... Council for Graduate Medical Education Dreyfus

Accreditation Council for Graduate Medical Education

© 2014 Accreditation Council for Graduate Medical Education

Realizing the Promise of CBME for Patients and Professionals


• Where we are and where we need to go • Why CBME and why now?

• What is needed to realize the “promise”

Outline


“The Journey of a 1,000 Miles Begins with a Single Step”


Where We Are As a Country


AHRQ Quality Report 2013 Measure Focus

Measure Name/Description

Baseline Rate

Most Recent Rate

Aspirational Target

Aspirin Use Outpatient visits at which adults with

cardiovascular disease are

prescribed/maintained on aspirin

47%13 53%14 Increase to 65% by 2017

Blood Pressure Control

Adults with hypertension who have adequately

controlled blood pressure

46%15 53%16 Increase to 65% by 2017

Cholesterol Management

Adults with high cholesterol who have

adequate control

33%17 32%18 Increase to 65% by 2017

Smoking Cessation

Outpatient visits at which current tobacco users received tobacco cessation counseling or cessation medications

23%19 22%20 Increase to 65% by 2017

http://www.ahrq.gov/workingforquality/nqs/nqs2013annlrpt.htm%23t4fn13










Arnie Milstein: ABIMF Forum 2010 “Since physician graduates of American medical education organizations typically lead or heavily influence US health care delivery, one source of indirect, broad, outcome-based evidence [of the effectiveness of the medical education enterprise] is the overall performance of the US health care system. The width of the performance gaps on the aims of effectiveness, safety and efficiency understandably reduces society’s confidence that physicians are adequately honoring their Hippocratic promises.”

Milstein A. Trailing Winds and Personal Risk Tolerance: An External Perspective on the Opportunity for Medical Educators to Fulfill Their Social Contract Permanently.


Who is Raising Concern? • Institute of Medicine (2008)

• Resident Duty Hours: Enhancing Sleep, Supervision, and Safety

• Retooling for an Aging America • Congress (2011-12)

• Reductions in GME funding • Request to IOM to review GME regulation

• MedPAC • June 2009 Report • October 2009 Hearing • June 2010 Report

• IOM Financing/Regulation report pending


Do We Have A

“Design Problem”?


Ghost of Donabedian…

In Medical Education…

Structure X Process ≠ Outcome


Current Model of Training and Practice Retirement

Competent


Training Requirements for Endo

Train in an accredited program: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQPIF/143_endocrinology_diab_metab_int_med_07132013.pdf

24 months total 12 months clinical Rate the six general

competencies and overall competence

Take a test at the end

http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQPIF/143_endocrinology_diab_metab_int_med_07132013.pdf




http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&docid=LiY1gW7M6R6ddM&tbnid=gyf9F6YGpNh97M:&ved=0CAUQjRw&url=http%3A%2F%2Fe3contentstrategy.blogspot.com%2F2010%2F11%2Fcreating-castle-from-content-building.html&ei=M8SZU-qVFcihyATCw4KIBA&bvm=bv.68911936,d.aWw&psig=AFQjCNFnIIq_PzxyAhRlJot7pZ1EoK0b7g&ust=1402672464929984


Certification Requirements for Endo


Endo Certification Exam Blueprint

http://www.abim.org/pdf/blueprint/endo_cert.pdf


What About Patients?


Geriatric Specific Processes of Care Resident Clinics (%) Practicing Physicians (%)

Documentation of: Gait evaluation 28 74**

Balance evaluation 22 66** Medical surrogate 28 54**

End-of-life preferences 29.5 49** Vision testing done 40 65**

Hearing assessment 23 40* Screening for:

Falls risk 19 61** Cognitive impairment 18 52**

Depression 34 63** * p < .01, ** p < .001 Lynn LA, Hess BJ, Conforti LN, Lipner RS, Holmboe ES. The Relationship Between Clinic Systems and Quality of Care for Older Adults in Residency Clinics and in Physician Practices. Acad Med. 2009; 84(12): 1732-1740


Hospital Comparisons on Quality and Resource Use (Higher scores represent better performance)

Non-teaching (N= 997) Teaching (N=186)

Quality Composite Score

Res

ourc

e us

e C

ompo

site

Sco

re

Exemplary Teaching Hospitals

Source: L. Binder, CEO of Leapfrog Group, email communication, March 2010


Evaluating Residency Programs Using Patient Outcomes JAMA 2009;302(12):1277-1283. Asch, DA, et.al.

0

2

4

6

8

10

12

14

Q5 Q4 Q3 Q2 Q1 Q1-Q5

Residency Program of Origin, Ranked (Quintile) by Program Complication Rate

Rate of Major Obstetric Complications by Graduates (%)

∆

Difference remains after correction for

USMLE performance

Excess Risk ∆ 32% Q1 vs Q5


Office-based Practice Competencies • Inter-Professional team skills • Clinical IT Meaningful Use skills • Population management skills • Reflective practice and CQI skills

• Care Coordination • Continuity of Care • Leadership and management skills • Systems thinking • Procedural Skills

Individual Physician Readiness: Gaps

Crosson Health Affairs 2011


The Medical Students “Strike Back”

Average # of physician visits in last six months of life (teaching hospitals in red) From: What Kind of Physician Will You Be? Variation in Health Care and Its Importance for Residency Training Dartmouth Institute for Health Policy & Clinical Practice 2012


How well do you understand the quality and safety of endocrinologic care or related care delivered in your

institution?

Where is Endocrinology today?


“Every system is perfectly designed to achieve the results it gets.”

Paul Batalden


Where We Are Trying to Go

Competency-based Medical Education


Educating Physicians1

• “Can medical education’s illustrious past serve as an adequate guide to a future of excellence? Flexner asserted that scientific inquiry and discovery, not past traditions and practices, should point the way to the future in both medicine and medical education…”

1Cooke M, Irby DM and O’Brien BC. Educating Physicians. 2010.


Educating Physicians • “…Medical training is inflexible, excessively long,

and not learner-centered. We found that clinical education is overly focused on inpatient clinical experience, supervised by clinical faculty who have less and less time to teach and who have ceded much of their teaching responsibilities to residents, and situated in hospitals with marginal capacity to support their teaching mission.”


Carnegie: Reforms Recommended

• Standardize learning outcomes • Individualize learning while allowing flexibility and

the opportunity to progress as learners achieve competency milestones

• Establish rigorous and progressively higher levels of competency across the continuum of medical education

• Develop a coherent framework for the continuum of medical education and establish effective mechanisms to coordinate standards Carnegie Foundation: Acad Med, Vol. 85 (2) 2010


Why CBME: Meet People and System Needs

27

Frenk J. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010


Early Principles: CBME

• World Health Organization (1978): • “The intended output of a competency-

based programme is a health professional who can practise medicine at a defined level of proficiency, in accord with local conditions, to meet local needs.”

McGaghie WC, Miller GE, Sajid AW, Telder TV. Competency-based Curriculum Development in Medical Education. World Health Organization, Switzerland, 1978.


When I say “Milestone”…

…What first comes to mind?


What Are Milestones for GME? • Elements of Resident Performance (Work Place)

• described as important by the literature and/or expert opinion

• Behaviorally Described in a Developmental Framework

• (Describe Aspirational Goals)

• Are a Subset of the elements of the Six Domains of Clinical Competency • designed to be markers of progress

• Are a potential framework for evaluation of progress along the continuum

• A tool to introduce new elements of the Competencies


Entrustable Professional Activities

• EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty

• The concept of “entrustable” means: • ‘‘a practitioner has demonstrated the necessary

knowledge, skills and attitudes to be trusted to perform this activity [unsupervised].’’1

1Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and

clinical practice? Acad Med. 2007; 82(6):542–547.


An Entrustable Professional Activity • Part of essential work for a qualified professional • Requires specific knowledge, skill, attitude • Acquired through training • Leads to recognized output • Observable and measureable, leading to a

conclusion • Reflects the competencies expected

• EPA’s together constitute the core of the profession

ten Cate et al. Acad Med 2007


Milestones and EPAs as Roadmap

Observations:

1) Journey not a straight line

2) More than one path (but not infinite paths)

3) “If you don’t know where you are going, any road will get you there”


Dreyfus & Dreyfus Development Model

Dreyfus SE and Dreyfus HL. 1980 Carraccio CL et al. Acad Med 2008;83:761-7

Time, Practice, Experience

Novice Advanced Beginner

Competent

Proficient

Expert/ Master

MS3 MS4

PGY1 PGY3


Design and Sequencing of Training Activities

Professional teachers and coaches

* Monitor students’ development * Design and select training tasks for individual students

From Anders Ericsson: Used by Permission


Self Assessment Conundrum

Individually generated summary judgment of one’s skill level

Inaccurate Poor performers overestimate Outstanding performers underestimate

Davis D et al. JAMA 2006; 296:1094-1102 Eva KW et al. Acad Med. 2005;80:S46-54


Boud and Molloy (2013)

“Feedback is seen not only as having influence on immediate tasks but of building students’ capability for making judgments about their subsequent work.” • Key concept: Think of feedback as part of

a series of PDSA/Kolb learning cycles

Boud D and Molloy E. Feedback in Higher and Professional Education. Routledge 2013.


What is Needed to Realize the Promise


Competency Based Education

Fixed length, variable outcome

Variable length, defined outcome

Structure/Process •Knowledge acquisition •Single subjective measure •Norm referenced evaluation •Evaluation setting removed •Emphasis on summative

Competency Based •Knowledge application •Multiple objective measures •Criterion referenced •Evaluation setting: DO •Emphasis on formative

Caraccio et al 2002

The Transition to Competency


Clinical Outcomes

Educational Outcomes

Learner Patient

Clinical Microsystem

Training Program

Traditional Medical Education Perspective

“Academic Culture”


Learner Patient

Clinical Microsystem

Training Program

A Needed Change in Perspective

Faculty

Institutional and System Professionalism

Educational Outcomes

Clinical Outcomes


The “Ultimate Outcomes”

• A competent (at a minimum) practitioner aligned with:

CMS Triple Aim

Achieve the Triple Aim


Observation and Safe Patient Care

Importance of appropriate supervision Entrustment

Trainee performance* X Appropriate level of supervision**

Must = Safe, effective patient-centered care

* a function of level of competence in context **a function of attending competence in context

Kogan JR, Conforti LN, Iobst WF, Holmboe ES. Reconceptualizing variable rater assessments as both an educational and clinical care problem. Acad Med. 2014; 89(5):721-7

http://www.ncbi.nlm.nih.gov/pubmed/24667513

http://www.ncbi.nlm.nih.gov/pubmed/24667513


Effective Professional Development: Must Align Curriculum with Assessment

Dreyfus SE and Dreyfus HL. 1980 Carraccio CL et al. Acad Med 2008;83:761-7

Time, Practice, Experience

Novice Advanced Beginner

Competent

Proficient

Expert/ Master

MILESTONES Curriculum

Assessment

Curriculum

Assessment

Curriculum

Assessment

Curriculum

Assessment

Curriculum

Assessment


The Assessment “System”

Assessments within Program:

• Direct observations • Audit and

performance data • Multi-source FB

• Simulation • ITExam

Judgment and Synthesis: Committee

Fellows

Faculty, PDs and others

Milestones and EPAs as Guiding Framework and Blueprint

Accreditation

Unit of Analysis: Program

Certification and Credentialing

Unit of Analysis: Individual

Institution and Program

This is a Human Process


Milestone Journey: Revised Conceptual Model of Rapid Cycle Change

Tomolo A M et al. Qual Saf Health Care 2009;18:217-224


Change and the Grief Reaction

• Denial • Anger • Bargaining

• Depression • Acceptance

“Unfunded mandate”

“I can’t and we shouldn’t”

“We need better tools”

“Evolution, not transformation”

“I need help to do this”

“I can do this – it is an opportunity”


Thank You & Questions

[email protected]

Documents

Realizing the Promise of CBME for Patients and Professionals€¦ · Realizing the Promise of CBME for Patients and Professionals ... Council for Graduate Medical Education Dreyfus