Advancing and Assessing Medical Professionalism

M. BROWNELL ANDERSON

PETER J. KATSUFRAKIS, MD, MBA

A MERICA N UNIVERSITY OF BEIRUT

JA NUARY, 2 014

Advancing and Assessing Medical Professionalism

Disclosure

American University of Beirut January 2014

Peter Katsufrakis has no financial relationships and no conflicts of interest relevant to this presentation.

M. Brownell Anderson has no financial relationships and no conflicts of interest relevant to this presentation.


Most important (in ABIM Charter)

What is different or overlap with AUB Charter

Behaviors that are most amenable to observation

Objectives


1. Review the ―Professionalism Charter‖ created in March 2013 and the ABIM Physician Charter

2. Evaluate appreciative inquiry as an approach to consider promoting medical professionalism at AUB.

3. Identify potential approaches to assess professionalism that align with curricular goals and objectives

4. Enjoy!

A Definition of Professionalism


Clinical Competence (Knowledge of Medicine)

Communication Skills

Ethical and Legal Understanding

EX

CE

LL

EN

CE

AL

TR

UIS

M

HU

MA

NIS

M

AC

CO

UN

TA

BIL

ITY

PROFESSIONALISM

Stern, et al. ―Measuring Medical Professionalism‖ 2006

American Board of Internal Medicine Physician Charter



THREE WORDS THAT DESCRIBE CHANGE

Basics of Appreciative Inquiry


Assumption:

Every system has something that works right

Methods

Asset, forward based qualitative method (semi-structured interview)

Focus on success; explore in depth key elements

Analysis Identification of: Themes

Strategies for overcoming barriers

Basics of Appreciative Inquiry Used with/for:


Indiana U – 2004 JGIM - Professionalism Used an organizational change methodology known as appreciative

inquiry, which focuses attention on existing capabilities and successful experiences as a foundation for creating more of what is desired

Univ of Washington & UMKC - Professionalism UW – Strategy for enhancing an institutional culture of professionalism

(2007 Acad Med) UMKC – Narrative storytelling as variant of AI to identify principles

contained in definitions of professionalism to deepen understanding (2010 Acad Med)

Learning in Interprofessional Teams Med Teacher/ AMEE Guide #28 (2009)

Faculty Development at: Emory, Indiana, Rochester, Baylor, Minnesota Core Curriculum session to enhance reflective learning - – Acad Med

2009

Directions for Exercise One


Step back… Think of recent experience (last 3-4 months) related to medical student curriculum Pick a time when you have felt most engaged, alive, absorbed,

excited, proud.

Everyone have an experience?

A learning experience with or between students and faculty

(Re) Affirmation of your role as a future physician, teacher, learner, educator, faculty member.

Occurred in any setting/context – in/out of class, as part of core/non-core pathway time, in a hallway/e-mail, in a clinic or the library, Simulation Center, Starbucks.

AI Success Worksheet Write it Down


Worksheet (continued)


Describe the experience in sufficient detail

What did it feel like?

How does it connect to something you value?

Who, What Where, When were you involved?

What did you and/or others do to contribute to this success?

What did you learn that might apply to other aspects of educational programs?

Is there an analogy, image, or metaphor that captures what you learned?

Worksheet #2- Directions


Tell Your Story to Your Group Record – key words/phrases

Debrief Group & Record Common Features/Themes on success

Identify someone in the group to give a 1 minute report of KEY THEMES

STRATEGIES TO BUILD ON? OVERCOME OBSTACLES?


Cross-Cutting Themes

DOES IT MATCH OUR THEMES?

EXPAND/ENRICH OUR UNDERSTANDING?

WHAT HAVE WE ADDED?

What does the Literature Tells Us?

What Does the Literature Tell Us?


50% of all organizational changes fail Beer, M, & Nohria, N. (Eds.). (2000). Breaking the code of change. Cambridge, MA: Harvard Business School.

Failure can:

Happen early failure of buy in/leadership group

Happen late, when success is not sustained failure to live up to agreements

Peer pressure

Conflicting priorities

So, how can change succeed?

Works on Successful Change: Identifying Key Characteristics


Herb Shepard: 8 Rules of Thumb for Change (1975)

John Kotter: 8 Steps of Change (1996)

Kerry Patterson, et al: Influencer (2008)

Six sources of influence

John D. Adams – review of various ―models:

8 + 4 themes for successful change

Herb Shephard‘s Rules of Thumb for Change Agents


Rule I: Stay alive This rule is a double entendre—

Herb advised us both to avoid ―self-sacrifice‖ and to be ―fully alive‖ in our work.

Rule II: Start where the system is

Rule III: Never work uphill Corollary 1: Don‘t build hills as

you go Corollary 2: Work in the most

promising area Corollary 3: Build resources Corollary 4: Don‘t over-organize Corollary 5: Don‘t argue if you

can‘t win

Rule IV: Innovation requires a good idea, initiative, and a few friends

Rule V: Load experiments for success

Rule VI: Light many fires

Rule VII: Keep an optimistic bias

Shepard, H. A. (1975). Rules of thumb for change agents. OD Practitioner 7(3), 1-5.

Kotter, J. P. (1996). Leading change. Cambridge, MA: Harvard Business School Press.


1. Establishing a sense of urgency

2. Creating a guiding coalition

3. Developing a vision and strategy

4. Communicating the change vision

5. Empowering employees for broad-based action

6. Generating short term wins

7. Consolidating gains and producing more change

8. Anchoring new approaches in the culture

Kerry Patterson, et al. The Influencer 6 Sources of Influence


Motivation Ability

Personal Make the Undesirable

Desirable

Surpass Your Limits

Social Harness Peer Pressure

Find Strength in Numbers

Structural Design Rewards and Demand

Accountability

Change the Environment

John Adams: 8 Themes for Successful Change

Understand and accept the need for change

Believe change is desirable and possible

Sufficient passionate commitment:

Changing habits (25% of people)

Specific deliverable goal and a few first steps

Structures/mechanisms that require repetitions of the new pattern

Feeling supported and safe

Versatility of mental models

Patience and perseverance


John Adams: 4 Additional Success Factors


Clear accountability

Explicit boundary management

Critical mass in alignment

Rewarding the new behavior and withdrawal of rewards for the old behavior

Theme Yes or N0

Theme

Yes or N0

Understand and accept the need for change

Versatility of mental models

Believe change is desirable and possible

Patience and perseverance

Sufficient passionate

commitment

Clear accountability – visible, vocal, persistent

sponsors and stakeholders

Specific deliverable/goal and a few first steps

Explicit boundary management

Structures/mechanisms that require repetition of new pattern

Critical mass in alignment

Feeling supported and safe Reward new and withdraw rewards for old behaviors.


ASSESSING PROFESSIONALISM:

IMPLEMENTATION


Advancing and Assessing Medical Professionalism-

Day Two

What Does Assessment Do?

Different types of assessments can measure:

Attitudes

Knowledge

Ability

Performance

Ensures minimum competency

Defines aspirational goals

Communicates values

What is important, or not important

Hierarchy of Abilities

Knowledge (knows)

Competence (knows how)

Performance (can do)

Practice (does)

Assessment Methods

Knowledge MCQ, short answer

Competence essay, oral, PMP

Performance simulations, bedside/lab exam

Practice audit, observation

Value Proposition for Assessment

Able to enhance products and services

Communicates values to stakeholders

Provides actionable information

Aligns activities to achieve outcomes

Objective: Improve Clinical Skills

of Students

Pre-1971: Year 3-4 =

discipline-based classes

Year 6 = didactic teaching

Evaluation = clinical bedside exam + multiple choice question (MCQ) exam

Post-1971: Year 3-4 =

organ-system-based

Year 6 = internship

Evaluation = ward assessments + MCQ; clinical bedside exam if unsatisfactory ward assessments

Newble DI, Jaeger K. The effect of assessments and examinations on the learning of medical students. Med Educ. 1983;17(3):165-71.

Few unsatisfactory ward ratings (3/480)

Academic success depended on MCQ exam

Students moved away from wards and into library to prepare

Clinical skills became less important as perceived by students


Outcomes of Curricular Change

Attempt to Remedy the Problem

Added 90 minute Objective Structured Clinical Exam (OSCE) to final examinations

Added 90 minute essay exam on clinical management

Retained ward ratings and MCQ exam

Surveyed students to assess impact of changes


Objective: Improve Clinical Skills of Students via USMLE Step 2 CS exam

Step 2 CS Exam introduced in 2004

Previously, basic skills were never or rarely observed throughout a student‘s medical school career

Approximately 2/3 US medical schools had a high-stakes clinical skills exam

Clerkship directors were surveyed at 109 US medical schools in 2005 about changes in teaching

Objective: Improve Clinical Skills of Students

Educational method

Use of method

increased (%)

Use of method

introduced (%)

Standardized patients 31 14

Direct observation of real patients 24 6

Simulators or synthetic mannequins 16 13

Faculty development programs 11 8

Percentage Change in the Introduction or

Use of Various Educational Methods

(n=88/109 respondents)

Gilliland WR et al. Changes in clinical skills education resulting from the introduction of the USMLE step 2 clinical skills examination. Medical Teacher. 2008;30:325-27.

Lessons About Assessment

Change in assessment method has greater effect than change in curriculum

Assessment methods influence study methods chosen by students

Assessment methods influence faculty behavior

Assessment method measurement characteristics are important

Assessments must have consequences to be effective

Assessing Professionalism Exercise #1 – Focus and Methods


Using Miller‘s Pyramid, what areas might we target when assessing professionalism?

What assessment methods could we use to measure each of these?

(e.g. test of knowledge of ethical principles; reflective writing exercise; observed performance in a clinical setting, etc.)

Assessing Professionalism Exercise #2 – Challenges


In small groups, discuss challenges to assessing professionalism generally and challenges of specific methods previously discussed

Wherever possible, suggest strategies to mitigate or remove the challenges

What must be done?

Who needs to be involved?

What are most likely approaches that will be successful in the context of the institution?

Assessing Professionalism Exercise #3 – Ideas to Action


Individually, identify one particular component of professionalism that is meaningful to you. How might you assess it?

In one sentence, describe the professionalism component

Identify 1-3 methods you might use to assess this aspect of professionalism

Pick one method, outline the tasks necessary that you might accomplish within the next month to implement this assessment

In pairs, describe and discuss your approach(es)

Prepare to report to the larger group

TAKING THE LITERATURE AND THEMES TOGETHER–

HOW DO WE ADVANCE AND ASSESS

PROFESSIONALISM?

Translating ―Themes‖ Into

Action Plans

ACTION PLANS


Identify a current change effort (implementing professionalism charter at AUB)

List the key features of your change strategy that connect to the success themes

What is the ONE thing – based on themes for success – that you can do to transform ―half empty‖ responses to change to positive/forward ―half full‖ perspective:

- 1st day you return to work

- Within one week of return

Brief Reports of Selected Action Plans


Final Comments, Summary & References

• AI ENGAGING METHOD

• Application to change anxiety

• Opportunity to talk about success

• LITERATURE ON “CHANGE”

• APPLICATION TO HOME SETTING?

Using Success to Manage Anxiety→ Moving

forward in same direction


http://executiveeducation.wharton.upenn.edu/wharton-at-work/0810/images/cartoon2b_lg-0810.gif








References & Additional Info (1)

Adams JD. Transforming Leadership. 2nd Edition. Ed. Alexandria, VA: Miles River Press, 1998.

Branch WT, Frankel R, Gracey CF, et al. A Good Clinician and a Caring Person: Longitudinal Faculty Development & the Enhancement of the Human Dimensions of Care. Acad Med 2009;84(1):117-125.

Cooperrider DL, Whitney D. Appreciative Inquiry: A Positive Revolution in Change. San Francisco, CA: Berrett-Koehler Publishers, 2005.

Cottingham AH, Suchman AL, Litzelman DK, et al. Enhancing the informal curriculum of a medical school: a case study in organization culture change. J Gen Intern Med 2008;23(6):715-22.

Fryer-Edwards K, Van Eaton E, Goldstein EA, et al. Overcoming Institutional Challenges through Continuous Professionalism Improvement. Acad Med 2007;82(11):1073-1078

Hammiick M, Olckers L, Campion-Smith C. Learning in interprofessional teams: AMEE Guide #38. Medical Teacher 2009;31(1):1-12.

Kotter,JP. Leading change. Cambridge, MA: Harvard Business School Press 1996.


References & Additional Info (2)

Kouzes JM, Posner BZ. The Leadership Challenge, 4th Edition. San Francisco: Jossey-Bass; 2007.

Kumar LR, Chacko TV. Using appreciative inquiry on learning styles to facilitate student learning. Med Educ 2010; 44(11):1121-22.

Patterson K, et al. The Influencer: The power to change anything. McGraw Hill. New York. 2008.

Quaintance JL, Arnold L, et al. What students learn about professionalism from faculty stories: An ‗appreciative inquiry‘ approach. Acad Med 2010;85(1):118-23.

Shepard, H. A. (1975). Rules of thumb for change agents. OD Practitioner 7(3), 1-5

Simpson D, Lal A, Redlich PR. Sentinel Events as Surgeons – Appreciative Inquiry & Identify Formation. Focus on Surgical Education 2009;26(1):17-20.

Suchman AL, Williamson PR, Litzelman DK, Frankel RM, Mossbarger DL, Inui TS. Relationship-Centered Care Initiative Discovery Team. Toward an informal curriculum that teaches professionalism: Transforming the social environment of a medical school. J Gen Intern Med. 2004;19(5 Pt 2):501-4.

Whitney D, Trosten-Bloom, A. The Power of Appreciative Inquiry: A Practical Guide to Positive Change – 2nd Edition. Berrett-Koehler Publishers, Ind. 2010.


•FINAL COMMENTS

•“PEARLS” FROM PARTICIPANTS


SUMMARY

M A R G A R E T M E A D

A N T H R O P O L O G I S T ( 1 9 0 1 – 1 9 7 8 )


Never doubt that a small group of thoughtful, committed citizens can

change the world. Indeed, it is the only thing that ever has.

SHUKRAN

Documents

Advancing and Assessing Medical Professionalism