Alliance webinar 3 2010 1014 final draftmedia01.commpartners.com/acme_eo2_docs/Session_III_-_3...10/13/2010 1 Welcome to today’s webinar! What member section do you belong to? Health

10/13/2010

1

Welcome to today’s webinar!What member section do you belong to?

Health Care Education AssociationFederal Health Care Educator/Government

Hospital/Health SystemMedical Education and Communication Company

Medical SchoolMedical Specialty Society

State Medical SocietyPharmaceutical

Other

Welcome to today’s webinar!How many people are participating in this

webinar at your location today?

Just me!2345678910

More than 10!

Planning CME Activities to Achieve Desired Outcomes

Session III: Designing a CME Activity

Thursday, October 14, 20102:00 – 3:00 PM ET

www.acme-assn.org

10/13/2010

2

Presenters

Don Moore, PhDDirector

Division of CMEVanderbilt University

Harry A. Gallis, MDConsulting Professor of

Medicine Internal Medicine/ID

Duke University

Learning Objectives for the Webinar Series

After participating in this series of webinars, you should be able to describe and discuss:

1.An outcomes-based approach to planning and assessing CME ti iti th t t ib t t hi i d i d tactivities that contributes to achieving desired outcomes.

2.A collaborative framework that facilitates and supports an outcomes-based approach to planning and assessing CME activities. 3.Criteria for selecting instructional techniques appropriate for achieving desired outcomes.4.A preliminary action plan for implementing an outcomes-based approach in your CME program.

Learning objectives for this session…

After participating in today's webinar, you should be able to describe and discuss:1.Planning learning activities by starting with then end in mind.2.Selecting learning strategies that will facilitate the accomplishment of desired outcomes

a. Predisposingb. Enablingc. Reinforcing

3.Link instructional design with assessment4.A preliminary action plan for designing an effective CME activity.

10/13/2010

3

Today’s Agenda…

• A conversation about designing a CME activity to achieve desired outcomes.– CME Director/Planner and Course DirectorCME Director/Planner and Course Director– CME Director/Planner and CME Faculty

Excelsior Health System

• An alarming increase in the number of deaths for patients with pneumonia admitted to facilities of Excelsior Heath System.

• The CEO asked Dr. Carey, an Infectious Disease expert and course director for a very “successful” CME course, to do some education around the issue.

Excelsior Health System

• Mr. Johnson, the Director of CEHP, wants to do more than the usual CME, and has– Convinced Dr. Carey to do a different kind ofConvinced Dr. Carey to do a different kind of

CME– Persuaded the speaker, Dr. Gallis, to move

away from lecturing and try active learning– Enlisted the assistance of the Quality and

Patient Safety organization to provide relevant data

10/13/2010

4

Dr Gallis’ Flyer from Last Year

After Conference Call with Dr. CareyArrived at by mutual agreement with faculty

The ‘Gap’

The DesiredResult

Excelsior Health SystemNeeds assessment question

Result of needs assessment

Next step

Is there a gap between current and desired patient health status…

Yes, mortality rate for pneumonia exceeds target.

Look for the cause of the gap in performance

Is there a gap between current and desired performance…

Yes, physicians are not practicing to pneumonia guideline when indicated.

Look for the cause of the gap in competence

p g

Is there a gap between current and desired competence…

Yes, when presented with patients with pneumonia in scenarios, physicians did not always follow guideline.

Look for the cause of the gap in procedural knowledge

Is there a gap between current and desired procedural knowledge…

No, physicians passed test containing cases and multiple choice questions.

Plan a CME activity to address the gap(s) in competence

10/13/2010

5

Linking Assessment and Planning

Planning• Predisposing activities• Enabling activities

– Presentation

Assessment• Needs assessment• Formative assessment• Summative assessment

– Example– Practice– Feedback

• Reinforcing activities


Needs Assessment

Summative Assessment

Predisposing Activities

Enabling Activities

Formative Assessment

Reinforcing Activities


Needs Assessment



Enabling Activities



10/13/2010

6

Needs Assessment:Gap analysis – patient health

GAP

current mortality

rate

target mortality

rate

Needs Assessment:Gap analysis – performance

Empiric antibiotic selection -hospitalsGuideline

GAP

USRegionHosp AHosp BExcelsior

current level

desired level


GAP

Empiric antibiotic selection -physiciansGuideline

current level

desired level

GuidelineUSRegionDoctor ADoctor BDoctor C

10/13/2010

7


• Problem – not following guideline when indicated

• MeasuresInfluenza pneumonia

GAP – Influenza, pneumonia

vaccination– Empiric antibiotic

selection– Chest radiograph– Assessment of co-morbid

conditions– Smoking assessment and

cessation intervention

current level

desired level


Influenza, pneumonia vaccination - hospitalsGuideline

GAP


current level

desired level


GAP

Influenza, pneumonia vaccination - physiciansGuideline

current level

desired level


10/13/2010

8



GAP


current level

desired level


GAP


current level

desired level


Needs Assessment:Gap analysis – competence

• Measures same as performance– Empiric antibiotic

selection

GAP

– Chest radiograph– Assessment of co-morbid


cessation intervention– Influenza, pneumonia

vaccination

• Measurement different

current level

desired level

10/13/2010

9


• Measurement– Standardized patient– Simulation– Scenario with ARS

GAP

Scenario with ARS– Scenario on-line– Self report

• Issues– Numbers– Reliability– Predictive validity

current level

desired level

Needs Assessment:Gap analysis – procedural knowledge

• MDs passed test containing cases and MCQs

• Measures

NO GAP

• Measures– Empiric antibiotic

selection– Chest radiograph– Assessment of co-morbid


cessation intervention– Influenza, pneumonia

vaccination

current level

desired level

Selecting Measures

• We chose a condition common to community and tertiary care hospitals and one amenable to the development of diverse types of educational activities and modalities by all types of CME providers

• There are numerous sources of guidelines, most of which derive from pertinent literature and best practices by experts or specialty groups

• Each of these guidelines includes a set of data that should be collected or “measured” in patients meeting certain diagnostic criteria

10/13/2010

10

Typical Guidelines

• Prophylactic antibiotics in surgery• “Door to balloon” times for patients in

ED’s with acute MIED s with acute MI• Hospitalized patients with community

acquired pneumonia• Diabetes, hypertension, asthma, etc

Sources of MeasuresHospital Comparison - CMS

• Geographic area• Conditions• Data

• Medical Conditions– Heart Attack– Heart Failure– Chronic Lung Disease– Pneumonia

– Outcome measures– Process measures– Patient satisfaction

• http://www.hospitalcompare.hhs.gov

Pneumonia– Diabetes in Adults– Chest Pain

• Surgical Conditions– Heart and Blood Vessels– Abdomen– Neck, Back, Arms, Legs– Bladder, Kidney, Prostate– Female Reproductive

Sources of MeasuresAHRQ – National Guideline Clearinghouse

• Public resource for evidence-based clinical practice guidelines.

• Contents– Diseases (2379)– Mental Disorders (266)– Treatments

• Chemicals and Drugs (1600)• Analytical, Diagnostic and Therapeutic Techniques and Equipment (2316)• Behavioral Disciplines and Activities (486)

• http://www.guideline.gov/

10/13/2010

11

Sources of MeasuresPhysician Consortium (PCPI)

• Physicians Consortium for Performance Improvement– convened by the AMA– representatives from

• CMS (Centers for Medicare and Medicaid Services)• NCQA (National Committee for Quality Assurance)• NCQA (National Committee for Quality Assurance)• AHRQ • numerous other professional and quality based organizations

• This web site contains documents on 266 measures and from that site we have selected Community Acquired Pneumonia as an example.

• http://www.ama-assn.org/ama/pub/physician-resources/clinical-practice-improvement/clinical-quality/physician-consortium-performance-improvement.shtml

Sources of MeasuresPhysician Consortium (PCPI)

Sources of Measures - PQRI• Physician Quality Reporting Initiative

– Pay for performance (P4P) – Physician quality reporting system, including an

incentive payment for eligible professionals (EPs)– Data to be reported on quality measures for covered

professional services furnished to Medicare beneficiaries.

• http://www.cms.gov/PQRI/15_MeasuresCodes.asp#TopOfPage

10/13/2010

12

PQRIQuality Measures for CAP

Ongoing Professional Practice Evaluation (OPPE)

• Joint Commission accredited organizations• Data on performance for all practitioners with

privileges on an ongoing basis rather than at the two year reappointment process.

• The type of data to be collected would need to be defined by individual medical staff departments and approved by the organized medical staff.

• The standards require an evaluation for all practitioners not just those with performance issues.

Measures fromProfessional Literature

10/13/2010

13

Sources forProfessional Practice Gaps

• We have chosen this example because hospitals have specific data collected on patients with this disease entity and these data may be used for documentation of professional practice gaps for h i i t ti b l t h it li dphysicians treating ambulatory or hospitalized

patients or emergency physicians making decisions about admission

• These gaps are tied to the performance measures in the set from the PCPI and are derived collaboratively from guidelines developed by the Infectious Diseases Society of America and the American Thoracic Society

Patient Data fromHealth System

• These data can be abstracted from direct review of patient records in collaboration with a hospital’s quality function or from publically reported information on the CMS website

• In the example that we are using we have chosen examples of possible gaps and have used agreed upon measures as our data set

• We have asked our faculty member to focus on these specific “gaps” and have chosen them as our target from improved performance and patient outcomes

• This is the “end” that we have “in mind”

Possible PerformanceMeasures in CAP

• Chest radiograph• Assessment of co-morbid conditions• Vital signs, oxygen saturation

M t l t t h d ti t t• Mental status, hydration status• Level of care rationale (home, ICU, etc)• Empiric antibiotic selection• Smoking assessment, smoking cessation

intervention• Influenza, pneumonia vaccination

10/13/2010

14

Developing Data Set

• The PCPI measures contain a flow sheet that could be adapted to patient charts.

• Here is an example of the data that should be collected and this should become a part of the CME activity.

Measures for Community-Acquired Pneumonia

• Time of arrival, time of first administration of antibiotic

• Does antibiotic derive from guideline

• Co-Morbid Conditions– Malignancy– Liver Disease– Congestive heart failure– COPD/Asthmafrom guideline

recommendations?• Full vital signs recorded• Assessment for co-

morbid conditions (other disease entities that predispose to higher mortality rates)

COPD/Asthma– Renal disease– Cerebrovascular disease– Diabetes

Measures for Community-Acquired Pneumonia, cont.

• Mental status (altered mental status associated with higher mortality and more dangerous organisms such as Legionella)

• Hydration status – volume depletion, shock associated with adverse outcomes

• O2 saturation – may indicate need for ICU• Chest X-Ray and description of radiographic findings• Immunization, smoking status• Level of care determination – home, admit, ICU

10/13/2010

15

Planning Decision-Making• Physicians seem to have procedural

knowledge.• Issue appears to be at the competence

level.level.• Decision - Plan CME activity to develop

competence.• How to plan a CME to achieve desired

outcomes at the competence level?


Needs Assessment



Enabling Activities



Predisposing CME Activities• “Predisposing” – Creates an inclination to do

something.• Predisposing CME activity

– Creates a teachable moment and/or– Helps a physician recognize a teachable moment that

already exists.• Content of predisposing CME Activities

– Data about current performance– Information about best practice– Comparison of current performance and best

practice.

10/13/2010

16

Predisposing CME Activities

• Presentation of national data• Presentation of local and blinded

physician specific dataphysician specific data• Selected recent local cases as

illustrations

Predisposing CME ActivityScores on Competence Assessment

MD A MD B MD C MD D MD E MD F

91-100%81-90%71-80%61-70%51-60%41-50%31-40%21-30%11-20%1-10%0


• Measurement– Standardized patient– Simulation– Scenario with ARS

GAP

Scenario with ARS– Scenario on-line– Self report

current level

desired level

10/13/2010

17



GAP


current level

desired level


GAP


current level

desired level




GAP


current level

desired level

10/13/2010

18


GAP


current level

desired level


Linking Planning and Assessment (Bob Fox)

HealthStatus

HealthSt tStatus

Intervention

CompetenceLearning

Other Explanations

Performance

Status

CompetenceLearning

Other Explanations

Performance

Predisposing CME Activities• “Predisposing” – Creates an inclination to do

something.• Predisposing CME activity

– Creates a teachable moment and/or– Helps a physician recognize a teachable moment

that already exists.• Content of predisposing CME Activities

– Data about current performance– Information about best practice– Comparison of current performance and best

practice.

10/13/2010

19


Needs Assessment



Enabling Activities



Enabling CME Activities• Enabling CME activities

– Provides opportunities for learners to develop competence.

– Examples of Enabling CME activitiesP t ti d t il d t b t d i ti f b t ti• Presentation: detailed step-by-step description of best practice

• Example: how best practice occurs in a specific practice setting– Case presentation– Skill demonstration

• Practice: opportunity to try out best practice– Authentic case discussion with ARS– Skills or animal lab

• Feedback: reinforcement or constructive criticism

Enabling CME Activities• I hear and I forget.• I see and I remember.• I do and I understand.

ConfuciusChinese philosopher & reformer

(551 BC - 479 BC)

10/13/2010

20

Enabling CME Activities - 1• Presentation – describe and explain

guideline.• Example – demonstrate use of flow sheet in

several case presentationsSeverity of illness and site of care– Severity of illness and site of care

– Antibiotic selection– Offering of smoking cessation counseling

• Practice - small group work – develop flow sheets for use in practice

• Feedback – report out – comments by faculty and other participants.

Enabling CME Activities - 2• Presentation – describe and explain guideline

and flow sheet.• Example – demonstrate use of flow sheet in

several case presentationsSeverity of illness and site of care– Severity of illness and site of care

– Antibiotic selection– Offering of smoking cessation counseling

• Practice – standardized patients.• Feedback – report out – comments by faculty

and other participants.

10/13/2010

21

Enabling CME Activities - 3• Presentation – describe and explain

guideline.• Example – demonstrate use of flow sheet in

several case presentations– Severity of illness and site of carey– Antibiotic selection– Offering of smoking cessation counseling

• Practice – scenarios with ARS at key decision points.

• Feedback – report out – comments by faculty and other participants.

Your plane will be flown by pilots who have been exposed to the principles of flight and the procedures

used for successful take offs and landings . . .


Needs Assessment



Enabling Activities



10/13/2010

22


• Monitoring the progress of learners toward accomplishing learning goals.

progress GAP

• During the educational activity, insuring correct performance – practice and feedback.

• After the educational activity, monitoring performance and providing feedback.

65% 75% 100%


Needs Assessment



Enabling Activities



Reinforcing CME Activities

• Reinforcing CME activities– To reinforce is to strengthen by adding extra support or

additional material– Strengthens cognitive imprint and facilitates incorporation of

new knowledge into the way of doing things• Examples of Reinforcing Activities

– Reminders– Commitment to Change– Practice Portfolios– Post-course Materials– Peers and Opinion Leaders– Communities of Practice

10/13/2010

23


• Sustained follow-up of initiative– Derivative educational materials – pocket cards,

clinical pharmacist consultationF ll t ti i t ff f– Follow up presentations in staff conferences, e.g., series presentations and group discussion in grand rounds or departmental meetings

• Non-educational strategies – reminders– Flow sheet on chart– EHR decision support: prompt


Needs Assessment



Enabling Activities




• Determining if goals and objectives have been achieved.

• What is acceptable? after CME

82% improvement GAP

p– Accept?– Continue CME?– Look at other factors.

• Share results.

improvement GAP

65% 100%

10/13/2010

24


• It’s not over until it’s over.

10/13/2010

25

Learning objectives for this session…

After participating in today's webinar, you should be able to describe and discuss:1.Planning learning activities by starting with then end in mind.2.Selecting learning strategies that will facilitate the accomplishment of desired outcomes

a. Predisposingb. Enablingc. Reinforcing

3.Link instructional design with assessment4.A preliminary action plan for designing an effective CME activity.

Framework for Planning and Assessing Learning Activities

CME Logic ModelINPUTS OUTPUTS OUTCOMES

Resources Activities Outputs Short-term Outcomes

Intermediate Outcomes

Long Term Outcomes Impact

If then, if then, if then, if then, if then, if then

Funding Planning CME activity Learners can state

Learners can demonstrate

Learners actually do

Health status of

Faculty

Content experts

Instructional designers

Evaluation specialists

Develop content

Design learning

Design assessment

Recruitment

Learners attend (Level 1)

Learners are satisfied (Level 2)

what to do(Level 3a)

Learners can describe how to do what they have learned(Level 3b)

in the educational setting that they can do what they have learned (Level 4)

what they learned in their practices (Level 5)

patients is improved (Level 6)

Health status of community is improved (Level 7)

10/13/2010

26

Thanks!

Questions

We welcome your questions and comments!

Please use the chat box on the left side of your screen to submit a question.

Thank you!

Please take a moment to complete the b i f l tibrief evaluation.

We appreciate your feedback!

10/13/2010

27

Thank you!

Thank you for your participation!

www.acme-assn.org

Documents

Alliance webinar 3 2010 1014 final draftmedia01.commpartners.com/acme_eo2_docs/Session_III_-_3...10/13/2010 1 Welcome to today’s webinar! What member section do you belong to? Health