10/13/2010
1
Welcome to today’s webinar!What member section do you belong to?
Health Care Education AssociationFederal Health Care Educator/Government
Hospital/Health SystemMedical Education and Communication Company
Medical SchoolMedical Specialty Society
State Medical SocietyPharmaceutical
Other
Welcome to today’s webinar!How many people are participating in this
webinar at your location today?
Just me!2345678910
More than 10!
Planning CME Activities to Achieve Desired Outcomes
Session III: Designing a CME Activity
Thursday, October 14, 20102:00 – 3:00 PM ET
www.acme-assn.org
10/13/2010
2
Presenters
Don Moore, PhDDirector
Division of CMEVanderbilt University
Harry A. Gallis, MDConsulting Professor of
Medicine Internal Medicine/ID
Duke University
Learning Objectives for the Webinar Series
After participating in this series of webinars, you should be able to describe and discuss:
1.An outcomes-based approach to planning and assessing CME ti iti th t t ib t t hi i d i d tactivities that contributes to achieving desired outcomes.
2.A collaborative framework that facilitates and supports an outcomes-based approach to planning and assessing CME activities. 3.Criteria for selecting instructional techniques appropriate for achieving desired outcomes.4.A preliminary action plan for implementing an outcomes-based approach in your CME program.
Learning objectives for this session…
After participating in today's webinar, you should be able to describe and discuss:1.Planning learning activities by starting with then end in mind.2.Selecting learning strategies that will facilitate the accomplishment of desired outcomes
a. Predisposingb. Enablingc. Reinforcing
3.Link instructional design with assessment4.A preliminary action plan for designing an effective CME activity.
10/13/2010
3
Today’s Agenda…
• A conversation about designing a CME activity to achieve desired outcomes.– CME Director/Planner and Course DirectorCME Director/Planner and Course Director– CME Director/Planner and CME Faculty
Excelsior Health System
• An alarming increase in the number of deaths for patients with pneumonia admitted to facilities of Excelsior Heath System.
• The CEO asked Dr. Carey, an Infectious Disease expert and course director for a very “successful” CME course, to do some education around the issue.
Excelsior Health System
• Mr. Johnson, the Director of CEHP, wants to do more than the usual CME, and has– Convinced Dr. Carey to do a different kind ofConvinced Dr. Carey to do a different kind of
CME– Persuaded the speaker, Dr. Gallis, to move
away from lecturing and try active learning– Enlisted the assistance of the Quality and
Patient Safety organization to provide relevant data
10/13/2010
4
Dr Gallis’ Flyer from Last Year
After Conference Call with Dr. CareyArrived at by mutual agreement with faculty
The ‘Gap’
The DesiredResult
Excelsior Health SystemNeeds assessment question
Result of needs assessment
Next step
Is there a gap between current and desired patient health status…
Yes, mortality rate for pneumonia exceeds target.
Look for the cause of the gap in performance
Is there a gap between current and desired performance…
Yes, physicians are not practicing to pneumonia guideline when indicated.
Look for the cause of the gap in competence
p g
Is there a gap between current and desired competence…
Yes, when presented with patients with pneumonia in scenarios, physicians did not always follow guideline.
Look for the cause of the gap in procedural knowledge
Is there a gap between current and desired procedural knowledge…
No, physicians passed test containing cases and multiple choice questions.
Plan a CME activity to address the gap(s) in competence
10/13/2010
5
Linking Assessment and Planning
Planning• Predisposing activities• Enabling activities
– Presentation
Assessment• Needs assessment• Formative assessment• Summative assessment
– Example– Practice– Feedback
• Reinforcing activities
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
10/13/2010
6
Needs Assessment:Gap analysis – patient health
GAP
current mortality
rate
target mortality
rate
Needs Assessment:Gap analysis – performance
Empiric antibiotic selection -hospitalsGuideline
GAP
USRegionHosp AHosp BExcelsior
current level
desired level
Needs Assessment:Gap analysis – performance
GAP
Empiric antibiotic selection -physiciansGuideline
current level
desired level
GuidelineUSRegionDoctor ADoctor BDoctor C
10/13/2010
7
Needs Assessment:Gap analysis – performance
• Problem – not following guideline when indicated
• MeasuresInfluenza pneumonia
GAP – Influenza, pneumonia
vaccination– Empiric antibiotic
selection– Chest radiograph– Assessment of co-morbid
conditions– Smoking assessment and
cessation intervention
current level
desired level
Needs Assessment:Gap analysis – performance
Influenza, pneumonia vaccination - hospitalsGuideline
GAP
USRegionHosp AHosp BExcelsior
current level
desired level
Needs Assessment:Gap analysis – performance
GAP
Influenza, pneumonia vaccination - physiciansGuideline
current level
desired level
GuidelineUSRegionDoctor ADoctor BDoctor C
10/13/2010
8
Needs Assessment:Gap analysis – performance
Empiric antibiotic selection -hospitalsGuideline
GAP
USRegionHosp AHosp BExcelsior
current level
desired level
Needs Assessment:Gap analysis – performance
GAP
Empiric antibiotic selection -physiciansGuideline
current level
desired level
GuidelineUSRegionDoctor ADoctor BDoctor C
Needs Assessment:Gap analysis – competence
• Measures same as performance– Empiric antibiotic
selection
GAP
– Chest radiograph– Assessment of co-morbid
conditions– Smoking assessment and
cessation intervention– Influenza, pneumonia
vaccination
• Measurement different
current level
desired level
10/13/2010
9
Needs Assessment:Gap analysis – competence
• Measurement– Standardized patient– Simulation– Scenario with ARS
GAP
Scenario with ARS– Scenario on-line– Self report
• Issues– Numbers– Reliability– Predictive validity
current level
desired level
Needs Assessment:Gap analysis – procedural knowledge
• MDs passed test containing cases and MCQs
• Measures
NO GAP
• Measures– Empiric antibiotic
selection– Chest radiograph– Assessment of co-morbid
conditions– Smoking assessment and
cessation intervention– Influenza, pneumonia
vaccination
current level
desired level
Selecting Measures
• We chose a condition common to community and tertiary care hospitals and one amenable to the development of diverse types of educational activities and modalities by all types of CME providers
• There are numerous sources of guidelines, most of which derive from pertinent literature and best practices by experts or specialty groups
• Each of these guidelines includes a set of data that should be collected or “measured” in patients meeting certain diagnostic criteria
10/13/2010
10
Typical Guidelines
• Prophylactic antibiotics in surgery• “Door to balloon” times for patients in
ED’s with acute MIED s with acute MI• Hospitalized patients with community
acquired pneumonia• Diabetes, hypertension, asthma, etc
Sources of MeasuresHospital Comparison - CMS
• Geographic area• Conditions• Data
• Medical Conditions– Heart Attack– Heart Failure– Chronic Lung Disease– Pneumonia
– Outcome measures– Process measures– Patient satisfaction
• http://www.hospitalcompare.hhs.gov
Pneumonia– Diabetes in Adults– Chest Pain
• Surgical Conditions– Heart and Blood Vessels– Abdomen– Neck, Back, Arms, Legs– Bladder, Kidney, Prostate– Female Reproductive
Sources of MeasuresAHRQ – National Guideline Clearinghouse
• Public resource for evidence-based clinical practice guidelines.
• Contents– Diseases (2379)– Mental Disorders (266)– Treatments
• Chemicals and Drugs (1600)• Analytical, Diagnostic and Therapeutic Techniques and Equipment (2316)• Behavioral Disciplines and Activities (486)
• http://www.guideline.gov/
10/13/2010
11
Sources of MeasuresPhysician Consortium (PCPI)
• Physicians Consortium for Performance Improvement– convened by the AMA– representatives from
• CMS (Centers for Medicare and Medicaid Services)• NCQA (National Committee for Quality Assurance)• NCQA (National Committee for Quality Assurance)• AHRQ • numerous other professional and quality based organizations
• This web site contains documents on 266 measures and from that site we have selected Community Acquired Pneumonia as an example.
• http://www.ama-assn.org/ama/pub/physician-resources/clinical-practice-improvement/clinical-quality/physician-consortium-performance-improvement.shtml
Sources of MeasuresPhysician Consortium (PCPI)
Sources of Measures - PQRI• Physician Quality Reporting Initiative
– Pay for performance (P4P) – Physician quality reporting system, including an
incentive payment for eligible professionals (EPs)– Data to be reported on quality measures for covered
professional services furnished to Medicare beneficiaries.
• http://www.cms.gov/PQRI/15_MeasuresCodes.asp#TopOfPage
10/13/2010
12
PQRIQuality Measures for CAP
Ongoing Professional Practice Evaluation (OPPE)
• Joint Commission accredited organizations• Data on performance for all practitioners with
privileges on an ongoing basis rather than at the two year reappointment process.
• The type of data to be collected would need to be defined by individual medical staff departments and approved by the organized medical staff.
• The standards require an evaluation for all practitioners not just those with performance issues.
Measures fromProfessional Literature
10/13/2010
13
Sources forProfessional Practice Gaps
• We have chosen this example because hospitals have specific data collected on patients with this disease entity and these data may be used for documentation of professional practice gaps for h i i t ti b l t h it li dphysicians treating ambulatory or hospitalized
patients or emergency physicians making decisions about admission
• These gaps are tied to the performance measures in the set from the PCPI and are derived collaboratively from guidelines developed by the Infectious Diseases Society of America and the American Thoracic Society
Patient Data fromHealth System
• These data can be abstracted from direct review of patient records in collaboration with a hospital’s quality function or from publically reported information on the CMS website
• In the example that we are using we have chosen examples of possible gaps and have used agreed upon measures as our data set
• We have asked our faculty member to focus on these specific “gaps” and have chosen them as our target from improved performance and patient outcomes
• This is the “end” that we have “in mind”
Possible PerformanceMeasures in CAP
• Chest radiograph• Assessment of co-morbid conditions• Vital signs, oxygen saturation
M t l t t h d ti t t• Mental status, hydration status• Level of care rationale (home, ICU, etc)• Empiric antibiotic selection• Smoking assessment, smoking cessation
intervention• Influenza, pneumonia vaccination
10/13/2010
14
Developing Data Set
• The PCPI measures contain a flow sheet that could be adapted to patient charts.
• Here is an example of the data that should be collected and this should become a part of the CME activity.
Measures for Community-Acquired Pneumonia
• Time of arrival, time of first administration of antibiotic
• Does antibiotic derive from guideline
• Co-Morbid Conditions– Malignancy– Liver Disease– Congestive heart failure– COPD/Asthmafrom guideline
recommendations?• Full vital signs recorded• Assessment for co-
morbid conditions (other disease entities that predispose to higher mortality rates)
COPD/Asthma– Renal disease– Cerebrovascular disease– Diabetes
Measures for Community-Acquired Pneumonia, cont.
• Mental status (altered mental status associated with higher mortality and more dangerous organisms such as Legionella)
• Hydration status – volume depletion, shock associated with adverse outcomes
• O2 saturation – may indicate need for ICU• Chest X-Ray and description of radiographic findings• Immunization, smoking status• Level of care determination – home, admit, ICU
10/13/2010
15
Planning Decision-Making• Physicians seem to have procedural
knowledge.• Issue appears to be at the competence
level.level.• Decision - Plan CME activity to develop
competence.• How to plan a CME to achieve desired
outcomes at the competence level?
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
Predisposing CME Activities• “Predisposing” – Creates an inclination to do
something.• Predisposing CME activity
– Creates a teachable moment and/or– Helps a physician recognize a teachable moment that
already exists.• Content of predisposing CME Activities
– Data about current performance– Information about best practice– Comparison of current performance and best
practice.
10/13/2010
16
Predisposing CME Activities
• Presentation of national data• Presentation of local and blinded
physician specific dataphysician specific data• Selected recent local cases as
illustrations
Predisposing CME ActivityScores on Competence Assessment
MD A MD B MD C MD D MD E MD F
91-100%81-90%71-80%61-70%51-60%41-50%31-40%21-30%11-20%1-10%0
Needs Assessment:Gap analysis – competence
• Measurement– Standardized patient– Simulation– Scenario with ARS
GAP
Scenario with ARS– Scenario on-line– Self report
current level
desired level
10/13/2010
17
Needs Assessment:Gap analysis – performance
Empiric antibiotic selection -hospitalsGuideline
GAP
USRegionHosp AHosp BExcelsior
current level
desired level
Needs Assessment:Gap analysis – performance
GAP
Empiric antibiotic selection -physiciansGuideline
current level
desired level
GuidelineUSRegionDoctor ADoctor BDoctor C
Needs Assessment:Gap analysis – performance
Empiric antibiotic selection -hospitalsGuideline
GAP
USRegionHosp AHosp BExcelsior
current level
desired level
10/13/2010
18
Needs Assessment:Gap analysis – performance
GAP
Empiric antibiotic selection -physiciansGuideline
current level
desired level
GuidelineUSRegionDoctor ADoctor BDoctor C
Linking Planning and Assessment (Bob Fox)
HealthStatus
HealthSt tStatus
Intervention
CompetenceLearning
Other Explanations
Performance
Status
CompetenceLearning
Other Explanations
Performance
Predisposing CME Activities• “Predisposing” – Creates an inclination to do
something.• Predisposing CME activity
– Creates a teachable moment and/or– Helps a physician recognize a teachable moment
that already exists.• Content of predisposing CME Activities
– Data about current performance– Information about best practice– Comparison of current performance and best
practice.
10/13/2010
19
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
Enabling CME Activities• Enabling CME activities
– Provides opportunities for learners to develop competence.
– Examples of Enabling CME activitiesP t ti d t il d t b t d i ti f b t ti• Presentation: detailed step-by-step description of best practice
• Example: how best practice occurs in a specific practice setting– Case presentation– Skill demonstration
• Practice: opportunity to try out best practice– Authentic case discussion with ARS– Skills or animal lab
• Feedback: reinforcement or constructive criticism
Enabling CME Activities• I hear and I forget.• I see and I remember.• I do and I understand.
ConfuciusChinese philosopher & reformer
(551 BC - 479 BC)
10/13/2010
20
Enabling CME Activities - 1• Presentation – describe and explain
guideline.• Example – demonstrate use of flow sheet in
several case presentationsSeverity of illness and site of care– Severity of illness and site of care
– Antibiotic selection– Offering of smoking cessation counseling
• Practice - small group work – develop flow sheets for use in practice
• Feedback – report out – comments by faculty and other participants.
Enabling CME Activities - 2• Presentation – describe and explain guideline
and flow sheet.• Example – demonstrate use of flow sheet in
several case presentationsSeverity of illness and site of care– Severity of illness and site of care
– Antibiotic selection– Offering of smoking cessation counseling
• Practice – standardized patients.• Feedback – report out – comments by faculty
and other participants.
10/13/2010
21
Enabling CME Activities - 3• Presentation – describe and explain
guideline.• Example – demonstrate use of flow sheet in
several case presentations– Severity of illness and site of carey– Antibiotic selection– Offering of smoking cessation counseling
• Practice – scenarios with ARS at key decision points.
• Feedback – report out – comments by faculty and other participants.
Your plane will be flown by pilots who have been exposed to the principles of flight and the procedures
used for successful take offs and landings . . .
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
10/13/2010
22
Formative Assessment
• Monitoring the progress of learners toward accomplishing learning goals.
progress GAP
• During the educational activity, insuring correct performance – practice and feedback.
• After the educational activity, monitoring performance and providing feedback.
65% 75% 100%
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
Reinforcing CME Activities
• Reinforcing CME activities– To reinforce is to strengthen by adding extra support or
additional material– Strengthens cognitive imprint and facilitates incorporation of
new knowledge into the way of doing things• Examples of Reinforcing Activities
– Reminders– Commitment to Change– Practice Portfolios– Post-course Materials– Peers and Opinion Leaders– Communities of Practice
10/13/2010
23
Reinforcing Activities
• Sustained follow-up of initiative– Derivative educational materials – pocket cards,
clinical pharmacist consultationF ll t ti i t ff f– Follow up presentations in staff conferences, e.g., series presentations and group discussion in grand rounds or departmental meetings
• Non-educational strategies – reminders– Flow sheet on chart– EHR decision support: prompt
Linking Assessment and Planning
Needs Assessment
Summative Assessment
Predisposing Activities
Enabling Activities
Formative Assessment
Reinforcing Activities
Summative Assessment
• Determining if goals and objectives have been achieved.
• What is acceptable? after CME
82% improvement GAP
p– Accept?– Continue CME?– Look at other factors.
• Share results.
improvement GAP
65% 100%
10/13/2010
24
Summative Assessment
• It’s not over until it’s over.
10/13/2010
25
Learning objectives for this session…
After participating in today's webinar, you should be able to describe and discuss:1.Planning learning activities by starting with then end in mind.2.Selecting learning strategies that will facilitate the accomplishment of desired outcomes
a. Predisposingb. Enablingc. Reinforcing
3.Link instructional design with assessment4.A preliminary action plan for designing an effective CME activity.
Framework for Planning and Assessing Learning Activities
CME Logic ModelINPUTS OUTPUTS OUTCOMES
Resources Activities Outputs Short-term Outcomes
Intermediate Outcomes
Long Term Outcomes Impact
If then, if then, if then, if then, if then, if then
Funding Planning CME activity Learners can state
Learners can demonstrate
Learners actually do
Health status of
Faculty
Content experts
Instructional designers
Evaluation specialists
Develop content
Design learning
Design assessment
Recruitment
Learners attend (Level 1)
Learners are satisfied (Level 2)
what to do(Level 3a)
Learners can describe how to do what they have learned(Level 3b)
in the educational setting that they can do what they have learned (Level 4)
what they learned in their practices (Level 5)
patients is improved (Level 6)
Health status of community is improved (Level 7)
10/13/2010
26
Thanks!
Questions
We welcome your questions and comments!
Please use the chat box on the left side of your screen to submit a question.
Thank you!
Please take a moment to complete the b i f l tibrief evaluation.
We appreciate your feedback!
10/13/2010
27
Thank you!
Thank you for your participation!
www.acme-assn.org
Recommended